语言大模型 / LLM

Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphs

Zheng Wang, Zhongyang Li, Zeren Jiang, Dandan Tu, Wei Shi

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments This paper has been accepted by EMNLP 2024

Role-RL: Online Long-Context Processing with Role Reinforcement Learning for Distinct LLMs in Their Optimal Roles

Lewei He, Tianyu Shi, Pengran Huang, Bingzhi Chen, Qianglong Chen, Jiahui Pan

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

Jiajie Zhang, Yushi Bai, Xin Lv, Wanjun Gu, Danqing Liu, Minhao Zou, Shulin Cao, Lei Hou, Yuxiao Dong, Ling Feng, Juanzi Li

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；SFT(abstract)；分类 cs.CL

Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialogues

KuanChao Chu, Yi-Pei Chen, Hideki Nakayama

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted to COLM 2024; camera-ready version

TaskGen: A Task-Based, Memory-Infused Agentic Framework using StrictJSON

John Chong Min Tan, Prince Saroj, Bharat Runwal, Hardik Maheshwari, Brian Lim Yi Sheng, Richard Cottrill, Alankrit Chona, Ambuj Kumar, Mehul Motani

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 53 pages

RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs

Bowen Tan, Yun Zhu, Lijuan Liu, Hongyi Wang, Yonghao Zhuang, Jindong Chen, Eric Xing, Zhiting Hu

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments RedCoast (Redco) has been released under Apache License 2.0 at https://github.com/tanyuqian/redco

Equipping Transformer with Random-Access Reading for Long-Context Understanding

Chenghao Yang, Zi Yang, Nan Hua

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL

Comments Preliminary works for a Google Student Researcher Project

Long Context Alignment with Short Instructions and Synthesized Positions

Wenhao Wu, Yizhong Wang, Yao Fu, Xiang Yue, Dawei Zhu, Sujian Li

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments preview

Holmes: Towards Distributed Training Across Clusters with Heterogeneous NIC Environment

Fei Yang, Shuang Peng, Ning Sun, Fangyu Wang, Yuanyuan Wang, Fu Wu, Jiezhong Qiu, Aimin Pan

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 12 pages

PerOS: Personalized Self-Adapting Operating Systems in the Cloud

Hongyu Hè

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 29 pages, 3 figures

Interactive Continual Learning: Fast and Slow Thinking

Biqing Qi, Xingquan Chen, Junqi Gao, Dong Li, Jianxing Liu, Ligang Wu, Bowen Zhou

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Accepted to CVPR 2024

Towards Full Authorship with AI: Supporting Revision with AI-Generated Views

Jiho Kim, Ray C. Flanagan, Noelle E. Haviland, ZeAi Sun, Souad N. Yakubu, Edom A. Maru, Kenneth C. Arnold

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 15 pages, 2 figures; Accepted to 5th Workshop on Human-AI Co-Creation with Generative Models (HAI-GEN) at ACM IUI 2024

$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens

Xinrong Zhang, Yingfa Chen, Shengding Hu, Zihang Xu, Junhao Chen, Moo Khai Hao, Xu Han, Zhen Leng Thai, Shuo Wang, Zhiyuan Liu, Maosong Sun

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Journal ref 2023.12.15ARR

Bridging the Preference Gap between Retrievers and LLMs

Zixuan Ke, Weize Kong, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

MemGPT: Towards LLMs as Operating Systems

Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Code and data available at https://research.memgpt.ai

Extending LLMs' Context Window with 100 Samples

Yikai Zhang, Junlong Li, Pengfei Liu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；SFT(abstract)；分类 cs.CL

Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memory

Lei Liu, Xiaoyan Yang, Yue Shen, Binbin Hu, Zhiqiang Zhang, Jinjie Gu, Guannan Zhang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Chit-Chat or Deep Talk: Prompt Engineering for Process Mining

Urszula Jessen, Michal Sroka, Dirk Fahland

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 11 pages, 3 figures

Controlled Memory Interference in Continual LLM Agents

持续大型语言模型智能体中的可控记忆干扰

Ao Ding, Hongzong LI, Shiqin Tang, Li Zhang, Liang Chen, Xuyang Chen, Zi Liang

机构 * China University of Geosciences (Beijing)（中国地质大学（北京）） ； Northwestern Polytechnical University（西北工业大学） ； The Hong Kong University of Science and Technology（香港科技大学） ； Chinese Academy of Sciences（中国科学院） ； The Hong Kong Polytechnic University（香港理工大学） ； École polytechnique fédérale de Lausanne（洛桑联邦理工学院） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 长上下文与记忆 ：LLM(title)；分类 cs.AI、cs.LG

AI总结 本研究提出可控记忆干扰（CMI）框架，揭示记忆进化受经验相互作用影响，为持续大型语言模型智能体的记忆系统提供诊断与学习方法，提升记忆更新区分度与任务性能。

Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation

通过反事实评估理解长上下文基础模型中的稀疏注意力选择性

Xingyu Ren, Youran Sun, Chugang Yi, Haizhao Yang

机构 * The Chinese University of Hong Kong（香港中文大学） ； University of Maryland（马里兰大学） 纠错

专题命中 长上下文与记忆 ：foundation model(title)；分类 cs.CL、cs.LG

AI总结 该研究通过反事实评估探究长上下文基础模型的稀疏注意力选择性，证实稀疏化会改变内容影响力，提出开放测量框架，相关结果无法通过聚合准确率检测。

GenoBERT: A Language Model for Accurate Genotype Imputation

GenoBERT：一种用于准确基因型推断的语言模型

Lei Huang, Chuan Qiu, Kuan-Jui Su, Anqi Liu, Yun Gong, Weiqiang Lin, Lindong Jiang, Chen Zhao, Meng Song, Jeffrey Deng, Qing Tian, Zhe Luo, Ping Gong, Hui Shen, Chaoyang Zhang, Hong-Wen Deng

机构 * School of Computing Sciences and Computer Engineering, University of Southern Mississippi（南密西西比大学计算科学与计算机工程学院） ； Tulane Center for Biomedical Informatics and Genomics, Deming Department of Medicine, School of Medicine, Tulane University（杜兰大学医学院德明医学系杜兰生物医学信息学与基因组学中心） ； Kennesaw State University（肯尼索州立大学） ； College of Science, Xi’an Shiyou University（西安石油大学理学院） ； Geisel School of Medicine, Dartmouth College（达特茅斯学院盖泽尔医学院） ； Environmental Laboratory, U.S. Army Engineer Research and Development Center（美国陆军工程兵研究与发展中心环境实验室） 纠错

专题命中 长上下文与记忆 ：language model(title)；分类 cs.AI、cs.LG

AI总结 GenoBERT是一种基于Transformer的无参考框架，通过自注意力机制捕捉短程和长程连锁不平衡依赖，实现高精度基因型推断。

Characterizing State Space Model and Hybrid Language Model Performance with Long Context

通过长上下文分析状态空间模型和混合语言模型的性能

Saptarshi Mitra, Rachid Karami, Haocheng Xu, Sitao Huang, Hyoukjun Kwon

专题命中 长上下文与记忆 ：language model(title)；分类 cs.AI、cs.LG

AI总结 本文研究了长上下文推理中Transformer、状态空间模型（SSM）及混合模型的性能差异，发现SSM在处理长上下文时效率显著高于Transformer，且内存占用更低。

Comments 13 pages, 7 figures

Agentic Unlearning: When LLM Agent Meets Machine Unlearning

代理反学习：当大语言模型代理遇见机器反学习

Bin Wang, Fan Wang, Pingping Wang, Jinyu Cong, Yang Yu, Yilong Yin, Zhongyi Han, Benzheng Wei

机构 * Center for Medical Artificial Intelligence, Shandong University of Traditional Chinese Medicine（山东中医药大学中医人工智能中心） ； School of Software, Shandong University（山东大学软件学院） ； School of Medical Information Engineering, Shandong University of Traditional Chinese Medicine（山东中医药大学医学信息工程学院） ； Shandong Huazhi Talent Technology Co., Ltd.（山东华智人才科技有限公司） 纠错

专题命中 长上下文与记忆 ：LLM(title)；分类 cs.AI、cs.LG

AI总结 本文提出SBU框架，通过同步双更新协议实现参数与记忆路径的联合反学习，有效减少敏感信息痕迹并保持数据保留质量。

Comments 9 pages, 6 figures, 6 tables

GPT-5 vs Other LLMs in Long Short-Context Performance

GPT-5与其他LLM在长短上下文性能中的比较

Nima Esmi, Maryam Nezhad-Moghaddam, Fatemeh Borhani, Asadollah Shahbahrami, Amin Daemdoost, Georgi Gaydadjiev

机构 * Bernoulli Institute RUG（伯努利研究所） ； ISRC, Khazar University（ISRC、哈扎尔大学） ； Department of Computer Engineering University of Guilan（计算机工程系、盖兰大学） ； QCE Department TU Delft（QCE部门、代尔夫特理工大学） 纠错

专题命中 长上下文与记忆 ：large language model(abstract,comments)；language model(abstract,comments)；分类 cs.CL、cs.AI

AI总结 本文比较了GPT-5与其他LLM在长短上下文任务中的性能，发现GPT-5在高输入量下保持高精度，揭示了模型在复杂任务中的实际表现与理论能力之间的差距。

Comments 10 pages, 7 figures. Accepted for publication in the 3rd International Conference on Foundation and Large Language Models (FLLM2025). IEEE. The final version will be available in IEEE Xplore

Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective

Minh Le, Tien Ngoc Luu, An Nguyen The, Thanh-Thien Le, Trang Nguyen, Tung Thanh Nguyen, Linh Ngo Van, Thien Huu Nguyen

专题命中 长上下文与记忆 ：prompting(title)；分类 cs.CL、cs.LG

Comments Oral presentation at AAAI 2025

Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM

Xiaoding Lu, Zongyi Liu, Adian Liusie, Vyas Raina, Vineet Mudupalli, Yuwen Zhang, William Beauchamp

专题命中 长上下文与记忆 ：LLM(title)；分类 cs.CL、cs.AI

Memory in humans and deep language models: Linking hypotheses for model augmentation

Omri Raccah, Phoebe Chen, Ted L. Willke, David Poeppel, Vy A. Vo

专题命中 长上下文与记忆 ：language model(title)；分类 cs.CL、cs.AI

Comments 6 figures

Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection

Minqian Liu, Shiyu Chang, Lifu Huang

专题命中 长上下文与记忆 ：prompting(title)；分类 cs.CL、cs.AI

Comments Accepted to COLING'22 Main Conference (Short paper). 9 pages, 2 figures, 3 tables

Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers

Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, David Belanger, Lucy Colwell, Adrian Weller

专题命中 长上下文与记忆 ：language model(title)；分类 cs.CL、cs.LG

Comments This arXiv submission has been deprecated. Please see "Rethinking Attention with Performers" at arXiv:2009.14794 for the most updated version of the paper

Extending Machine Language Models toward Human-Level Language Understanding

James L. McClelland, Felix Hill, Maja Rudolph, Jason Baldridge, Hinrich Schütze

专题命中 长上下文与记忆 ：language model(title)；分类 cs.CL、cs.AI