语言大模型 / LLM

Advancing Retrieval-Augmented Generation for Structured Enterprise and Internal Data

Chandana Cheerla

机构 * IIT Roorkee（罗尔基大学） 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Rethinking Associative Memory Mechanism in Induction Head

Shuo Wang, Issei Sato

机构 * The University of Tokyo（东京大学） 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments COLM 2025

State and Memory is All You Need for Robust and Reliable AI Agents

Matthew Muhoberac, Atharva Parikh, Nirvi Vakharia, Saniya Virani, Aco Radujevic, Savannah Wood, Meghav Verma, Dimitri Metaxotos, Jeyaraman Soundararajan, Thierry Masquelin, Alexander G. Godfrey, Sean Gardner, Dobrila Rudnicki, Sam Michael, Gaurav Chopra

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 5 Main Figures, 10 Extended Data Figures (37 Pages) for Manuscript ; 9 Supplementary Tables, 40 Supplementary Figures (180 Pages) for Supporting Information

Pairing Analogy-Augmented Generation with Procedural Memory for Procedural Q&A

K Roth, Rushil Gupta, Simon Halle, Bang Liu

机构 * Université de Montréal & Mila（蒙特利尔大学及Mila） ； Thales Canada（加拿大泰雷兹公司） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能 chair） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

TracLLM: A Generic Framework for Attributing Long Context LLMs

Yanting Wang, Wei Zou, Runpeng Geng, Jinyuan Jia

机构 * Pennsylvania State University（宾夕法尼亚州立大学） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments To appear in USENIX Security Symposium 2025. The code and data are at: https://github.com/Wang-Yanting/TracLLM

QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm

Qirui Zhou, Shaohui Peng, Weiqiang Xiong, Haixin Chen, Yuanbo Wen, Haochen Li, Ling Li, Qi Guo, Yongwei Zhao, Ke Gao, Ruizhi Chen, Yanjun Wu, Chen Zhao, Yunji Chen

机构 * SKL of Processors, Institute of Computing Technology, CAS（处理器国家重点实验室，计算技术研究所，中国科学院） ； Intelligent Software Research Center, Institute of Software, CAS（智能软件研究中心，软件研究所，中国科学院） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Refract ICL: Rethinking Example Selection in the Era of Million-Token Models

Arjun R. Akula, Kazuma Hashimoto, Krishna Srinivasan, Aditi Chaudhary, Karthik Raman, Michael Bendersky

机构 * Google DeepMind（谷歌DeepMind） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems

Guibin Zhang, Muxin Fu, Guancheng Wan, Miao Yu, Kun Wang, Shuicheng Yan

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Policy Induction: Predicting Startup Success via Explainable Memory-Augmented In-Context Learning

Xianling Mu, Joseph Ternasky, Fuat Alican, Yigit Ihlamur

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Conversational Exploration of Literature Landscape with LitChat

Mingyu Huang, Shasha Zhou, Yuxuan Chen, Ke Li

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Distance between Relevant Information Pieces Causes Bias in Long-Context LLMs

Runchu Tian, Yanghao Li, Yuepeng Fu, Siyang Deng, Qinyu Luo, Cheng Qian, Shuo Wang, Xin Cong, Zhong Zhang, Yesai Wu, Yankai Lin, Huadong Wang, Xiaojiang Liu

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 Findings

NExtLong: Toward Effective Long-Context Training without Long Documents

Chaochen Gao, Xing Wu, Zijia Lin, Debing Zhang, Songlin Hu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICML 2025. Corresponding authors: xing wu, and songlin hu

SELF: Self-Extend the Context Length With Logistic Growth Function

Phat Thanh Dang, Saahil Thoppay, Wang Yang, Qifan Wang, Vipin Chaudhary, Xiaotian Han

机构 * Case Western Reserve University（凯斯西储大学） ； Meta 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 11 pages, 5 figures, 3 tables

What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices

Zhi Chen, Qiguang Chen, Libo Qin, Qipeng Guo, Haijun Lv, Yicheng Zou, Wanxiang Che, Hang Yan, Kai Chen, Dahua Lin

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Research Center for Social Computing and Interactive Robotics（社会计算与交互机器人研究中心） ； Harbin Institute of Technology（哈尔滨工业大学） ； School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院） 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 Camera Ready. Code is available at: https://github.com/WowCZ/LongMIT

Long Term Memory: The Foundation of AI Self-Evolution

Xun Jiang, Feng Li, Han Zhao, Jiahao Qiu, Jiaying Wang, Jun Shao, Shihao Xu, Shu Zhang, Weiling Chen, Xavier Tang, Yize Chen, Mengyue Wu, Weizhi Ma, Mengdi Wang, Tianqiao Chen

机构 * Tianqiao and Chrissy Chen Institute（田桥和克里斯西·陈研究所） ； Princeton University（普林斯顿大学） ； Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院） ； Shanghai Jiao Tong University（上海交通大学） ； Shanda Group（尚世集团） 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

Comments 56 pages, 13 figures

MARK: Memory Augmented Refinement of Knowledge

Anish Ganguli, Prabal Deb, Debleena Banerjee

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Towards Long Context Hallucination Detection

Siyi Liu, Kishaloy Halder, Zheng Qi, Wei Xiao, Nikolaos Pappas, Phu Mon Htut, Neha Anna John, Yassine Benajiba, Dan Roth

机构 * AWS AI Labs（AWS人工智能实验室） ； University of Pennsylvania（宾夕法尼亚大学） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory

Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, Deshraj Yadav

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformation

Bowen Cao, Deng Cai, Wai Lam

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

ReAttention: Training-Free Infinite Context with Finite Attention Scope

Xiaoran Liu, Ruixiao Li, Qipeng Guo, Zhigeng Liu, Yuerong Song, Kai Lv, Hang Yan, Linlin Li, Qun Liu, Xipeng Qiu

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 11 figures, Accepted by ICLR 2025

AI-native Memory 2.0: Second Me

Jiale Wei, Xiang Ying, Tao Gao, Fangyi Bao, Felix Tao, Jingbo Shang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Shifting Long-Context LLMs Research from Input to Output

Yuhao Wu, Yushi Bai, Zhiqing Hu, Shangqing Tu, Ming Shan Hee, Juanzi Li, Roy Ka-Wei Lee

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Preprint

LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs -- No Silver Bullet for LC or RAG Routing

Kuan Li, Liwen Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Shuai Wang, Minhao Cheng

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 22 pages

DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory

Yutong Wang, Jiali Zeng, Xuebo Liu, Derek F. Wong, Fandong Meng, Jie Zhou, Min Zhang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted as a conference paper at ICLR 2025

Journal ref Published as a conference paper at ICLR 2025

BaKlaVa -- Budgeted Allocation of KV cache for Long-context Inference

Ahmed Burak Gulhan, Krishna Teja Chitty-Venkata, Murali Emani, Mahmut Kandemir, Venkatram Vishwanath

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts

Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval

Ting-Rui Chiang, Dani Yogatama

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

APOLLO: SGD-like Memory, AdamW-level Performance

Hanqing Zhu, Zhenyu Zhang, Wenyan Cong, Xi Liu, Sem Park, Vikas Chandra, Bo Long, David Z. Pan, Zhangyang Wang, Jinwon Lee

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted to MLSys 2025; the newest version with new experiments

Better RAG using Relevant Information Gain

Marc Pickett, Jeremy Hartman, Ayan Kumar Bhowmick, Raquib-ul Alam, Aditya Vempaty

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 4 page paper submitted to EMNLP

GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?

Yang Zhou, Hongyi Liu, Zhuoming Chen, Yuandong Tian, Beidi Chen

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI