语言大模型 / LLM - arXivDaily 专题

2402.17463 2024-05-30 cs.CL 89%

Training-Free Long-Context Scaling of Large Language Models

Chenxin An, Fei Huang, Jun Zhang, Shansan Gong, Xipeng Qiu, Chang Zhou, Lingpeng Kong

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2405.18093 2024-05-29 cs.DC cs.LG 89%

Pipette: Automatic Fine-grained Large Language Model Training Configurator for Real-World Clusters

Jinkyu Yim, Jaeyong Song, Yerim Choi, Jaebeen Lee, Jaewon Jung, Hongsun Jang, Jinho Lee

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.LG

Comments published at DATE 2024

2405.17915 2024-05-29 cs.CL 89%

Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models

Longze Chen, Ziqiang Liu, Wanwei He, Yunshui Li, Run Luo, Min Yang

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 13 pages, 5 figures, ACL 2024

2312.09397 2024-05-09 cs.AI 89%

Personalized Autonomous Driving with Large Language Models: Field Experiments

Can Cui, Zichong Yang, Yupeng Zhou, Yunsheng Ma, Juanwu Lu, Lingxi Li, Yaobin Chen, Jitesh Panchal, Ziran Wang

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

2404.13501 2024-04-23 cs.AI 89%

A Survey on the Memory Mechanism of Large Language Model based Agents

Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, Ji-Rong Wen

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Comments 39 pages, 5 figures, 4 tables

2403.01411 2024-03-05 cs.CL 89%

OVEL: Large Language Model as Memory Manager for Online Video Entity Linking

Haiquan Zhao, Xuwu Wang, Shisong Chen, Zhixu Li, Xin Zheng, Yanghua Xiao

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments 13 pages, 6 figures

2309.16292 2024-02-23 cs.RO cs.CL 89%

DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models

Licheng Wen, Daocheng Fu, Xin Li, Xinyu Cai, Tao Ma, Pinlong Cai, Min Dou, Botian Shi, Liang He, Yu Qiao

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Published as a conference paper at ICLR 2024

2402.13449 2024-02-22 cs.CL 89%

CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory

Zexue He, Leonid Karlinsky, Donghyun Kim, Julian McAuley, Dmitry Krotov, Rogerio Feris

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2402.11573 2024-02-20 cs.CL 89%

BGE Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models

Kun Luo, Zheng Liu, Shitao Xiao, Kang Liu

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2401.07793 2024-01-17 cs.CL 89%

Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization

Ninglu Shao, Shitao Xiao, Zheng Liu, Peitian Zhang

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2312.01954 2023-12-05 cs.CL 89%

Zero- and Few-Shots Knowledge Graph Triplet Extraction with Large Language Models

Andrea Papaluca, Daniel Krefl, Sergio Mendez Rodriguez, Artem Lensky, Hanna Suominen

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2305.13829 2023-10-25 cs.CL 89%

Learning from Mistakes via Cooperative Study Assistant for Large Language Models

Danqing Wang, Lei Li

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023 main conference

2310.03473 2023-10-06 cs.CL 89%

Controllable Multi-document Summarization: Coverage & Coherence Intuitive Policy with Large Language Model Based Rewards

Litton J Kurisinkel, Nancy F chen

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

2212.10947 2023-08-02 cs.CL 89%

Parallel Context Windows for Large Language Models

Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments The 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)

2605.15156 2026-05-21 cs.CL cs.AI cs.LG 89%

MeMo: Memory as a Model

MeMo：记忆作为模型

Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong, Arun Verma, Alok Prakash, Nancy F. Chen, Bryan Kian Hsiang Low, Daniela Rus, Armando Solar-Lezama

机构 * Institute of Data Science, National University of Singapore（数据科学研究院，新加坡国立大学）； Integrative Sciences and Engineering Programme, NUSGS（整合科学与工程计划，NUSGS）； Agency for Science, Technology, Research (A*STAR)（科技研究局（A*STAR））； Department of Computer Science, National University of Singapore（计算机科学系，新加坡国立大学）； University of Tokyo（东京大学）； Liquid AI ； CSAIL, Massachusetts Institute of Technology（CSAIL，麻省理工学院）； AI Singapore ； Singapore-MIT Alliance for Research and Technology Centre, Singapore（新加坡-麻省理工学院研究与技术中心，新加坡）

专题命中长上下文与记忆：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

AI总结本文提出MeMo框架，通过在不改变LLM参数的情况下将新知识编码到专用记忆模型中，解决了大型语言模型在需要及时领域特定信息的应用中的问题，同时具备处理复杂跨文档关系、抗检索噪声、避免灾难性遗忘、无需访问LLM权重或输出logits以及检索成本与语料库大小无关等优势。

Comments MeMo augments any LLM with up-to-date or domain-specific knowledge via a trained memory model, avoiding costly retraining, mitigating catastrophic forgetting, and remaining robust to retrieval noise

2608.02560 2026-08-04 cs.LG cs.AI cs.IR 新提交 89%

Structured Memory for Edge Language Models: Persistent Context and Corpus Retrieval via O(1) SSM State Injection

边缘语言模型的结构化记忆：通过O(1) SSM状态注入实现持久上下文与语料库检索

Anusha Madan Gopal, Aras Pirbadian, Kristofor D. Carlson, M Anthony Lewis, Jonathan Tapson

机构 * BrainChip Inc.（BrainChip公司）

专题命中长上下文与记忆：LLM(summary_cn,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

AI总结该研究针对边缘语言模型提出PRECOG与SMC机制，将SSM预填充成本压缩至O(1)，在1.2B参数的TENNs-LLM上实现约4500倍预填充加速，达到与RAG相当的答案质量。

2605.25475 2026-07-07 cs.CL cs.AI 版本更新 89%

IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference

IndexMem: 基于潜在记忆的学习型KV缓存驱逐策略用于长上下文LLM推理

Xintong Yang, Hao Gu, Binxing Xu, Lujun Li, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Yike Guo, Sirui Han

机构 * The Hong Kong University of Science（香港科学与技术大学）； Zhejiang University（浙江大学）

专题命中长上下文与记忆：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结提出一种可学习的索引器预测KV重要性，并结合轻量级潜在记忆模块压缩被驱逐的令牌，以在有限KV预算下实现准确的长上下文推理。

2606.29824 2026-06-30 cs.CL cs.AI 89%

Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering

神经程序记忆：通过隐式激活引导赋予LLM智能体能力

Chengfeng Zhao, Yuqiao Tan, Shizhu He, Yequan Wang, Jun Zhao, Kang Liu

机构 * Institute of Automation, CAS（中国科学院自动化研究所）； University of Chinese Academy of Sciences（中国科学院大学）； Beijing Academy of Artificial Intelligence（北京人工智能研究院）

专题命中长上下文与记忆：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结提出神经程序记忆（NPM），一种无训练框架，通过隐式激活引导而非显式指令表示智能体记忆，从历史对比经验中提取程序技能为激活空间中的引导向量，直接激活任务相关神经机制以指导执行。

2606.08340 2026-06-09 cs.AI cs.LG cs.MA 新提交 89%

Benchmarking Open-Ended Multi-Agent Coordination in Language Agents

开放式多智能体协作在语言智能体中的基准测试

Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker, Alexander Rutherford, Davide Paglieri, Aidan Scannell, Henry Gouk, Elliot J. Crowley, Tim Rocktäschel, Amos Storkey

机构 * University of Edinburgh（爱丁堡大学）； University of Oxford（牛津大学）； University College London（伦敦大学学院）

专题命中长上下文与记忆：LLM(summary_cn,abstract)；language agent(title)；language model(abstract)；分类 cs.AI、cs.LG

AI总结提出基于JAX的开放式多智能体协作基准Alem，评估13种现代LLM在长时生存世界中的零样本协作能力，发现协调能力是前沿LLM智能体的独立瓶颈。

Comments 42 pages, preprint

2602.14209 2026-06-08 cs.LG cs.CL 版本更新 89%

MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLM

MAGE：在块扩散LLM中，全[MASK]块已经知道在哪里看

Omin Kwon, Yeonjae Kim, Doyeon Kim, Minseo Kim, Yeonhong Park, Jae W. Lee

机构 * Seoul National University（首尔国立大学）； Meta

专题命中长上下文与记忆：LLM(title,title_cn)；分类 cs.CL、cs.LG

AI总结针对块扩散LLM长上下文推理中KV缓存导致的内存瓶颈，提出无训练方法MAGE，利用块扩散训练目标的对齐特性，在第一步确定整个轨迹的KV子集，实现近无损精度和显著加速。

2604.17535 2026-04-21 cs.CL cs.AI 89%

OPSDL: On-Policy Self-Distillation for Long-Context Language Models

OPSDL：针对长上下文语言模型的在线自蒸馏

Xinsen Zhang, Zhenkai Ding, Tianjun Pan, Run Yang, Chun Kang, Xue Xiong, Jingnan Gu

机构 * Baidu Inc（百度公司）

专题命中长上下文与记忆：language model(title,abstract)；SFT(abstract,abstract_cn)；large language model(abstract)；post-training(abstract)

AI总结 OPSDL通过在线自蒸馏方法提升长上下文能力，利用模型自身短上下文能力作为自教师，通过点-wise KL散度提供每token监督信号，有效减少幻觉并提升样本效率，优于传统后训练方法。

Comments 9 pages, 1 figure

2512.21280 2025-12-25 cs.CL cs.AI 89%

SMART SLM: Structured Memory and Reasoning Transformer, A Small Language Model for Accurate Document Assistance

SMART SLM：结构记忆与推理变换器，一种用于准确文档辅助的小语言模型

Divij Dudeja, Mayukha Pal

机构 * ABB Ability Innovation Center（ABB能力创新中心）； Indian Institute of Information Technology（印度信息科技学院）

专题命中长上下文与记忆：language model(title)；small language model(title)；SLM(title)；分类 cs.CL、cs.AI

AI总结 SMART SLM通过结构化记忆与推理变换器，提升工程文档处理的准确性与效率。

2411.05232 2024-11-11 cs.CL cs.AI 89%

Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities

Shengzhi Li, Kittipat Kampa, Rongyu Lin, Bohang Li, Shichao Pei

专题命中长上下文与记忆：LLM(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Comments We share our latest dataset on https://github.com/findalexli/Abstract2Appendix

2607.07387 2026-07-09 cs.MA cs.SI physics.soc-ph 新提交 89%

A Large Language Model-Driven Agent-Based Modeling Framework with Multi-Round Communication for Simulating Vaccine Opinion Dynamics

一种用于模拟疫苗舆论动态的基于多轮通信的大语言模型驱动的基于代理的建模框架

Bo Zhang, Na Jiang

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

AI总结该研究引入大语言模型（Qwen3-8B）集成到基于代理的建模框架，以疫苗接种舆论动态为例，通过启用不同认知模块模拟舆论动态，发现不同模块对突发舆论有相反影响，再现了社会影响的非线性行为模式，证明框架有效性和潜力。

Comments 11 pages, 5 figures

2607.02716 2026-07-07 cs.MA 新提交 89%

Evaluating Large Language Models for Decision-Making in Agent-Based Urban Mobility Simulations

在基于代理的城市交通模拟中评估用于决策的大语言模型

Bruno Cascaes Alves, Míriam Blank Born, Ulisses Gilioli Francescatto Júnior, Felipe Moura Goulart, Letícia Brandão Caldas, Marilton Sanchotene de Aguiar

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

AI总结研究在多智能体模拟中集成大语言模型作为决策组件，提出混合架构，通过API连接GAMA平台与外部基于大语言模型的模块，能指导智能体重规划行为，比较不同场景下效果，显示其可丰富行为表示。

2512.12313 2026-06-16 cs.CR 版本更新 89%

Taint-Based Code Slicing for LLMs-based Malicious NPM Package Detection

基于污点的代码切片用于基于LLM的恶意NPM包检测

Dang-Khoa Nguyen, Gia-Thang Ho, Quang-Minh Pham, Tuyet A. Dang-Thi, Minh-Khanh Vu, Thanh-Cong Nguyen, Phat T. Tran-Truong, Duc-Ly Vu

专题命中长上下文与记忆：LLM(title_cn,summary_cn)；large language model(abstract)；language model(abstract)

AI总结提出基于污点代码切片的LLM框架，通过隔离安全相关数据流，将输入token数平均减少99.75%，在2537个包上达到87.04%的检测准确率，优于基线方法。

Comments 18 pages, 1 figure, 5 tables, 2 algorithms

2606.05250 2026-06-05 cs.SE 89%

Towards Persistent Case-Based Memory for Autonomous Data Science: A CBR-Augmented R&D-Agent with a Locally Deployable Small Language Model

面向自主数据科学的持久化案例记忆：一种CBR增强的R&D-Agent与本地可部署的小语言模型

Felix Stocker

专题命中长上下文与记忆：language model(title,abstract)；small language model(title,abstract)；SLM(abstract_cn)

AI总结本文提出一种CBR增强的R&D-Agent，通过持久化案例库和小语言模型Gemma 4 31B Dense，在Kaggle竞赛中实现方向性精度提升和方差降低。

Comments 14 pages, 8 figures, 8 tables; preprint, not submitted to any venue; code available at https://github.com/stofe94/cbr-rd-agent

2510.11683 2026-06-01 cs.LG cs.AI cs.CL 89%

Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models

边界引导策略优化：面向扩散大语言模型的内存高效强化学习

Nianyi Lin, Jiajie Zhang, Lei Hou, Juanzi Li

机构 * Tsinghua University（清华大学）

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结针对扩散大语言模型中似然函数难以处理导致强化学习内存开销大的问题，提出边界引导策略优化（BGPO），通过构造满足线性和等价性的下界实现内存高效训练，在数学求解、代码生成和规划任务中显著优于现有方法。

2605.28009 2026-05-28 cs.CL cs.AI cs.LG 89%

MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models

MemGuard：防止长期记忆增强型大语言模型中的记忆污染

Hyeonjeong Ha, Jeonghwan Kim, Cheng Qian, Jiayu Liu, William M. Campbell, Yue Wu, Yuji Zhang, Kathleen McKeown, Dilek Hakkani-Tur, Heng Ji

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； Columbia University（哥伦比亚大学）； Capital One

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结提出MemGuard，一种类型感知的记忆框架，通过显式分配功能角色、维护类型隔离记忆间的关联并选择性组合必要类型的证据，防止异构记忆污染，提升记忆可靠性最高28.27%并减少检索token数最高5.8倍。

2604.17265 2026-05-13 cs.IR 89%

MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search

MemSearch-o1：通过基于推理的内存增长增强大语言模型的代理搜索

Sheng Zhang, Junyi Li, Yingyi Zhang, Pengyue Jia, Yichao Wang, Xiaowei Qian, Wenlin Zhang, Maolin Wang, Yong Liu, Xiangyu Zhao

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；LLM(abstract_cn)

AI总结 MemSearch-o1通过基于推理的内存增长和回溯机制，解决代理搜索中的内存稀释问题，提升大语言模型的推理能力。

1. 长上下文与记忆 4738 篇

Training-Free Long-Context Scaling of Large Language Models

Pipette: Automatic Fine-grained Large Language Model Training Configurator for Real-World Clusters

Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models

Personalized Autonomous Driving with Large Language Models: Field Experiments

A Survey on the Memory Mechanism of Large Language Model based Agents

OVEL: Large Language Model as Memory Manager for Online Video Entity Linking

DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models

CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory

BGE Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models

Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization

Zero- and Few-Shots Knowledge Graph Triplet Extraction with Large Language Models

Learning from Mistakes via Cooperative Study Assistant for Large Language Models

Controllable Multi-document Summarization: Coverage & Coherence Intuitive Policy with Large Language Model Based Rewards

Parallel Context Windows for Large Language Models

MeMo: Memory as a Model

Structured Memory for Edge Language Models: Persistent Context and Corpus Retrieval via O(1) SSM State Injection

IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference

Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering

Benchmarking Open-Ended Multi-Agent Coordination in Language Agents

MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLM

OPSDL: On-Policy Self-Distillation for Long-Context Language Models

SMART SLM: Structured Memory and Reasoning Transformer, A Small Language Model for Accurate Document Assistance

Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities

A Large Language Model-Driven Agent-Based Modeling Framework with Multi-Round Communication for Simulating Vaccine Opinion Dynamics

Evaluating Large Language Models for Decision-Making in Agent-Based Urban Mobility Simulations

Taint-Based Code Slicing for LLMs-based Malicious NPM Package Detection

Towards Persistent Case-Based Memory for Autonomous Data Science: A CBR-Augmented R&D-Agent with a Locally Deployable Small Language Model

Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models

MemGuard: Preventing Memory Contamination in Long-Term Memory-Augmented Large Language Models

MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search