语言大模型 / LLM - arXivDaily 专题

2403.08424 2024-10-01 cs.CR cs.AI cs.CL 88%

Distract Large Language Models for Automatic Jailbreak Attack

Zeguan Xiao, Yan Yang, Guanhua Chen, Yun Chen

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024

2409.18454 2024-09-30 cs.CL cs.AI 88%

Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications

Aditi Godbole, Jabin Geevarghese George, Smita Shandilya

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

2409.04164 2024-09-09 cs.CL cs.LG cs.SE 88%

Can OpenSource beat ChatGPT? -- A Comparative Study of Large Language Models for Text-to-Code Generation

Luis Mayer, Christian Heumann, Matthias Aßenmacher

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Conference Paper accepted at the 9th SwissText Conference (2024)

2408.13987 2024-08-27 cs.CL cs.AI 88%

Focused Large Language Models are Stable Many-Shot Learners

Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Heda Wang, Yao Hu, Kan Li

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages

2408.10151 2024-08-20 cs.CL cs.LG 88%

Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Models

Amey Hengle, Prasoon Bajpai, Soham Dan, Tanmoy Chakraborty

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

2402.03009 2024-08-20 cs.CL cs.AI 88%

UniMem: Towards a Unified View of Long-Context Large Language Models

Junjie Fang, Likai Tang, Hongzhe Bi, Yujia Qin, Si Sun, Zhenyu Li, Haolun Li, Yongjian Li, Xin Cong, Yankai Lin, Yukun Yan, Xiaodong Shi, Sen Song, Zhiyuan Liu, Maosong Sun

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments COLM 2024

2408.00655 2024-08-15 cs.AI cs.CL 88%

SentenceVAE: Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context

Hongjun An, Yifan Chen, Zhe Sun, Xuelong Li

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments update the article

2401.02509 2024-07-09 q-bio.NC cs.AI cs.CL 88%

Memory, Consciousness and Large Language Model

Jitang Li, Jinzheng Li

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

2308.16137 2024-06-26 cs.CL cs.AI 88%

LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models

Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, Sinong Wang

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments NAACL 2024 Outstanding paper, 9 pages, 6 figures

2406.15765 2024-06-25 cs.LG cs.CL 88%

Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration

Zhongzhi Yu, Zheng Wang, Yonggan Fu, Huihong Shi, Khalid Shaikh, Yingyan Celine Lin

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

2402.02244 2024-05-30 cs.CL cs.LG 88%

Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models

Xindi Wang, Mahsa Salmani, Parsa Omidi, Xiangyu Ren, Mehdi Rezagholizadeh, Armaghan Eshaghi

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to IJCAI 2024 Survey Track -- camera-ready version

2404.19234 2024-05-01 cs.AI cs.CL cs.DB 88%

Multi-hop Question Answering over Knowledge Graphs using Large Language Models

Abir Chakraborty

专题命中长上下文与记忆：language model(title,abstract)；large language model(title)；LLM(abstract)；分类 cs.CL、cs.AI

2401.18058 2024-02-01 cs.CL cs.LG 88%

LongAlign: A Recipe for Long Context Alignment of Large Language Models

Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, Juanzi Li

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

2311.04661 2024-01-26 cs.CL cs.LG 88%

Massive Editing for Large Language Models via Meta Learning

Chenmien Tan, Ge Zhang, Jie Fu

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

2401.08683 2024-01-18 cs.AR cs.AI cs.LG cs.PL cs.SE 88%

Zero-Shot RTL Code Generation with Attention Sink Augmented Large Language Models

Selim Sandal, Ismail Akturk

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

2307.02738 2023-10-04 cs.AI cs.CL cs.SC 88%

RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language Models

Brandon Kynoch, Hugo Latapie, Dwane van der Sluis

专题命中长上下文与记忆：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 7 figures, 1 table, Our code is publicly available online at: https://github.com/cisco-open/DeepVision/tree/main/recallm

2509.18868 2025-09-24 cs.AI 88%

Memory in Large Language Models: Mechanisms, Evaluation and Evolution

Dianxing Zhang, Wendong Li, Kani Song, Jiaye Lu, Gang Li, Liuchun Yang, Sheng Li

专题命中长上下文与记忆：large language model(title)；language model(title)；LLM(abstract,comments)；pretraining(abstract)

Comments 50 pages, 1 figure, 8 tables This is a survey/framework paper on LLM memory mechanisms and evaluation

2606.29178 2026-06-30 cs.AI cs.CL cs.LG 88%

Selective Memory Retention for Long-Horizon LLM Agents

面向长周期LLM智能体的选择性记忆保留

Pranath Reddy

专题命中长上下文与记忆：LLM(title,title_cn)；分类 cs.CL、cs.AI、cs.LG

AI总结提出TraceRetain框架，通过可解释特征评分和驱逐机制管理有限外部记忆，在噪声环境下保持任务成功率，验证了选择性记忆保留对长周期智能体的有效性。

Comments Accepted at the International Conference on Machine Learning (ICML) 2026

2606.09900 2026-06-10 cs.CL cs.AI cs.IR cs.LG 新提交 88%

Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History

更少上下文，更高准确率：一种用于LLM Agent的双时间记忆引擎，其中精简检索上下文优于完整历史

Liuyin Wang

机构 * Independent Researcher（独立研究者）

专题命中长上下文与记忆：LLM(title,title_cn)；分类 cs.CL、cs.AI、cs.LG

AI总结提出一种双时间记忆引擎Engram，通过混合读取路径从约9.6k token的检索片段中回答，在LongMemEval_S上达到83.6%准确率，比完整历史（79k token）高10.4个百分点，且无错误。

Comments 14 pages, 4 figures, 3 tables. Code, reproducible harness, and raw per-question logs: https://github.com/ly-wang19/engram

2605.13542 2026-05-14 cs.AI cs.CL cs.LG cs.MA 88%

RealICU: Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation

RealICU: 大语言模型能否理解长期上下文ICU数据？一个超越行为模仿的基准测试

Chengzhi Shen, Weixiang Shen, Tobias Susetzky, Chen, Chen, Jun Li, Yuyuan Liu, Xuepeng Zhang, Zhenyu Gong, Daniel Rueckert, Jiazhen Pan

机构 * Technical University of Munich（慕尼黑技术大学）； TUM University Hospital（TUM大学医院）； LMU Munich（慕尼黑大学）； University of Sheffield（谢菲尔德大学）； University of Oxford（牛津大学）； Zhongshan Hospital Fudan University（复旦大学中山医院）； Sun Yat-sen University Cancer Center（中山大学肿瘤中心）； Imperial College London（伦敦帝国学院）； Munich Center for Machine Learning（慕尼黑机器学习中心）； relAI – Konrad Zuse School of Excellence in Reliable AI（relAI – 卡诺夫茨卓越可靠人工智能学校）

专题命中长上下文与记忆：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 RealICU通过真实ICU场景评估大语言模型的长期上下文理解能力，提出四个医生驱动任务，揭示现有模型在临床推荐中的召回与安全性的权衡问题。

2604.20915 2026-04-24 cs.LG cs.AI cs.CL cs.SE math.OC 88%

Absorber LLM: Harnessing Causal Synchronization for Test-Time Training

吸收式大语言模型：利用因果同步进行测试时训练

Zhixin Zhang, Shabo Zhang, Chengcan Wu, Zeming Wei, Meng Sun

机构 * Peking University（北京大学）

专题命中长上下文与记忆：LLM(title,summary_cn)；分类 cs.CL、cs.AI、cs.LG

AI总结本文提出Absorber LLM，通过自监督因果同步解决长上下文保留问题，优化模型在测试时的内存和准确率。

2604.19769 2026-04-23 cs.CL cs.AI cs.LG 88%

TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference

TTKV：面向长上下文LLM推理的时序分层KV缓存

Gradwell Dzikanyanga, Weihao Yang, Hao Huang, Donglei Wu, Shihao Wang, Wen Xia, Sanjeeb K C

机构 * Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））； Guangzhou University（广州大学）

专题命中长上下文与记忆：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 TTKV通过时序分层缓存框架优化KV存储，利用HBM和DRAM解耦快慢存储，根据时间接近性分配KV状态，减少跨层通信开销，提升推理效率。

2601.03938 2026-04-21 cs.LG cs.AI cs.CL 88%

FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning

FOREVER: 基于遗忘曲线的记忆回放用于语言模型连续学习

Yujie Feng, Hao Wang, Jian Li, Xu Chu, Zhaolu Kang, Yiran Liu, Yasha Wang, Philip S. Yu, Xiao-Ming Wu

机构 * The Hong Kong Polytechnic University（香港理工大学）； ShineMo Ltd., China（中国 ShineMo 公司）； Solar System of OVB, Tencent, China（腾讯 OVB 太阳系中国）； Peking University（北京大学）； University College London（伦敦大学学院）； University of Illinois Chicago（伊利诺伊大学芝加哥分校）

专题命中长上下文与记忆：language model(title,abstract)；LLM(abstract,abstract_cn)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 FOREVER通过引入遗忘曲线概念，提出了一种基于模型自身时间的连续学习框架，有效缓解了语言模型在持续学习中的灾难性遗忘问题。

Comments ACL 2026 Camera-ready

2604.14993 2026-04-17 cs.DC cs.PF 88%

Serving Chain-structured Jobs with Large Memory Footprints with Application to Large Foundation Model Serving

为具有大内存足迹的链式作业服务而设计的链式结构作业服务

Tingyang Sun, Ting He, I-Hong Hou

专题命中长上下文与记忆：foundation model(title,abstract)；LLM(abstract,abstract_cn)；large language model(abstract)；language model(abstract)

AI总结本文研究了大基础模型服务中的链式结构作业服务问题，提出了一种基于块放置和缓存分配的新型问题解决方案，通过流水线并行化解决了服务器链组成问题，显著提升了响应时间。

Comments Technical report

2506.15001 2026-01-09 cs.CL cs.AI cs.LG 88%

Memory Tokens: Large Language Models Can Generate Reversible Sentence Embeddings

记忆令牌：大语言模型可以生成可逆的句子嵌入

Ignacio Sastre, Aiala Rosá

机构 * Instituto de Computación, Facultad de Ingeniería, Universidad de la República（计算机研究所，工程学院，乌拉圭共和国大学）

专题命中长上下文与记忆：large language model(title,comments)；language model(title,comments)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结本研究通过引入记忆令牌，使大语言模型能够生成可逆的句子嵌入，实现对原始文本的精确重建，展示了LLM在记忆检索与文本生成中的新能力。

Comments This paper will be presented at The First Workshop on Large Language Model Memorization (L2M2) at ACL 2025

2512.23862 2026-01-01 cs.LG cs.AI cs.CL 88%

Probing the Limits of Compressive Memory: A Study of Infini-Attention in Small-Scale Pretraining

探测压缩记忆的极限：对Infini-Attention在小规模预训练中的研究

Ruizhe Huang, Kexuan Zhang, Yihao Fang, Baifeng Yu

机构 * Huawei Technologies Canada Co., Ltd（华为技术加拿大有限公司）

专题命中长上下文与记忆：pretraining(title,abstract)；language model(abstract)；small language model(abstract)；SLM(abstract)

AI总结本研究通过Infini-Attention在小规模预训练中提升SLMs的长上下文能力，验证了其在有限参数下的有效性。

2407.01178 2025-01-29 cs.CL cs.AI cs.LG 88%

$\text{Memory}^3$: Language Modeling with Explicit Memory

Hongkang Yang, Zehao Lin, Wenjin Wang, Hao Wu, Zhiyu Li, Bo Tang, Wenqiang Wei, Jinbo Wang, Zeyun Tang, Shichao Song, Chenyang Xi, Yu Yu, Kai Chen, Feiyu Xiong, Linpeng Tang, Weinan E

专题命中长上下文与记忆：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Journal ref Journal of Machine Learning, 3(2024), 300-346

2404.03868 2024-10-03 cs.CL cs.AI cs.LG 88%

Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction

Bowen Zhang, Harold Soh

专题命中长上下文与记忆：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 18 pages, 3 figures, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

2406.05369 2024-06-11 cs.CL cs.AI cs.LG 88%

Venn Diagram Prompting : Accelerating Comprehension with Scaffolding Effect

Sakshi Mahendru, Tejul Pandit

专题命中长上下文与记忆：prompting(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Preprint. 10 pages, Accepted in 2024 the 6th World Symposium on Artificial Intelligence (WSAI 2024)

2405.06907 2024-05-24 cs.CL cs.AI cs.LG cs.PL 88%

AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents

Shuyuan Xu, Zelong Li, Kai Mei, Yongfeng Zhang

专题命中长上下文与记忆：LLM(title,abstract)；large language model(abstract)；language model(abstract)；language agent(abstract)

Comments 12 pages, 6 figures, comments and suggestions are welcome

1. 长上下文与记忆 4738 篇

Distract Large Language Models for Automatic Jailbreak Attack

Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications

Can OpenSource beat ChatGPT? -- A Comparative Study of Large Language Models for Text-to-Code Generation

Focused Large Language Models are Stable Many-Shot Learners

Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Models

UniMem: Towards a Unified View of Long-Context Large Language Models

SentenceVAE: Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context

Memory, Consciousness and Large Language Model

LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models

Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration

Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models

Multi-hop Question Answering over Knowledge Graphs using Large Language Models

LongAlign: A Recipe for Long Context Alignment of Large Language Models

Massive Editing for Large Language Models via Meta Learning

Zero-Shot RTL Code Generation with Attention Sink Augmented Large Language Models

RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language Models

Memory in Large Language Models: Mechanisms, Evaluation and Evolution

Selective Memory Retention for Long-Horizon LLM Agents

Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History

RealICU: Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation

Absorber LLM: Harnessing Causal Synchronization for Test-Time Training

TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference

FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning

Serving Chain-structured Jobs with Large Memory Footprints with Application to Large Foundation Model Serving

Memory Tokens: Large Language Models Can Generate Reversible Sentence Embeddings

Probing the Limits of Compressive Memory: A Study of Infini-Attention in Small-Scale Pretraining

$\text{Memory}^3$: Language Modeling with Explicit Memory

Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction

Venn Diagram Prompting : Accelerating Comprehension with Scaffolding Effect

AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents