语言大模型 / LLM - arXivDaily 专题

2503.15450 2025-12-03 cs.CL 83%

SkyLadder: Better and Faster Pretraining via Context Window Scheduling

SkyLadder: 通过上下文窗口调度实现更优更高效的预训练

Tongyao Zhu, Qian Liu, Haonan Wang, Shiqi Chen, Xiangming Gu, Tianyu Pang, Min-Yen Kan

机构 * National University of Singapore（新加坡国立大学）； Sea AI Lab（Sea AI实验室）； City University of Hong Kong（香港城市大学）

专题命中长上下文与记忆：pretraining(title,abstract)；LLM(abstract)；分类 cs.CL

AI总结 SkyLadder通过上下文窗口调度策略，在保持基准性能的同时，提升了长上下文任务的表现，并加快了训练速度。

Comments Accepted to NeurIPS 2025. 10 pages

2511.09971 2025-11-14 cs.CL 83%

NumPert: Numerical Perturbations to Probe Language Models for Veracity Prediction

Peter Røysland Aarnes, Vinay Setty

机构 * University of Stavanger（斯塔万格大学）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Accepted in ICJNLP/AACL SRW

2510.27418 2025-11-03 cs.CL 83%

Dynamic Affective Memory Management for Personalized LLM Agents

Junfeng Lu, Yueyan Li

机构 * Beijing University of Posts and Telecommunications（北京邮电大学）

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 12 pasges, 8 figures

2510.18121 2025-10-22 cs.LG cs.DC 83%

Efficient Long-context Language Model Training by Core Attention Disaggregation

Yonghao Zhuang, Junda Chen, Bo Pang, Yi Gu, Yibo Zhu, Yimin Jiang, Ion Stoica, Eric Xing, Hao Zhang

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.LG

2510.13363 2025-10-16 cs.CL 83%

D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree

Xiang Lei, Qin Li, Min Zhang, Min Zhang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 8 pages, 6 figures (main content); 25 pages, 18 figures (total)

2510.13334 2025-10-16 cs.CL 83%

Taming the Fragility of KV Cache Eviction in LLM Inference

Yuan Feng, Haoyu Guo, JunLin Lv, S. Kevin Zhou, Xike Xie

机构 * University of Science and Technology of China（中国科学技术大学）； USTC ； Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research（数据暗夜实验室、MIRACLE中心、苏州先进研究所）

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2505.20444 2025-10-09 cs.LG cs.CV 83%

HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models

Haoran Li, Yingjie Qin, Baoyuan Ou, Lai Xu, Ruiwen Xu

机构 * Carnegie Mellon University（卡内基梅隆大学）； Xiaohongshu Inc.（小红书公司）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.LG

Comments NeurIPS 2025

2510.03978 2025-10-07 cs.CV cs.CL 83%

No Tokens Wasted: Leveraging Long Context in Biomedical Vision-Language Models

Min Woo Sun, Alejandro Lozano, Javier Gamazo Tejero, Vishwesh Nath, Xiao Xiao Sun, James Burgess, Yuhui Zhang, Kun Yuan, Robert Tibshirani, Sean Huver, Serena Yeung-Levy

机构 * Stanford University（斯坦福大学）； NVIDIA USA（NVIDIA公司）

专题命中长上下文与记忆：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2502.01630 2025-09-26 cs.AI 83%

TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialogues

Yubin Ge, Salvatore Romeo, Jason Cai, Raphael Shu, Monica Sunkara, Yassine Benajiba, Yi Zhang

机构 * University of Illinois Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）； Amazon Web Services（亚马逊网络服务）

专题命中长上下文与记忆：LLM(title,abstract)；prompting(abstract)；分类 cs.AI

Comments Accepted at ACL 2025 Findings

2504.03770 2025-07-18 cs.CR cs.AI 83%

JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model

Yi Nian, Shenzhe Zhu, Yuehan Qin, Li Li, Ziyi Wang, Chaowei Xiao, Yue Zhao

机构 * University of Southern California（南加州大学）； University of Toronto（多伦多大学）； University of Maryland（马里兰大学）； University of Wisconsin-Madison（威斯康星大学麦迪逊分校）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.AI

2412.16545 2025-06-26 cs.CL 83%

Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models

Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang, Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu

机构 * Tencent AI Lab（腾讯AI实验室）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments ACL 2025

2506.11440 2025-06-16 cs.CL 83%

AbsenceBench: Language Models Can't Tell What's Missing

Harvey Yiyun Fu, Aryan Shrivastava, Jared Moore, Peter West, Chenhao Tan, Ari Holtzman

机构 * University of Chicago（芝加哥大学）； Stanford University（斯坦福大学）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 23 pages, 8 figures. Code and data are publicly available at https://github.com/harvey-fin/absence-bench

2402.16617 2025-06-11 cs.CL 83%

Long-Context Language Modeling with Parallel Context Encoding

Howard Yen, Tianyu Gao, Danqi Chen

机构 * Princeton Language and Intelligence (PLI)（普林斯顿语言与智能中心）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments ACL 2024. Code, models, and data are available at https://github.com/princeton-nlp/CEPE

2506.06316 2025-06-10 cs.IR cs.AI 83%

A Reinforcement-Learning-Enhanced LLM Framework for Automated A/B Testing in Personalized Marketing

Haoyang Feng, Yanjun Dai, Yuan Gao

专题命中长上下文与记忆：LLM(title,abstract)；language model(abstract)；分类 cs.AI

2506.03090 2025-06-04 cs.CL 83%

Literary Evidence Retrieval via Long-Context Language Models

Katherine Thai, Mohit Iyyer

机构 * UMass Amherst（马萨诸塞大学阿姆赫斯特分校）； University of Maryland, College Park（马里兰大学学院公园分校）

专题命中长上下文与记忆：language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments ACL 2025

2505.13844 2025-05-21 cs.CL 83%

Improve Language Model and Brain Alignment via Associative Memory

Congchi Yin, Yongpeng Zhang, Xuyun Wen, Piji Li

机构 * College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（人工智能学院，南京航空航天大学）； The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education（脑机智能技术重点实验室，教育部）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Accepted by Findings of ACL 2025

2504.15544 2025-04-23 cs.CL 83%

llm-jp-modernbert: A ModernBERT Model Trained on a Large-Scale Japanese Corpus with Long Context Length

Issa Sugiura, Kouta Nakayama, Yusuke Oda

机构 * Kyoto University（京都大学）； NII LLMC（日本信息处理学会大语言模型中心）

专题命中长上下文与记忆：LLM(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments 9 pages, 5 figures

2503.23053 2025-04-01 cs.CL 83%

A Training-free LLM Framework with Interaction between Contextually Related Subtasks in Solving Complex Tasks

Hongjia Liu, Jinlong Li

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2503.09202 2025-03-13 cs.CL 83%

Token Weighting for Long-Range Language Modeling

Falko Helm, Nico Daheim, Iryna Gurevych

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Accepted to NAACL 2025 (Findings). For the code, see https://github.com/UKPLab/naacl2025-token-weighting

2503.00353 2025-03-04 cs.CL cs.IR 83%

U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack

Yunfan Gao, Yun Xiong, Wenlong Wu, Zijing Huang, Bohan Li, Haofen Wang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2312.17296 2025-03-03 cs.CL 83%

Structured Packing in LLM Training Improves Long Context Utilization

Konrad Staniszewski, Szymon Tworkowski, Sebastian Jaszczur, Yu Zhao, Henryk Michalewski, Łukasz Kuciński, Piotr Miłoś

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments AAAI'25

2502.20082 2025-02-28 cs.CL 83%

LongRoPE2: Near-Lossless LLM Context Window Scaling

Ning Shang, Li Lyna Zhang, Siyuan Wang, Gaokai Zhang, Gilsinia Lopez, Fan Yang, Weizhu Chen, Mao Yang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2502.00792 2025-02-04 cs.AI 83%

RTBAgent: A LLM-based Agent System for Real-Time Bidding

Leng Cai, Junxuan He, Yikai Li, Junjie Liang, Yuanping Lin, Ziming Quan, Yawen Zeng, Jin Xu

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted by WWW 2025

2501.16899 2025-01-29 cs.RO cs.AI 83%

RDMM: Fine-Tuned LLM Models for On-Device Robotic Decision Making with Enhanced Contextual Awareness in Specific Domains

Shady Nasrat, Myungsu Kim, Seonil Lee, Jiho Lee, Yeoncheol Jang, Seung-joon Yi

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

2501.03458 2025-01-08 eess.IV cs.AI cs.CV 83%

Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation

Xiao Wang, Fuling Wang, Haowen Wang, Bo Jiang, Chuanfu Li, Yaowei Wang, Yonghong Tian, Jin Tang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments In Peer Review

2412.10079 2024-12-16 cs.CL 83%

Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA

George Arthur Baker, Ankush Raut, Sagi Shaier, Lawrence E Hunter, Katharina von der Wense

专题命中长上下文与记忆：language model(title,abstract)；prompting(abstract)；分类 cs.CL

2408.10188 2024-12-16 cs.CV cs.CL 83%

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, Song Han

专题命中长上下文与记忆：language model(title,abstract)；foundation model(abstract)；分类 cs.CL

Comments Code and models are available at https://github.com/NVlabs/VILA/tree/main/longvila

2411.05928 2024-11-12 cs.CL 83%

Reducing Distraction in Long-Context Language Models by Focused Learning

Zijun Wu, Bingyuan Liu, Ran Yan, Lei Chen, Thomas Delteil

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2402.18216 2024-10-15 cs.CL 83%

LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History

Akash Gupta, Ivaxi Sheth, Vyas Raina, Mark Gales, Mario Fritz

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 20 pages, 13 figures, 20 tables, EMNLP Main Conference 2024

2410.00344 2024-10-08 cs.SD cs.LG eess.AS 83%

Integrating Text-to-Music Models with Language Models: Composing Long Structured Music Pieces

Lilac Atassi

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2404.11976

1. 长上下文与记忆 4760 篇

SkyLadder: Better and Faster Pretraining via Context Window Scheduling

NumPert: Numerical Perturbations to Probe Language Models for Veracity Prediction

Dynamic Affective Memory Management for Personalized LLM Agents

Efficient Long-context Language Model Training by Core Attention Disaggregation

D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree

Taming the Fragility of KV Cache Eviction in LLM Inference

HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models

No Tokens Wasted: Leveraging Long Context in Biomedical Vision-Language Models

TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialogues

JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model

Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models

AbsenceBench: Language Models Can't Tell What's Missing

Long-Context Language Modeling with Parallel Context Encoding

A Reinforcement-Learning-Enhanced LLM Framework for Automated A/B Testing in Personalized Marketing

Literary Evidence Retrieval via Long-Context Language Models

Improve Language Model and Brain Alignment via Associative Memory

llm-jp-modernbert: A ModernBERT Model Trained on a Large-Scale Japanese Corpus with Long Context Length

A Training-free LLM Framework with Interaction between Contextually Related Subtasks in Solving Complex Tasks

Token Weighting for Long-Range Language Modeling

U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack

Structured Packing in LLM Training Improves Long Context Utilization

LongRoPE2: Near-Lossless LLM Context Window Scaling

RTBAgent: A LLM-based Agent System for Real-Time Bidding

RDMM: Fine-Tuned LLM Models for On-Device Robotic Decision Making with Enhanced Contextual Awareness in Specific Domains

Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation

Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Reducing Distraction in Long-Context Language Models by Focused Learning

LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History

Integrating Text-to-Music Models with Language Models: Composing Long Structured Music Pieces