语言大模型 / LLM - arXivDaily 专题

2506.01963 2025-06-04 cs.LG cs.CL 84%

Breaking Quadratic Barriers: A Non-Attention LLM for Ultra-Long Context Horizons

Andrew Kiruluta, Preethi Raju, Priscilla Burity

机构 * School of Information UC Berkeley（信息学院 UC 伯克利）

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2412.18120 2025-06-03 cs.CL cs.AI 84%

Do Language Models Understand the Cognitive Tasks Given to Them? Investigations with the N-Back Paradigm

Xiaoyang Hu, Richard L. Lewis

机构 * Brown University（布朗大学）； University of Michigan（密歇根大学）

专题命中长上下文与记忆：language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 Findings

2505.03434 2025-05-07 cs.AI cs.LG 84%

Procedural Memory Is Not All You Need: Bridging Cognitive Gaps in LLM-Based Agents

Schaun Wheeler, Olivier Jeunen

机构 * aampe

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted to the workshop on Hybrid AI for Human-Centric Personalization (HyPer), co-located with ACM UMAP '25

2504.20117 2025-05-06 cs.SE cs.AI cs.CL cs.MA 84%

ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologies

Shubham Gandhi, Dhruv Shah, Manasi Patwardhan, Lovekesh Vig, Gautam Shroff

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Journal ref AI4Research Workshop @ AAAI 2025

2504.14519 2025-04-22 cs.LG cs.AI 84%

SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training

Zhouyang Li, Yuliang Liu, Wei Zhang, Tailing Yuan, Bin Chen, Chengru Song, Di Zhang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2504.05607 2025-04-09 cs.CL cs.AI 84%

FactGuard: Leveraging Multi-Agent Systems to Generate Answerable and Unanswerable Questions for Enhanced Long-Context LLM Extraction

Qian-Wen Zhang, Fang Li, Jie Wang, Lingfeng Qiao, Yifei Yu, Di Yin, Xing Sun

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2503.15888 2025-03-21 cs.CL cs.AI 84%

Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models

Baolong Bi, Shenghua Liu, Yiwei Wang, Yilong Xu, Junfeng Fang, Lingrui Mei, Xueqi Cheng

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2411.09009 2025-03-12 cs.LG cs.CL 84%

Cut Your Losses in Large-Vocabulary Language Models

Erik Wijmans, Brody Huval, Alexander Hertzberg, Vladlen Koltun, Philipp Krähenbühl

专题命中长上下文与记忆：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments To appear in ICLR 2025 (Oral). Code is available at https://github.com/apple/ml-cross-entropy

2410.02694 2025-03-07 cs.CL cs.AI 84%

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly

Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, Danqi Chen

专题命中长上下文与记忆：language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025. Project page: https://princeton-nlp.github.io/HELMET/

2407.07972 2025-03-03 cs.LG cs.AI 84%

Deconstructing What Makes a Good Optimizer for Language Models

Rosie Zhao, Depen Morwani, David Brandfonbrener, Nikhil Vyas, Sham Kakade

专题命中长上下文与记忆：language model(title,abstract)；prompting(abstract)；分类 cs.AI、cs.LG

Comments 21 pages, ICLR 2025

2502.19529 2025-02-28 cs.CL cs.AI 84%

Cognitive networks highlight differences and similarities in the STEM mindsets of human and LLM-simulated trainees, experts and academics

Edith Haim, Lars van den Bergh, Cynthia S. Q. Siew, Yoed N. Kenett, Daniele Marinazzo, Massimo Stella

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Keywords: cognitive network science; mindset measurement; associative knowledge; artificial intelligence; simulated participants

2502.15732 2025-02-25 cs.LG cs.AI cs.DB cs.SE 84%

Data Wrangling Task Automation Using Code-Generating Language Models

Ashlesha Akella, Krishnasuri Narayanam

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted at AAAI 2025 Demo

2502.10420 2025-02-18 cs.AI cs.CL 84%

Position: Stop Acting Like Language Model Agents Are Normal Agents

Elija Perrier, Michael Timothy Bennett

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Under review

2501.09645 2025-01-17 cs.AI cs.CL cs.HC 84%

CarMem: Enhancing Long-Term Memory in LLM Voice Assistants through Category-Bounding

Johannes Kirmayr, Lukas Stappen, Phillip Schneider, Florian Matthes, Elisabeth André

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted for presentation at the International Conference on Computational Linguistics (COLING 2025)

2501.02336 2025-01-07 cs.CL cs.AI 84%

AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference

Zhuomin He, Yizhen Yao, Pengfei Zuo, Bin Gao, Qinya Li, Zhenzhe Zheng, Fan Wu

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 9 pages,10 figures, AAAI

2409.10516 2025-01-03 cs.LG cs.CL 84%

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval

Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Zhenhua Han, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, Lili Qiu

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 19 pages

2412.16443 2024-12-24 cs.LG cs.AI 84%

Has LLM Reached the Scaling Ceiling Yet? Unified Insights into LLM Regularities and Constraints

Charles Luo

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2312.11193 2024-12-13 cs.CL cs.AI 84%

Training With "Paraphrasing the Original Text" Teaches LLM to Better Retrieve in Long-context Tasks

Yijiong Yu, Yongfeng Huang, Zhixiao Qi, Zhe Zhou

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Our code and datasets are available at https://github.com/yuyijiong/train_with_paraphrasing

2403.04317 2024-11-05 cs.LG cs.CL 84%

Online Adaptation of Language Models with a Memory of Amortized Contexts

Jihoon Tack, Jaehyung Kim, Eric Mitchell, Jinwoo Shin, Yee Whye Teh, Jonathan Richard Schwarz

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Published as a conference proceeding for NeurIPS 2024

2408.09434 2024-10-08 cs.CL cs.AI 84%

HySem: A context length optimized LLM pipeline for unstructured tabular extraction

Narayanan PP, Anantharaman Palacode Narayana Iyer

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 7 tables, 10 figures, 2 algorithms

2311.04939 2024-09-09 cs.CL cs.AI 84%

LooGLE: Can Long-Context Language Models Understand Long Contexts?

Jiaqi Li, Mengmeng Wang, Zilong Zheng, Muhan Zhang

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2407.03651 2024-07-16 cs.CL cs.AI 84%

Evaluating Language Model Context Windows: A "Working Memory" Test and Inference-time Correction

Amanda Dsouza, Christopher Glaze, Changho Shin, Frederic Sala

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2403.19887 2024-07-04 cs.CL cs.LG 84%

Jamba: A Hybrid Transformer-Mamba Language Model

Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, Omri Abend, Raz Alon, Tomer Asida, Amir Bergman, Roman Glozman, Michael Gokhman, Avashalom Manevich, Nir Ratner, Noam Rozen, Erez Shwartz, Mor Zusman, Yoav Shoham

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Webpage: https://www.ai21.com/jamba

2406.13121 2024-06-21 cs.CL cs.AI cs.IR 84%

Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?

Jinhyuk Lee, Anthony Chen, Zhuyun Dai, Dheeru Dua, Devendra Singh Sachan, Michael Boratko, Yi Luan, Sébastien M. R. Arnold, Vincent Perot, Siddharth Dalmia, Hexiang Hu, Xudong Lin, Panupong Pasupat, Aida Amini, Jeremy R. Cole, Sebastian Riedel, Iftekhar Naim, Ming-Wei Chang, Kelvin Guu

专题命中长上下文与记忆：language model(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments 29 pages. Dataset available at https://github.com/google-deepmind/loft

2404.14469 2024-06-18 cs.CL cs.AI 84%

SnapKV: LLM Knows What You are Looking for Before Generation

Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, Deming Chen

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2406.06110 2024-06-11 cs.CL cs.AI 84%

Recurrent Context Compression: Efficiently Expanding the Context Window of LLM

Chensen Huang, Guibo Zhu, Xuepeng Wang, Yifei Luo, Guojing Ge, Haoran Chen, Dong Yi, Jinqiao Wang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2402.08567 2024-06-04 cs.CL cs.CR cs.CV cs.LG cs.MA 84%

Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast

Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, Min Lin

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments ICML 2024

2404.16811 2024-04-29 cs.CL cs.AI 84%

Make Your LLM Fully Utilize the Context

Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 7 figures, 3 tables, 9 examples

2402.11550 2024-03-14 cs.CL cs.AI 84%

LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration

Jun Zhao, Can Zu, Hao Xu, Yi Lu, Wei He, Yiwen Ding, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2403.04797 2024-03-11 cs.CL cs.LG 84%

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding

Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

1. 长上下文与记忆 4760 篇

Breaking Quadratic Barriers: A Non-Attention LLM for Ultra-Long Context Horizons

Do Language Models Understand the Cognitive Tasks Given to Them? Investigations with the N-Back Paradigm

Procedural Memory Is Not All You Need: Bridging Cognitive Gaps in LLM-Based Agents

ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologies

SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training

FactGuard: Leveraging Multi-Agent Systems to Generate Answerable and Unanswerable Questions for Enhanced Long-Context LLM Extraction

Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models

Cut Your Losses in Large-Vocabulary Language Models

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly

Deconstructing What Makes a Good Optimizer for Language Models

Cognitive networks highlight differences and similarities in the STEM mindsets of human and LLM-simulated trainees, experts and academics

Data Wrangling Task Automation Using Code-Generating Language Models

Position: Stop Acting Like Language Model Agents Are Normal Agents

CarMem: Enhancing Long-Term Memory in LLM Voice Assistants through Category-Bounding

AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval

Has LLM Reached the Scaling Ceiling Yet? Unified Insights into LLM Regularities and Constraints

Training With "Paraphrasing the Original Text" Teaches LLM to Better Retrieve in Long-context Tasks

Online Adaptation of Language Models with a Memory of Amortized Contexts

HySem: A context length optimized LLM pipeline for unstructured tabular extraction

LooGLE: Can Long-Context Language Models Understand Long Contexts?

Evaluating Language Model Context Windows: A "Working Memory" Test and Inference-time Correction

Jamba: A Hybrid Transformer-Mamba Language Model

Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?

SnapKV: LLM Knows What You are Looking for Before Generation

Recurrent Context Compression: Efficiently Expanding the Context Window of LLM

Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast

Make Your LLM Fully Utilize the Context

LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding