语言大模型 / LLM

Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Stories

David Wilmot, Frank Keller

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to the EMNLP 2021 Conference as a long-paper, 9 pages, 15 pages with appendices and references, 2 figures, 4 tables

Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle

Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 12 pages, 10 figures

Journal ref NAACL 2021

Segatron: Segment-Aware Transformer for Language Modeling and Understanding

He Bai, Peng Shi, Jimmy Lin, Yuqing Xie, Luchen Tan, Kun Xiong, Wen Gao, Ming Li

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted by AAAI 2021

Cross-Utterance Language Models with Acoustic Error Sampling

G. Sun, C. Zhang, P. C. Woodland

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 5 pages

Multi-scale Transformer Language Models

Sandeep Subramanian, Ronan Collobert, Marc'Aurelio Ranzato, Y-Lan Boureau

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

LAMOL: LAnguage MOdeling for Lifelong Language Learning

Fan-Keng Sun, Cheng-Hao Ho, Hung-Yi Lee

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Language models and Automated Essay Scoring

Pedro Uria Rodriguez, Amir Jafari, Christopher M. Ormerod

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 17 pages, 7 figures

Multiplicative Models for Recurrent Language Modeling

Diego Maupomé, Marie-Jean Meurs

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 10 pages, pre-print from Proceedings of CICLing 2019: 20th International Conference on Computational Linguistics and Intelligent Text Processing

Calibration, Entropy Rates, and Memory in Language Models

Mark Braverman, Xinyi Chen, Sham M. Kakade, Karthik Narasimhan, Cyril Zhang, Yi Zhang

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Language Modeling through Long Term Memory Network

Anupiya Nugaliyadde, Kok Wai Wong, Ferdous Sohel, Hong Xie

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments The paper is accepted to be published in IJCNN 2019

Multi-cell LSTM Based Neural Language Model

Thomas Cherian, Akshay Badola, Vineet Padmanabhan

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 7 pages including 5 figures

Lattice Rescoring Strategies for Long Short Term Memory Language Models in Speech Recognition

Shankar Kumar, Michael Nirschl, Daniel Holtmann-Rice, Hank Liao, Ananda Theertha Suresh, Felix Yu

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at ASRU 2017

Journal ref Proceedings of ASRU 2017

Unbounded cache model for online language modeling with open vocabulary

Edouard Grave, Moustapha Cisse, Armand Joulin

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to NIPS 2017

Long-Short Range Context Neural Networks for Language Modeling

Youssef Oualil, Mittul Singh, Clayton Greenberg, Dietrich Klakow

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Published at EMNLP'16

Regularizing and Optimizing LSTM Language Models

Stephen Merity, Nitish Shirish Keskar, Richard Socher

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Dynamic Entity Representations in Neural Language Models

Yangfeng Ji, Chenhao Tan, Sebastian Martschat, Yejin Choi, Noah A. Smith

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2017 camera-ready version

Improving Neural Language Models with a Continuous Cache

Edouard Grave, Armand Joulin, Nicolas Usunier

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Submitted to ICLR 2017

Compressing Neural Language Models by Sparse Word Representations

Yunchuan Chen, Lili Mou, Yan Xu, Ge Li, Zhi Jin

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ACL-16, pages 226--235

A Factorized Recurrent Neural Network based architecture for medium to large vocabulary Language Modelling

Anantharaman Palacode Narayana Iyer

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages

Scaling Recurrent Neural Network Language Models

Will Williams, Niranjani Prasad, David Mrva, Tom Ash, Tony Robinson

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Human-like fleeting memory improves language learning but impairs reading time prediction in transformer language models

类人短暂记忆提升语言学习但损害变压器语言模型的阅读时间预测

Abishek Thamma, Micha Heilbron

机构 * University of Amsterdam, Amsterdam Brain and Cognition（阿姆斯特丹大学，阿姆斯特丹脑与认知中心） ； Vrije Universiteit Amsterdam, Department of Informatics（阿姆斯特丹自由大学，信息学院） ； Max Planck Institute for Psycholinguistics（马克斯·普朗克心理学语言学研究所） 纠错

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL；LLM(comments)

AI总结 研究探讨了短暂记忆对语言学习和阅读时间预测的影响，发现短暂记忆提升语言学习但损害阅读时间预测，挑战了传统认知科学观点。

Comments v2: Revised after peer review. Accepted for publication in Transactions of the Association for Computational Linguistics v3: Added link to code repository. Code: https://github.com/drhanjones/fmt-llm

Journal ref Transactions of the Association for Computational Linguistics 14 (2026) 877-892

Overcoming the Impedance Mismatch: A Theoretical Roadmap for Fusing Foundation Models and Knowledge Graphs

克服阻抗不匹配：融合基础模型与知识图谱的理论路线图

Sahil Rajesh Dhayalkar

机构 * Arizona State University（亚利桑那州立大学） 纠错

专题命中 长上下文与记忆 ：foundation model(title,abstract)；分类 cs.AI

AI总结 本文提出“阻抗不匹配”概念，形式化分析基础模型与知识图谱的结构与几何摩擦，通过三级层次分类揭示现有方法的局限，并给出理论路线图实现真正的语义融合。

Comments 12 pages. Accepted at the ACL 2026 4th Workshop on Towards Knowledgeable Foundation Models (https://openreview.net/forum?id=hXDYsNAq8m)

How Foundational are Foundation Models for Time Series Forecasting?

Nouha Karaouli, Denis Coquenet, Elisa Fromont, Martial Mermillod, Marina Reyboz

机构 * Univ. Rennes（里昂大学） ； CNRS（法国国家科学研究中心） ； Inria（法国国家信息与自动化技术研究所） ； IRISA - UMR 6074（国际信息与自动化研究所） ； Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学） ； Univ. Savoie Mont Blanc（萨瓦伊-蒙布莱大学） ； LPNC（实验室） ； CEA（法国原子能委员会） ； LIST（法国国家信息与自动化技术研究所） 纠错

专题命中 长上下文与记忆 ：foundation model(title,abstract)；分类 cs.LG

Comments Typo rectified in this v3 version. Accepted at NeurIPS 2025 Workshop on Recent Advances in Time Series Foundation Models (BERT2S)

Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps

Yijiong Yu, Yongfeng Huang, Zhixiao Qi, Wei Wang, Weifeng Liu, Ran Chen, Ji Pei

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL；LLM(comments)

Comments Our code is publicly available at https://github.com/yuyijiong/hard_retrieval_for_llm and the datasets is at https://huggingface.co/datasets/yuyijiong/difficult_retrieval

In-Context Ensemble Learning from Pseudo Labels Improves Video-Language Models for Low-Level Workflow Understanding

Moucheng Xu, Evangelos Chatzaroulas, Luc McCutcheon, Abdul Ahad, Hamzah Azeem, Janusz Marecki, Ammar Anwar

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.AI

Comments To appear in NeurIPS Workshop on Video-Language Models 2024

Q-First: Attention and Feed-Forward Concurrency at the Smallest Change to the Block

Q-First：在块的最小改动下实现注意力与前馈的并发

WenJie Fan

专题命中 长上下文与记忆 ：LLM(summary_cn,abstract)

AI总结 该研究提出Q-First协议，通过交换LLM解码器块的注意力与前馈子层顺序消除依赖，实现并发，验证其精度并训练8种变体证明改动影响有界且符合要求。

Comments 17 pages,3 figures

CogChat: Knowledge Graph-Augmented Conversational AI with Heterogeneous Graph Transformer for Cognitive Grounding in Design Generation

CogChat：结合知识图谱与异构图变换器的对话式人工智能，用于设计生成中的认知接地

Jiin Choi, Kyung Hoon Hyun

专题命中 长上下文与记忆 ：LLM(summary_cn,abstract)

AI总结 本研究提出结合异构图变换器的CogChat框架，以设计师专属动态知识图谱为基础，提升设计对话的上下文保留度与意图解读效果，降低认知负荷，为LLM交互的长期上下文管理提供新方法。

HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Management

HiSparse：通过分层KV缓存管理扩展稀疏注意力解码

Zhiqiang Xie, Zhangheng Huang, Tingwei Huang, Ziyi Xu, Ruiyang Ma, Christos Kozyrakis

专题命中 长上下文与记忆 ：LLM(summary_cn,abstract)

AI总结 HiSparse是一种分层KV缓存，通过将KV历史存主机、GPU设固定缓存等方式，提升长上下文稀疏注意力LLM解码吞吐量达4.7倍，未增加每token成本，已集成到SGLang并在多平台验证。

UniEvo-RS: Omni-Prompt Unified Remote Sensing Segmentation with Representative Exemplar-Driven Prototype Evolution

UniEvo-RS：基于代表性示例驱动原型演化的全提示统一遥感分割

Kunquan Zhang, Peilang Li, Xikun Hu, Yunkai Yang, Yushan Zou, Zhiwei Zhang, Runmin Dong

专题命中 长上下文与记忆 ：LLM(abstract,abstract_cn)；language model(abstract)；prompting(abstract)

AI总结 UniEvo-RS是配备代表性示例驱动原型演化的全提示统一遥感分割框架，通过多指令提示数据集与原型演化机制，在批量标注中对未见过的类别实现无需训练的精度提升，性能达当前最优。

Comments 18 pages, 8 figures, 9 tables

FedWorld: Scope-Aware Federation of Agent World Models

FedWorld：感知范围的智能体世界模型联邦

Yuchao Hou

专题命中 长上下文与记忆 ：LLM(abstract,abstract_cn)；large language model(abstract)；language model(abstract)

AI总结 FEDWORLD是一种感知范围的智能体世界模型联邦协议，通过交换结构化抽象转移规则，减少冲突动态下的负迁移，提升智能体任务成功率。