语言大模型 / LLM

Compute Can't Handle the Truth: Why Communication Tax Prioritizes Memory and Interconnects in Modern AI Infrastructure

Myoungsoo Jung

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Efficient Long Context Fine-tuning with Chunk Flow

Xiulong Yuan, Hongtao Xu, Wenting Shen, Ang Wang, Xiafei Qiu, Jie Zhang, Yuqiong Liu, Bowen Yu, Junyang Lin, Mingzhen Li, Weile Jia, Yong Li, Wei Lin

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

SAM4D: Segment Anything in Camera and LiDAR Streams

Jianyun Xu, Song Wang, Ziqian Ni, Chunyong Hu, Sheng Yang, Jianke Zhu, Qiang Li

机构 * Unmanned Vehicle Dept., CaiNiao Inc., Alibaba Group（阿里巴巴集团 Cainiao 部门） ； Zhejiang University（浙江大学） 纠错

专题命中 长上下文与记忆 ：foundation model(abstract)；prompting(abstract)

Comments Accepted by ICCV2025, Project Page: https://SAM4D-Project.github.io

Evaluating Long Range Dependency Handling in Code Generation LLMs

Yannick Assogba, Donghao Ren

机构 * Apple（苹果公司） 纠错

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 36 pages, 18 figures

Journal ref Transactions on Machine Learning Research, 2025. ISSN 2835-8856. https://openreview.net/forum?id=yzACI2vFaX

In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly

Puneesh Deora, Bhavya Vasudeva, Tina Behnia, Christos Thrampoulidis

机构 * University of British Columbia（不列颠哥伦比亚大学） ； University of Southern California（南加州大学） 纠错

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 28 pages, 19 figures

PhysID: Physics-based Interactive Dynamics from a Single-view Image

Sourabh Vasant Gothe, Ayon Chattopadhyay, Gunturi Venkata Sai Phani Kiran, Pratik, Vibhav Agarwal, Jayesh Rajkumar Vachhani, Sourav Ghosh, Parameswaranath VM, Barath Raj KR

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Comments Published in 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Project page: https://physid.github.io/

Journal ref 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India, 2025, pp. 1-5

SeqPE: Transformer with Sequential Position Encoding

Huayang Li, Yahui Liu, Hongyu Sun, Deng Cai, Leyang Cui, Wei Bi, Peilin Zhao, Taro Watanabe

机构 * Nara Institute of Science and Technology（奈良科学技术研究所） ； Kuaishou Technology（快手科技） ； Tencent（腾讯） 纠错

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

FLAM: Frame-Wise Language-Audio Modeling

Yusong Wu, Christos Tsirigotis, Ke Chen, Cheng-Zhi Anna Huang, Aaron Courville, Oriol Nieto, Prem Seetharaman, Justin Salamon

专题命中 长上下文与记忆 ：LLM(abstract)；language model(abstract)

Comments Accepted at ICML 2025 V2: fixed small typo on eq. 15 and eq. 17

ReTern: Exploiting Natural Redundancy and Sign Transformations for Enhanced Fault Tolerance in Compute-in-Memory based Ternary LLMs

Akul Malhotra, Sumeet Kumar Gupta

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

ATiM: Autotuning Tensor Programs for Processing-in-DRAM

Yongwon Shin, Dookyung Kang, Hyojin Sung

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Comments 17 pages, 15 figures

Cosmos: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search

Seoyoung Ko, Hyunjeong Shim, Wanju Doh, Sungmin Yun, Jinin So, Yongsuk Kwon, Sang-Soo Park, Si-Dong Roh, Minyong Yoon, Taeksang Song, Jung Ho Ahn

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Comments 4 pages, 5 figures, to appear at IEEE Computer Architecture Letters

LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrieval

Zhenyu Ning, Guangda Liu, Qihao Jin, Wenchao Ding, Minyi Guo, Jieru Zhao

机构 * School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院） ； College of Intelligent Robotics and Advanced Manufacturing, Fudan University（复旦大学智能机器人与先进制造学院） ； Guizhou Provincial Laboratory of Big Data, College of Computer Science and Technology, Guizhou University（贵州大数据省实验室，贵州大学计算机科学与技术学院） 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Exploring Next Token Prediction For Optimizing Databases

Yeasir Rayhan, Walid G. Aref

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Comments To appear at aiDM@SIGMOD'25

DyGEnc: Encoding a Sequence of Textual Scene Graphs to Reason and Answer Questions in Dynamic Scenes

Sergey Linok, Vadim Semenov, Anastasia Trunova, Oleg Bulichev, Dmitry Yudin

机构 * Center for Cognitive Modeling（认知建模中心） ； Moscow Institute of Physics and Technology（莫斯科物理技术学院） ； Innopolis University（Innopolis大学） ； AIRI 纠错

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Comments 8 pages, 5 figures, 6 tables

DiffSpec: Differential Testing with LLMs using Natural Language Specifications and Code Artifacts

Nikitha Rao, Elizabeth Gilbert, Harrison Green, Tahina Ramananandro, Nikhil Swamy, Claire Le Goues, Sarah Fakhoury

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Balancing Pipeline Parallelism with Vocabulary Parallelism

Man Tsung Yeung, Penghui Qi, Min Lin, Xinyi Wan

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Selective Attention Improves Transformer

Yaniv Leviathan, Matan Kalman, Yossi Matias

机构 * Google Research（谷歌研究） 纠错

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2025

MR. Video: "MapReduce" is the Principle for Long Video Understanding

Ziqi Pang, Yu-Xiong Wang

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；language model(abstract)

Comments Preprint

Membership Testing for Semantic Regular Expressions

Yifei Huang, Matin Amini, Alexis Le Glaunec, Konstantinos Mamouras, Mukund Raghothaman

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Forgetting Transformer: Softmax Attention with a Forget Gate

Zhixuan Lin, Evgenii Nikishin, Xu Owen He, Aaron Courville

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a conference paper at ICLR 2025; Fixed an issue with the attention map visualization

The Devil is in the Distributions: Explicit Modeling of Scene Content is Key in Zero-Shot Video Captioning

Mingkai Tian, Guorong Li, Yuankai Qi, Amin Beheshti, Javen Qinfeng Shi, Anton van den Hengel, Qingming Huang

专题命中 长上下文与记忆 ：language model(abstract)；prompting(abstract)

Comments 13 pages

Dewey Long Context Embedding Model: A Technical Report

Dun Zhang, Panxiang Zou, Yudong Zhou

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Comments 5 pages, 1 figure

MoonCast: High-Quality Zero-Shot Podcast Generation

Zeqian Ju, Dongchao Yang, Jianwei Yu, Kai Shen, Yichong Leng, Zhengtao Wang, Xu Tan, Xinyu Zhou, Tao Qin, Xiangyang Li

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI、cs.LG

EmBARDiment: an Embodied AI Agent for Productivity in XR

Riccardo Bovo, Steven Abreu, Karan Ahuja, Eric J Gonzalez, Li-Te Cheng, Mar Gonzalez-Franco

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Journal ref IEEE Virtual Reality Conference 2025

EliteKV: Scalable KV Cache Compression via RoPE Frequency Selection and Joint Low-Rank Projection

Yuhao Zhou, Sirui Song, Boyang Liu, Zhiheng Xi, Senjie Jin, Xiaoran Fan, Zhihao Zhang, Wei Li, Xuanjing Huang

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 13 pages, 8 figures

GPT4Image: Large Pre-trained Models Help Vision Models Learn Better on Perception Task

Ning Ding, Yehui Tang, Zhongqian Fu, Chao Xu, Kai Han, Yunhe Wang

专题命中 长上下文与记忆 ：LLM(abstract)；prompting(abstract)

Comments GitHub: https://github.com/huawei-noah/Efficient-Computing/tree/master/GPT4Image/

Be CIM or Be Memory: A Dual-mode-aware DNN Compiler for CIM Accelerators

Shixin Zhao, Yuming Li, Bing Li, Yintao He, Mengdi Wang, Yinhe Han, Ying Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Comments 16 pages, 18 figures, accepted to ASPLOS 2025

Memory Helps, but Confabulation Misleads: Understanding Streaming Events in Videos with MLLMs

Gengyuan Zhang, Mingcong Ding, Tong Liu, Yao Zhang, Volker Tresp

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)

Comments Short paper (5 pages)

AQUA: Network-Accelerated Memory Offloading for LLMs in Scale-Up GPU Domains

Abhishek Vijaya Kumar, Gianni Antichi, Rachee Singh

专题命中 长上下文与记忆 ：LLM(abstract)；language model(abstract)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs

Kan Zhu, Tian Tang, Qinyu Xu, Yile Gu, Zhichen Zeng, Rohan Kadekodi, Liangyu Zhao, Ang Li, Arvind Krishnamurthy, Baris Kasikci

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL、cs.AI、cs.LG