语言大模型 / LLM

QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning

Fanqi Wan, Weizhou Shen, Shengyi Liao, Yingcheng Shi, Chenliang Li, Ziyi Yang, Ji Zhang, Fei Huang, Jingren Zhou, Ming Yan

专题命中 长上下文与记忆 ：SFT(abstract)；分类 cs.CL

Comments Technical Report

Memory-Efficient Super-Resolution of 3D Micro-CT Images Using Octree-Based GANs: Enhancing Resolution and Segmentation Accuracy

Evgeny Ugolkov, Xupeng He, Hyung Kwak, Hussein Hoteit

专题命中 长上下文与记忆 ：post-training(abstract)；分类 cs.LG

Comments 31 pages, 15 figures

MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention

Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Comments Accepted at ICML 2025

In-Context Learning Boosts Speech Recognition via Human-like Adaptation to Speakers and Language Varieties

Nathan Roll, Calbert Graham, Yuka Tatsumi, Kim Tien Nguyen, Meghan Sumner, Dan Jurafsky

机构 * Stanford University（斯坦福大学） ； University of Cambridge（剑桥大学） 纠错

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments 15 pages; 3 figures

Subquadratic Algorithms and Hardness for Attention with Any Temperature

Shreya Gupta, Boyang Huang, Barna Saha, Yinzhan Xu, Christopher Ye

机构 * University of California, San Diego（加州大学圣地亚哥分校） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.LG

Comments 34 pages, 2 figures, abstract shortened to meet arXiv requirements

Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)

Artem Riabinin, Egor Shulgin, Kaja Gruntkowska, Peter Richtárik

机构 * King Abdullah University of Science and Technology (KAUST)（国王阿卜杜勒阿齐兹大学科学与技术学院） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.LG

$\infty$-Video: A Training-Free Approach to Long Video Understanding via Continuous-Time Memory Consolidation

Saul Santos, António Farinhas, Daniel C. McNamee, André F. T. Martins

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Comments 17 pages, 7 figures

SPECTRE: An FFT-Based Efficient Drop-In Replacement to Self-Attention for Long Contexts

Jacob Fein-Ashley, Neelesh Gupta, Rajgopal Kannan, Viktor Prasanna

机构 * University of Southern California（南加州大学） ； DEVCOM ARL Army Research Office（DEVCOM ARL陆军研究办公室） 纠错

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Hierarchical Document Refinement for Long-context Retrieval-augmented Generation

Jiajie Jin, Xiaoxi Li, Guanting Dong, Yuyao Zhang, Yutao Zhu, Yongkang Wu, Zhonghua Li, Qi Ye, Zhicheng Dou

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院） ； Huawei Poisson Lab（华为Poisson实验室） 纠错

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.CL

Securing Agentic AI: A Comprehensive Threat Model and Mitigation Framework for Generative AI Agents

Vineeth Sai Narajala, Om Narayan

机构 * Amazon Web Services（亚马逊网络服务） 纠错

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.AI

Comments 12 pages, 2 figures, 1 table, typos corrected, references added

Estimating Optimal Context Length for Hybrid Retrieval-augmented Multi-document Summarization

Adithya Pratapa, Teruko Mitamura

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

MrSteve: Instruction-Following Agents in Minecraft with What-Where-When Memory

Junyeong Park, Junmo Cho, Sungjin Ahn

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.LG

Comments Accepted to ICLR 2025

Endo3R: Unified Online Reconstruction from Dynamic Monocular Endoscopic Video

Jiaxin Guo, Wenzhen Dong, Tianyu Huang, Hao Ding, Ziyi Wang, Haomin Kuang, Qi Dou, Yun-Hui Liu

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.AI

SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysis

Junho Kim, Hyunjun Kim, Hosu Lee, Yong Man Ro

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.AI

Comments Project page: https://ivy-lvlm.github.io/SALOVA/

Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning

Shengyuan Hu, Yiwei Fu, Zhiwei Steven Wu, Virginia Smith

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.LG

Comments ICLR 2025, 32 pages, 8 figures, 9 tables

Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data

Seiji Maekawa, Hayate Iso, Nikita Bhutani

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments ICLR2025

See What You Are Told: Visual Attention Sink in Large Multimodal Models

Seil Kang, Jinyeong Kim, Junhyeok Kim, Seong Jae Hwang

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI

Unlocking a New Rust Programming Experience: Fast and Slow Thinking with LLMs to Conquer Undefined Behaviors

Renshuang Jiang, Pan Dong, Zhenling Duan, Yu Shi, Xiaoxiang Fang, Yan Ding, Jun Ma, Shuai Zhao, Zhe Jiang

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL

T-REX: A 68-567 μs/token, 0.41-3.95 μJ/token Transformer Accelerator with Reduced External Memory Access and Enhanced Hardware Utilization in 16nm FinFET

Seunghyun Moon, Mao Li, Gregory Chen, Phil Knag, Ram Krishnamurthy, Mingoo Seok

专题命中 长上下文与记忆 ：post-training(abstract)；分类 cs.AI

Comments Accepted to IEEE ISSCC 2025

Wavelet-based Positional Representation for Long Context

Yui Oka, Taku Hasegawa, Kyosuke Nishida, Kuniko Saito

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments Accepted to ICLR 2025. 28 pages, 11 figures

World Model on Million-Length Video And Language With Blockwise RingAttention

Hao Liu, Wilson Yan, Matei Zaharia, Pieter Abbeel

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Mamba Fusion: Learning Actions Through Questioning

Zhikang Dong, Apoorva Beedu, Jason Sheinkopf, Irfan Essa

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI

Towards Lifelong Dialogue Agents via Timeline-based Memory Management

Kai Tzu-iunn Ong, Namyoung Kim, Minju Gwak, Hyungjoo Chae, Taeyoon Kwon, Yohan Jo, Seung-won Hwang, Dongha Lee, Jinyoung Yeo

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL

Comments Accepted to NAACL 2025

vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention

Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.LG

Comments To appear in ASPLOS 2025

Cognitive Kernel: An Open-source Agent System towards Generalist Autopilots

Hongming Zhang, Xiaoman Pan, Hongwei Wang, Kaixin Ma, Wenhao Yu, Dong Yu

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.AI

Memory makes computation universal, remember?

Erik Garrison

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Low-Resource Fast Text Classification Based on Intra-Class and Inter-Class Distance Calculation

Yanxu Mao, Peipei Liu, Tiehan Cui, Congying Liu, Datao You

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective

Meizhi Zhong, Chen Zhang, Yikun Lei, Xikai Liu, Yan Gao, Yao Hu, Kehai Chen, Min Zhang

专题命中 长上下文与记忆 ：pretraining(abstract)；分类 cs.CL

Enhancing Post-Hoc Attributions in Long Document Comprehension via Coarse Grained Answer Decomposition

Pritika Ramu, Koustava Goswami, Apoorv Saxena, Balaji Vasan Srinivasan

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL

Comments EMNLP 2024

AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers

Jake Grigsby, Justin Sasek, Samyak Parajuli, Daniel Adebi, Amy Zhang, Yuke Zhu

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Comments NeurIPS 2024