语言大模型 / LLM

Prompt-responsive Object Retrieval with Memory-augmented Student-Teacher Learning

Malte Mosbach, Sven Behnke

机构 * Autonomous Intelligent Systems Group, University of Bonn, Germany（波恩大学自主智能系统小组） ； Lamarr Institute for ML and AI, Bonn, Germany（贝勒人工智能与机器学习研究所） 纠错

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.AI、cs.LG

LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement

Zhifan Ye, Kejing Xia, Yonggan Fu, Xin Dong, Jihoon Hong, Xiangchi Yuan, Shizhe Diao, Jan Kautz, Pavlo Molchanov, Yingyan Celine Lin

机构 * Georgia Institute of Technology（佐治亚理工学院） ； NVIDIA（英伟达） 纠错

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICLR 2025

Exploring Next Token Prediction in Theory of Mind (ToM) Tasks: Comparative Experiments with GPT-2 and LLaMA-2 AI Models

Pavan Yadav, Nikhil Khandalkar, Krishna Shinde, Lokesh B. Ramegowda, Rajarshi Das

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI

Comments 75 pages, 60 figures

CacheFormer: High Attention-Based Segment Caching

Sushant Singh, Ausif Mahmood

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI、cs.LG

Transfer Learning for Temporal Link Prediction

Ayan Chatterjee, Barbara Ikica, Babak Ravandi, John Palowitch

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.AI、cs.LG

Comments 14 pages, 7 figures

PhishLang: A Real-Time, Fully Client-Side Phishing Detection Framework Using MobileBERT

Sayak Saha Roy, Shirin Nilizadeh

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI、cs.LG

Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory

Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, James Zou

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.LG

Comments https://github.com/suzgunmirac/dynamic-cheatsheet

Gemma 3 Technical Report

Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean-bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, Iain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, Christopher A. Choquette-Choo, CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimczak-Plucińska, Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wieting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju-yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Rubenstein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Põder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger, Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean-Baptiste Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, Léonard Hussenot

专题命中 长上下文与记忆 ：post-training(abstract)；分类 cs.CL、cs.AI

Masked Mixers for Language Generation and Retrieval

Benjamin L. Badger

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.LG

Comments 31 pages, 9 figures, 4 tables, 14 supplementary figures, 10 supplementary tables

Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRs

Michael Wornow, Suhana Bedi, Miguel Angel Fuentes Hernandez, Ethan Steinberg, Jason Alan Fries, Christopher Re, Sanmi Koyejo, Nigam H. Shah

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.AI、cs.LG

SCBench: A KV Cache-Centric Analysis of Long-Context Methods

Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL、cs.LG

Comments Accepted at ICLR 2025

State space models can express n-gram languages

Vinoth Nandakumar, Qiang Qu, Peng Mi, Tongliang Liu

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.LG

Comments Published in "Transactions on Machine Learning Research", 2025

Gated Delta Networks: Improving Mamba2 with Delta Rule

Songlin Yang, Jan Kautz, Ali Hatamizadeh

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2025 camera ready

Memory Mosaics

Jianyu Zhang, Niklas Nolte, Ranajoy Sadhukhan, Beidi Chen, Léon Bottou

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI、cs.LG

R$^3$Mem: Bridging Memory Retention and Retrieval via Reversible Compression

Xiaoqiang Wang, Suyuchen Wang, Yun Zhu, Bang Liu

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI

Comments Work in progress

MutaGReP: Execution-Free Repository-Grounded Plan Search for Code-Use

Zaid Khan, Ali Farhadi, Ranjay Krishna, Luca Weihs, Mohit Bansal, Tanmay Gupta

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL、cs.AI

Comments Project page: zaidkhan.me/MutaGReP

Piece of Table: A Divide-and-Conquer Approach for Selecting Subtables in Table Question Answering

Wonjin Lee, Kyumin Kim, Sungjae Lee, Jihun Lee, Kwang In Kim

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI

Redefining Robot Generalization Through Interactive Intelligence

Sharmita Dey

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.AI、cs.LG

Extractive Schema Linking for Text-to-SQL

Michael Glass, Mustafa Eyceoz, Dharmashankar Subramanian, Gaetano Rossiello, Long Vu, Alfio Gliozzo

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL、cs.AI

LeMo: Enabling LEss Token Involvement for MOre Context Fine-tuning

Tuowei Wang, Xingyu Chen, Kun Li, Ting Cao, Ju Ren, Yaoxue Zhang

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL、cs.AI

Attention is All You Need Until You Need Retention

M. Murat Yaslioglu

专题命中 长上下文与记忆 ：pretraining(abstract)；分类 cs.AI、cs.LG

Parallelizing Linear Transformers with the Delta Rule over Sequence Length

Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, Yoon Kim

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.LG

Comments Final camera ready

The Backpropagation of the Wave Network

Xin Zhang, Victor S. Sheng

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI

MSWA: Refining Local Attention with Multi-ScaleWindow Attention

Yixing Xu, Shivank Nag, Dong Li, Lu Tian, Emad Barsoum

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI

Syzygy: Dual Code-Test C to (safe) Rust Translation using LLMs and Dynamic Analysis

Manish Shetty, Naman Jain, Adwait Godbole, Sanjit A. Seshia, Koushik Sen

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.AI、cs.LG

Comments Project webpage at https://syzygy-project.github.io/. Preliminary version accepted at LLM4Code 2025, 34 pages

Memory Layers at Scale

Vincent-Pierre Berges, Barlas Oğuz, Daniel Haziza, Wen-tau Yih, Luke Zettlemoyer, Gargi Ghosh

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.AI

An Adversarial Example for Direct Logit Attribution: Memory Management in GELU-4L

Jett Janiak, Can Rager, James Dao, Yeu-Tong Lau

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI、cs.LG

Journal ref BlackboxNLP Workshop 2024, pages 232-237

Test Time Learning for Time Series Forecasting

Panayiotis Christou, Shichu Chen, Xupeng Chen, Parijat Dube

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI、cs.LG

2D Matryoshka Sentence Embeddings

Xianming Li, Zongxi Li, Jing Li, Haoran Xie, Qing Li

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.LG

Comments Decoupled with ESE

Selective Attention: Enhancing Transformer through Principled Context Control

Xuechen Zhang, Xiangyu Chang, Mingchen Li, Amit Roy-Chowdhury, Jiasi Chen, Samet Oymak

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL、cs.LG