语言大模型 / LLM

Pause-Tuning for Long-Context Comprehension: A Lightweight Approach to LLM Attention Recalibration

James Begin, Namit Agrawal, Eshan Singh, Yicheng Fu, Sean O'Brien, Vasu Sharma, Kevin Zhu

专题命中 长上下文与记忆 ：LLM(title)；language model(abstract)；分类 cs.CL、cs.AI

CompAct: Compressed Activations for Memory-Efficient LLM Training

Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki, Assaf Schuster

专题命中 长上下文与记忆 ：LLM(title)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Accepted to NAACL 2025

GIRAFFE: Design Choices for Extending the Context Length of Visual Language Models

Mukai Li, Lei Li, Shansan Gong, Qi Liu

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Working in progress

Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model

Zirui Liu, Guanchu Wang, Shaochen Zhong, Zhaozhuo Xu, Daochen Zha, Ruixiang Tang, Zhimeng Jiang, Kaixiong Zhou, Vipin Chaudhary, Shuai Xu, Xia Hu

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

One Thousand and One Pairs: A "novel" challenge for long-context language models

Marzena Karpinska, Katherine Thai, Kyle Lo, Tanya Goyal, Mohit Iyyer

专题命中 长上下文与记忆 ：language model(title)；LLM(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024, camera ready

Extracting Prompts by Inverting LLM Outputs

Collin Zhang, John X. Morris, Vitaly Shmatikov

专题命中 长上下文与记忆 ：LLM(title)；language model(abstract)；分类 cs.CL、cs.LG

DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models

Sara Vera Marjanović, Haeun Yu, Pepa Atanasova, Maria Maistro, Christina Lioma, Isabelle Augenstein

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 6 figures, Accepted to Findings of EMNLP 2024

Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contexts

Aditya Sharma, Michael Saxon, William Yang Wang

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Findings of EMNLP 2024

MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistants

Zeyu Zhang, Quanyu Dai, Luyu Chen, Zeren Jiang, Rui Li, Jieming Zhu, Xu Chen, Yi Xie, Zhenhua Dong, Ji-Rong Wen

专题命中 长上下文与记忆 ：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 25 tables, 1 figure

Beyond Memorization: The Challenge of Random Memory Access in Language Models

Tongyao Zhu, Qian Liu, Liang Pang, Zhengbao Jiang, Min-Yen Kan, Min Lin

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 4 figures; accepted by ACL 2024 (oral)

Pre-Trained Foundation Model representations to uncover Breathing patterns in Speech

Vikramjit Mitra, Anirban Chatterjee, Ke Zhai, Helen Weng, Ayuko Hill, Nicole Hay, Christopher Webb, Jamie Cheng, Erdrin Azemi

专题命中 长上下文与记忆 ：foundation model(title,abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 6 figures, BioKDD workshop paper

B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory

Luca Zancato, Arjun Seshadri, Yonatan Dukler, Aditya Golatkar, Yantao Shen, Benjamin Bowman, Matthew Trager, Alessandro Achille, Stefano Soatto

专题命中 长上下文与记忆 ：foundation model(title)；language model(abstract)；分类 cs.CL、cs.LG

Extending Input Contexts of Language Models through Training on Segmented Sequences

Petros Karypis, Julian McAuley, George Karypis

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 11 pages, 3 figures

An Empirical Study of Mamba-based Language Models

Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, Garvit Kulshreshtha, Vartika Singh, Jared Casper, Jan Kautz, Mohammad Shoeybi, Bryan Catanzaro

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Presentations are not always linear! GNN meets LLM for Document-to-Presentation Transformation with Attribution

Himanshu Maheshwari, Sambaran Bandyopadhyay, Aparna Garimella, Anandhavelu Natarajan

专题命中 长上下文与记忆 ：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments This paper is under review in a conference

Rehearsal-Free Modular and Compositional Continual Learning for Language Models

Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, Hinrich Schütze

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models

Yabin Zhang, Wenjie Zhu, Hui Tang, Zhiyuan Ma, Kaiyang Zhou, Lei Zhang

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments CVPR2024; Codes are available at \url{https://github.com/YBZh/DMN}

TensorBank: Tensor Lakehouse for Foundation Model Training

Romeo Kienzler, Leonardo Pondian Tizzei, Benedikt Blumenstiel, Zoltan Arnold Nagy, S. Karthik Mukkavilli, Johannes Schmude, Marcus Freitag, Michael Behrendt, Daniel Salles Civitarese, Naomi Simumba, Daiki Kimura, Hendrik Hamann

专题命中 长上下文与记忆 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

Hong Liu, Zhiyuan Li, David Hall, Percy Liang, Tengyu Ma

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models

Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 42 figures, 4 tables

Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models

Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Qiuxia Li, Jun Zhao

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at LREC-COLING 2024

Compressed Context Memory For Online Language Model Interaction

Jang-Hyun Kim, Junyoung Yeom, Sangdoo Yun, Hyun Oh Song

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ICLR 2024. Add streaming setting results and training set analyses

Generative Spoken Language Model based on continuous word-sized audio tokens

Robin Algayres, Yossi Adi, Tu Anh Nguyen, Jade Copet, Gabriel Synnaeve, Benoit Sagot, Emmanuel Dupoux

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Conference paper at EMNLP 2023

Attention Sorting Combats Recency Bias In Long Context Language Models

Alexander Peysakhovich, Adam Lerer

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

RCMHA: Relative Convolutional Multi-Head Attention for Natural Language Modelling

Herman Sugiharto, Aradea, Husni Mubarok

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 13 pages, 13 figures, 6 tables

Leveraging Vision-Language Models for Granular Market Change Prediction

Christopher Wimmer, Navid Rekabsaz

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at Multimodal AI for Financial Forecasting Workshop (Muffin) at AAAI 2023

Training Language Models with Memory Augmentation

Zexuan Zhong, Tao Lei, Danqi Chen

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2022. Our code and models are available at https://github.com/princeton-nlp/TRIME

Transformer-Based Language Models for Software Vulnerability Detection

Chandra Thapa, Seung Ick Jang, Muhammad Ejaz Ahmed, Seyit Camtepe, Josef Pieprzyk, Surya Nepal

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments 16 pages

Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models

Patrick Huber, Giuseppe Carenini

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 9 pages

Journal ref In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)

Relational Memory Augmented Language Models

Qi Liu, Dani Yogatama, Phil Blunsom

专题命中 长上下文与记忆 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to TACL, pre MIT Press publication version