语言大模型 / LLM

Implicit In-context Learning

Zhuowei Li, Zihao Xu, Ligong Han, Yunhe Gao, Song Wen, Di Liu, Hao Wang, Dimitris N. Metaxas

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

The Role of Sparsity for Length Generalization in Transformers

Noah Golowich, Samy Jelassi, David Brandfonbrener, Sham M. Kakade, Eran Malach

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

On the Robustness of Transformers against Context Hijacking for Linear Classification

Tianle Li, Chenyang Zhang, Xingwu Chen, Yuan Cao, Difan Zou

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

MoBA: Mixture of Block Attention for Long-Context LLMs

Enzhe Lu, Zhejun Jiang, Jingyuan Liu, Yulun Du, Tao Jiang, Chao Hong, Shaowei Liu, Weiran He, Enming Yuan, Yuzhi Wang, Zhiqi Huang, Huan Yuan, Suting Xu, Xinran Xu, Guokun Lai, Yanru Chen, Huabin Zheng, Junjie Yan, Jianlin Su, Yuxin Wu, Neo Y. Zhang, Zhilin Yang, Xinyu Zhou, Mingxing Zhang, Jiezhong Qiu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 15 pages

RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations

Zunhai Su, Zhe Chen, Wang Shen, Hanyu Wei, Linge Li, Huangqi Yu, Kehong Yuan

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Profiling Apple Silicon Performance for ML Training

Dahua Feng, Zhiming Xu, Rongxiang Wang, Felix Xiaozhu Lin

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Adjoint sharding for very long context training of state space models

Xingzi Xu, Amir Tavanaei, Kavosh Asadi, Karim Bouyarmane

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Casevo: A Cognitive Agents and Social Evolution Simulator

Zexun Jiang, Yafang Shi, Maoxu Li, Hongjiang Xiao, Yunxiao Qin, Qinglan Wei, Ye Wang, Yuan Zhang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Ultra-High-Definition Dynamic Multi-Exposure Image Fusion via Infinite Pixel Learning

Xingchi Chen, Zhuoran Zheng, Xuerui Li, Yuying Chen, Shu Wang, Wenqi Ren

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Interactive AI with Retrieval-Augmented Generation for Next Generation Networking

Ruichen Zhang, Hongyang Du, Yinqiu Liu, Dusit Niyato, Jiawen Kang, Sumei Sun, Xuemin Shen, H. Vincent Poor

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 10 pages, 4 figures

Eigen Attention: Attention in Low-Rank Space for KV Cache Compression

Utkarsh Saxena, Gobinda Saha, Sakshi Choudhary, Kaushik Roy

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 page, 6 figures, 6 tables

Positive Experience Reflection for Agents in Interactive Text Environments

Philip Lippmann, Matthijs T. J. Spaan, Jie Yang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To appear at NeurIPS 2024 Language Gamification workshop

IANUS: Integrated Accelerator based on NPU-PIM Unified Memory System

Minseok Seo, Xuan Truong Nguyen, Seok Joong Hwang, Yongkee Kwon, Guhyun Kim, Chanwook Park, Ilkon Kim, Jaehan Park, Jeongbin Kim, Woojae Shin, Jongsoon Won, Haerang Choi, Kyuyoung Kim, Daehan Kwon, Chunseok Jeong, Sangheon Lee, Yongseok Choi, Wooseok Byun, Seungcheol Baek, Hyuk-Jae Lee, John Kim

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Updated version of the paper accepted to ASPLOS 2024

Journal ref ASPLOS 2024

Selection-p: Self-Supervised Task-Agnostic Prompt Compression for Faithfulness and Transferability

Tsz Ting Chung, Leyang Cui, Lemao Liu, Xinting Huang, Shuming Shi, Dit-Yan Yeung

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 14 pages, 5 figures, 10 tables, EMNLP 2024 Findings

CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios

Luning Wang, Shiyao Li, Xuefei Ning, Zhihang Yuan, Shengen Yan, Guohao Dai, Yu Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 4th NeurIPS Efficient Natural Language and Speech Processing Workshop (ENLSP-IV 2024)

Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach

Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2024 industry track

Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning

Chengsong Huang, Langlin Huang, Jiaxin Huang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Composite Learning Units: Generalized Learning Beyond Parameter Updates to Transform LLMs into Adaptive Reasoners

Santosh Kumar Radha, Oktay Goktas

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Optimal Workload Placement on Multi-Instance GPUs

Bekir Turkkan, Pavankumar Murali, Pavithra Harsha, Rohan Arora, Gerard Vanloo, Chandra Narayanaswami

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 14 pages

Language to Map: Topological map generation from natural language path instructions

Hideki Deguchi, Kazuki Shibata, Shun Taguchi

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 7 pages, 7 figures. Accepted to IEEE International Conference on Robotics and Automation (ICRA) 2024

BMX: Entropy-weighted Similarity and Semantic-enhanced Lexical Search

Xianming Li, Julius Lipp, Aamir Shakir, Rui Huang, Jing Li

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments correct the affiliation order

Ascend-CC: Confidential Computing on Heterogeneous NPU for Emerging Generative AI Workloads

Aritra Dhar, Clément Thorens, Lara Magdalena Lazier, Lukas Cavigelli

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding

Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, Qing Li

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments ECCV-24; Project page: videoagent.github.io; First two authors contributed equally

DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs

Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, Yu-Gang Jiang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Project Page: https://deepstack-vl.github.io/

LOLAMEME: Logic, Language, Memory, Mechanistic Framework

Jay Desai, Xiaobo Guo, Srinivasan H. Sengamedu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments https://openreview.net/pdf?id=73dhbcXxtV

MileBench: Benchmarking MLLMs in Long Context

Dingjie Song, Shunian Chen, Guiming Hardy Chen, Fei Yu, Xiang Wan, Benyou Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 31 pages, 13 figures, 14 tables; We add results of GPT-4o in this version

TransformerFAM: Feedback attention is working memory

Dongseong Hwang, Weiran Wang, Zhuoyuan Huo, Khe Chai Sim, Pedro Moreno Mengibar

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 26 pages, 12 figures, 14 tables

Traffic Performance GPT (TP-GPT): Real-Time Data Informed Intelligent ChatBot for Transportation Surveillance and Management

Bingzhang Wang, Zhiyu Cai, Muhammad Monjurul Karim, Chenxi Liu, Yinhai Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 8 pages, 5 figures, submitted to 27th IEEE International Conference on Intelligent Transportation Systems (IEEE ITSC 2024)

MovieChat+: Question-aware Sparse Memory for Long Video Question Answering

Enxin Song, Wenhao Chai, Tian Ye, Jenq-Neng Hwang, Xi Li, Gaoang Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Unlocking Memories with AI: Exploring the Role of AI-Generated Cues in Personal Reminiscing

Jun Li Jeung, Janet Yi-Ching Huang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)