语言大模型 / LLM - arXivDaily 专题

2309.00236 2024-09-19 cs.LG cs.CL cs.CR 73%

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons

专题命中长上下文与记忆：language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

Comments Project page at https://image-hijacks.github.io

2405.00790 2024-09-17 cs.AR cs.AI cs.DC cs.LG cs.PF 73%

SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators

Mohanad Odema, Luke Chen, Hyoukjun Kwon, Mohammad Abdullah Al Faruque

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments MICRO'24

2408.16967 2024-09-02 cs.CL cs.AI 73%

MemLong: Memory-Augmented Retrieval for Long Text Modeling

Weijie Liu, Zecheng Tang, Juntao Li, Kehai Chen, Min Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2408.12570 2024-08-23 cs.CL cs.LG 73%

Jamba-1.5: Hybrid Transformer-Mamba Models at Scale

Jamba Team, Barak Lenz, Alan Arazi, Amir Bergman, Avshalom Manevich, Barak Peleg, Ben Aviram, Chen Almagor, Clara Fridman, Dan Padnos, Daniel Gissin, Daniel Jannai, Dor Muhlgay, Dor Zimberg, Edden M Gerber, Elad Dolev, Eran Krakovsky, Erez Safahi, Erez Schwartz, Gal Cohen, Gal Shachaf, Haim Rozenblum, Hofit Bata, Ido Blass, Inbal Magar, Itay Dalmedigos, Jhonathan Osin, Julie Fadlon, Maria Rozman, Matan Danos, Michael Gokhman, Mor Zusman, Naama Gidron, Nir Ratner, Noam Gat, Noam Rozen, Oded Fried, Ohad Leshno, Omer Antverg, Omri Abend, Opher Lieber, Or Dagan, Orit Cohavi, Raz Alon, Ro'i Belson, Roi Cohen, Rom Gilad, Roman Glozman, Shahar Lev, Shaked Meirom, Tal Delbari, Tal Ness, Tomer Asida, Tom Ben Gal, Tom Braude, Uriya Pumerantz, Yehoshua Cohen, Yonatan Belinkov, Yuval Globerson, Yuval Peleg Levy, Yoav Shoham

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Webpage: https://www.ai21.com/jamba

2311.09198 2024-08-15 cs.CL cs.AI 73%

Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training

Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song, Yibo Liu, Qianguo Sun, Yuxin Liang, Hao Wang, Enming Zhang, Jiaxing Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 main conference

2408.00274 2024-08-02 cs.CL cs.AI 73%

QUITO: Accelerating Long-Context Reasoning through Query-Guided Context Compression

Wenshan Wang, Yihang Wang, Yixing Fan, Huaming Liao, Jiafeng Guo

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2407.08608 2024-07-16 cs.LG cs.AI 73%

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, Tri Dao

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2404.02060 2024-06-13 cs.CL cs.AI 73%

Long-context LLMs Struggle with Long In-context Learning

Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, Wenhu Chen

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2406.06556 2024-06-12 cs.CL cs.AI 73%

Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach

Sambaran Bandyopadhyay, Himanshu Maheshwari, Anandhavelu Natarajan, Apoorv Saxena

专题命中长上下文与记忆：LLM(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

2406.06124 2024-06-11 cs.CL cs.AI 73%

Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation

Aadharsh Aadhithya A, Sachin Kumar S, Soman K. P

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 6 pages, 2 figures

2406.06025 2024-06-11 cs.SE cs.CL cs.LG 73%

RepoQA: Evaluating Long Context Code Understanding

Jiawei Liu, Jia Le Tian, Vijay Daita, Yuxiang Wei, Yifeng Ding, Yuhan Katherine Wang, Jun Yang, Lingming Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2310.10195 2024-06-07 cs.LG cs.CL 73%

AdaLomo: Low-memory Optimization with Adaptive Learning Rate

Kai Lv, Hang Yan, Qipeng Guo, Haijun Lv, Xipeng Qiu

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments ACL 2024 camera ready version

2406.00057 2024-06-06 cs.CL cs.LG 73%

Toward Conversational Agents with Context and Time Sensitive Long-term Memory

Nick Alonso, Tomás Figliolia, Anthony Ndirango, Beren Millidge

专题命中长上下文与记忆：language model(abstract)；prompting(abstract)；分类 cs.CL、cs.LG

2405.12981 2024-05-22 cs.LG cs.CL 73%

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

William Brandon, Mayank Mishra, Aniruddha Nrusimha, Rameswar Panda, Jonathan Ragan Kelly

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2405.12035 2024-05-21 cs.AI cs.CL cs.IR 73%

KG-RAG: Bridging the Gap Between Knowledge and Creativity

Diego Sanmartin

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2404.10308 2024-04-17 cs.LG cs.AI 73%

Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs

Woomin Song, Seunghyuk Oh, Sangwoo Mo, Jaehyung Kim, Sukmin Yun, Jung-Woo Ha, Jinwoo Shin

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted to ICLR 2024. The first two authors contributed equally

2404.08674 2024-04-16 cs.CL cs.AI cs.HC 73%

Effects of Different Prompts on the Quality of GPT-4 Responses to Dementia Care Questions

Zhuochun Li, Bo Xie, Robin Hilsabeck, Alyssa Aguirre, Ning Zou, Zhimeng Luo, Daqing He

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2403.04787 2024-04-09 cs.CL cs.AI 73%

Ever-Evolving Memory by Blending and Refining the Past

Seo Hyun Kim, Keummin Ka, Yohan Jo, Seung-won Hwang, Dongha Lee, Jinyoung Yeo

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 4 figures, 7 tables

2310.03294 2024-04-02 cs.LG cs.AI cs.DC 73%

DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training

Dacheng Li, Rulin Shao, Anze Xie, Eric P. Xing, Xuezhe Ma, Ion Stoica, Joseph E. Gonzalez, Hao Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2305.16344 2024-03-08 cs.CL cs.AI 73%

Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs

Chongjian Yue, Xinrun Xu, Xiaojun Ma, Lun Du, Hengyu Liu, Zhiming Ding, Yanbing Jiang, Shi Han, Dongmei Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2402.15923 2024-02-27 cs.LG cs.AI cs.MM 73%

Predicting Outcomes in Video Games with Long Short Term Memory Networks

Kittimate Chulajata, Sean Wu, Fabien Scalzo, Eun Sang Cha

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 7 pages, 2 Figures, 2 Tables. Kittimate Chulajata and Sean Wu are considered co-first authors

2309.10400 2024-02-22 cs.CL cs.LG 73%

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training

Dawei Zhu, Nan Yang, Liang Wang, Yifan Song, Wenhao Wu, Furu Wei, Sujian Li

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2024

2311.14652 2024-02-06 cs.LG cs.CL stat.ML 73%

One Pass Streaming Algorithm for Super Long Token Attention Approximation in Sublinear Space

Raghav Addanki, Chenyang Li, Zhao Song, Chiwun Yang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2401.09486 2024-02-06 cs.LG cs.CL 73%

LoMA: Lossless Compressed Memory Attention

Yumeng Wang, Zhenyang Xiao

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2305.03731 2024-02-05 cs.AI cs.CL q-bio.NC 73%

Working Memory Capacity of ChatGPT: An Empirical Study

Dongyu Gong, Xingchen Wan, Dingmin Wang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at the 38th AAAI Conference on Artificial Intelligence (AAAI-24)

2311.03498 2023-12-20 cs.CL cs.LG 73%

In-Context Exemplars as Clues to Retrieving from Large Associative Memory

Jiachen Zhao

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Presented at Neural Conversational AI @ ICML 2023 and Associative Memory & Hopfield Networks @ NeurIPS 2023

2312.08402 2023-12-15 cs.LG cs.AI 73%

LDM$^2$: A Large Decision Model Imitating Human Cognition with Dynamic Memory Enhancement

Xingjin Wang, Linjing Li, Daniel Zeng

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Findings of the Association for Computational Linguistics: EMNLP 2023

2312.07305 2023-12-13 cs.CL cs.AI 73%

SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion

Yuxiang Guo

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments work in progress

2309.10954 2023-12-07 cs.CL cs.LG 73%

In-Context Learning for Text Classification with Many Labels

Aristides Milios, Siva Reddy, Dzmitry Bahdanau

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 12 pages, 4 figures

2310.05869 2023-12-04 cs.LG cs.AI 73%

HyperAttention: Long-context Attention in Near-Linear Time

Insu Han, Rajesh Jayaram, Amin Karbasi, Vahab Mirrokni, David P. Woodruff, Amir Zandieh

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

1. 长上下文与记忆 4789 篇

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators

MemLong: Memory-Augmented Retrieval for Long Text Modeling

Jamba-1.5: Hybrid Transformer-Mamba Models at Scale

Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training

QUITO: Accelerating Long-Context Reasoning through Query-Guided Context Compression

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

Long-context LLMs Struggle with Long In-context Learning

Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach

Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation

RepoQA: Evaluating Long Context Code Understanding

AdaLomo: Low-memory Optimization with Adaptive Learning Rate

Toward Conversational Agents with Context and Time Sensitive Long-term Memory

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

KG-RAG: Bridging the Gap Between Knowledge and Creativity

Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs

Effects of Different Prompts on the Quality of GPT-4 Responses to Dementia Care Questions

Ever-Evolving Memory by Blending and Refining the Past

DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training

Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs

Predicting Outcomes in Video Games with Long Short Term Memory Networks

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training

One Pass Streaming Algorithm for Super Long Token Attention Approximation in Sublinear Space

LoMA: Lossless Compressed Memory Attention

Working Memory Capacity of ChatGPT: An Empirical Study

In-Context Exemplars as Clues to Retrieving from Large Associative Memory

LDM$^2$: A Large Decision Model Imitating Human Cognition with Dynamic Memory Enhancement

SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion

In-Context Learning for Text Classification with Many Labels

HyperAttention: Long-context Attention in Near-Linear Time