语言大模型 / LLM

Transformer verbatim in-context retrieval across time and scale

Kristijan Armeni, Marko Pranjić, Senja Pollak

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments accepted to Conference on Natural Language Learning 2024 (https://www.conll.org/)

Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Training

Cheng Luo, Jiawei Zhao, Zhuoming Chen, Beidi Chen, Anima Anandkumar

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.LG

Summarizing long regulatory documents with a multi-step pipeline

Mika Sie, Ruby Beek, Michiel Bots, Sjaak Brinkkemper, Albert Gatt

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments Published in: Proceedings of the 6th Workshop on Natural Legal Language Processing (NLLP 2024)

Pixtral 12B

Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, Soham Ghosh, Amélie Héliou, Paul Jacob, Albert Q. Jiang, Kartik Khandelwal, Timothée Lacroix, Guillaume Lample, Diego Las Casas, Thibaut Lavril, Teven Le Scao, Andy Lo, William Marshall, Louis Martin, Arthur Mensch, Pavankumar Muddireddy, Valera Nemychnikova, Marie Pellat, Patrick Von Platen, Nikhil Raghuraman, Baptiste Rozière, Alexandre Sablayrolles, Lucile Saulnier, Romain Sauvestre, Wendy Shang, Roman Soletskyi, Lawrence Stewart, Pierre Stock, Joachim Studnia, Sandeep Subramanian, Sagar Vaze, Thomas Wang, Sophia Yang

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

The Effect of Surprisal on Reading Times in Information Seeking and Repeated Reading

Keren Gruteke Klein, Yoav Meiri, Omer Shubi, Yevgeni Berzak

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments Accepted to CoNLL

Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-context Models

Xinyu Liu, Runsong Zhao, Pengcheng Huang, Chunyang Xiao, Bei Li, Jingang Wang, Tong Xiao, Jingbo Zhu

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Enhancing Retrieval in QA Systems with Derived Feature Association

Keyush Shah, Abhishek Goyal, Isaac Wasserman

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL

Deep Learning based Optical Image Super-Resolution via Generative Diffusion Models for Layerwise in-situ LPBF Monitoring

Francis Ogoke, Sumesh Kalambettu Suresh, Jesse Adamczyk, Dan Bolintineanu, Anthony Garland, Michael Heiden, Amir Barati Farimani

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.LG

Retrofitting Temporal Graph Neural Networks with Transformer

Qiang Huang, Xiao Yan, Xin Wang, Susie Xi Rao, Zhichao Han, Fangcheng Fu, Wentao Zhang, Jiawei Jiang

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Comments conference Under review

Linear Recency Bias During Training Improves Transformers' Fit to Reading Times

Christian Clark, Byung-Doh Oh, William Schuler

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Agent Workflow Memory

Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, Graham Neubig

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Retrieval Or Holistic Understanding? Dolce: Differentiate Our Long Context Evaluation Tasks

Zi Yang

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.CL

Code-switching in text and speech reveals information-theoretic audience design

Debasmita Bhattacharya, Marten van Schijndel

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments Submitted to Journal of Memory and Language on 7 June 2024

SR-CIS: Self-Reflective Incremental System with Decoupled Memory and Reasoning

Biqing Qi, Junqi Gao, Xinquan Chen, Dong Li, Weinan Zhang, Bowen Zhou

专题命中 长上下文与记忆 ：post-training(abstract)；分类 cs.AI

xT: Nested Tokenization for Larger Context in Large Images

Ritwik Gupta, Shufan Li, Tyler Zhu, Jitendra Malik, Trevor Darrell, Karttikeya Mangalam

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI

Comments Accepted to the 2024 International Conference on Machine Learning (ICML)

Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs

Hao-Tien Lewis Chiang, Zhuo Xu, Zipeng Fu, Mithun George Jacob, Tingnan Zhang, Tsang-Wei Edward Lee, Wenhao Yu, Connor Schenck, David Rendleman, Dhruv Shah, Fei Xia, Jasmine Hsu, Jonathan Hoech, Pete Florence, Sean Kirmani, Sumeet Singh, Vikas Sindhwani, Carolina Parada, Chelsea Finn, Peng Xu, Sergey Levine, Jie Tan

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI

Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments

Djamahl Etchegaray, Zi Huang, Tatsuya Harada, Yadan Luo

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.AI

Comments To appear in ECCV 2024. Source code: https://github.com/djamahl99/findnpropagate

Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models

Kyra Ahrens, Hans Hergen Lehmann, Jae Hee Lee, Stefan Wermter

专题命中 长上下文与记忆 ：foundation model(abstract)；分类 cs.LG

Comments Accepted for publication in Transactions of Machine Learning Research (TMLR) journal

Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations

Nuo Chen, Hongguang Li, Juhua Huang, Baoyuan Wang, Jia Li

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments 17pages, 5 figures

Controllable Text Generation with Residual Memory Transformer

Hanqing Zhang, Sun Si, Haiming Wu, Dawei Song

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments github:https://github.com/littlehacker26/Residual_Memory_Transformer

Journal ref ACL 2024

Positive concave deep equilibrium models

Mateusz Gabor, Tomasz Piotrowski, Renato L. G. Cavalcante

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

The Role of Global and Local Context in Named Entity Recognition

Arthur Amalvy, Vincent Labatut, Richard Dufour

专题命中 长上下文与记忆 ：prompting(abstract)；分类 cs.CL

Comments Accepted to ACL2023

Journal ref 61st Annual Meeting of the Association for Computational Linguistics, 2023, p.714-722

Towards Exact Gradient-based Training on Analog In-memory Computing

Zhaoxian Wu, Tayfun Gokmen, Malte J. Rasch, Tianyi Chen

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Comments 10 pages, 5 figures,2 tables

3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding

Xindian Ma, Wenyuan Liu, Peng Zhang, Nan Xu

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

SMR: State Memory Replay for Long Sequence Modeling

Biqing Qi, Junqi Gao, Kaiyan Zhang, Dong Li, Jianxing Liu, Ligang Wu, Bowen Zhou

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Journal ref Findings of the Association for Computational Linguistics, 2024

Length Generalization of Causal Transformers without Position Encoding

Jie Wang, Tao Ji, Yuanbin Wu, Hang Yan, Tao Gui, Qi Zhang, Xuanjing Huang, Xiaoling Wang

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective

Zhen Qin, Xuyang Shen, Dong Li, Weigao Sun, Stan Birchfield, Richard Hartley, Yiran Zhong

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments Technical report. Yiran Zhong is the corresponding author

Dynamic Activation Pitfalls in LLaMA Models: An Empirical Study

Chi Ma, Mincong Huang, Chao Wang, Yujie Wang, Lei Yu

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.LG

Length-Aware Multi-Kernel Transformer for Long Document Classification

Guangzeng Han, Jack Tsao, Xiaolei Huang

专题命中 长上下文与记忆 ：language model(abstract)；分类 cs.CL

Comments Accepted to SEM 2024

A Global Data-Driven Model for The Hippocampus and Nucleus Accumbens of Rat From The Local Field Potential Recordings (LFP)

Maedeh Sadeghi, Mahdi Aliyari Shoorehdeli, Shole jamali, Abbas Haghparast

专题命中 长上下文与记忆 ：LLM(abstract)；分类 cs.LG