语言大模型 / LLM - arXivDaily 专题

2306.13421 2024-07-23 cs.CL 79%

Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval

Ohad Rubin, Jonathan Berant

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments Accepted to TACL 2024

2407.12784 2024-07-18 cs.LG cs.CR cs.IR 79%

AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases

Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, Bo Li

专题命中长上下文与记忆：LLM(title,abstract)；分类 cs.LG

Comments 22 pages, 13 figures, 7 tables

2406.09444 2024-06-24 eess.AS cs.CL cs.SD 79%

GenDistiller: Distilling Pre-trained Language Models based on an Autoregressive Generative Model

Yingying Gao, Shilei Zhang, Chao Deng, Junlan Feng

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments arXiv admin note: text overlap with arXiv:2310.13418

2402.17682 2024-06-14 cs.CL 79%

NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents

Tamara Czinczoll, Christoph Hönes, Maximilian Schall, Gerard de Melo

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments accepted at ACL 2024; camera-ready version; 9 pages

2402.12363 2024-06-11 cs.CL 79%

Emergent Word Order Universals from Cognitively-Motivated Language Models

Tatsuki Kuribayashi, Ryo Ueda, Ryo Yoshida, Yohei Oseki, Ted Briscoe, Timothy Baldwin

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL 2024 main conference, 22 pages

2310.15494 2023-12-21 cs.CL 79%

TRAMS: Training-free Memory Selection for Long-range Language Modeling

Haofei Yu, Cunxiang Wang, Yue Zhang, Wei Bi

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments Findings of EMNLP 2023

2312.07813 2023-12-14 cs.OS cs.LG 79%

On a Foundation Model for Operating Systems

Divyanshu Saxena, Nihal Sharma, Donghyun Kim, Rohit Dwivedula, Jiayi Chen, Chenxi Yang, Sriram Ravula, Zichao Hu, Aditya Akella, Sebastian Angel, Joydeep Biswas, Swarat Chaudhuri, Isil Dillig, Alex Dimakis, P. Brighten Godfrey, Daehyeok Kim, Chris Rossbach, Gang Wang

专题命中长上下文与记忆：foundation model(title,abstract)；分类 cs.LG

Comments Machine Learning for Systems Workshop at 37th NeurIPS Conference, 2023, New Orleans, LA, USA

2311.05997 2023-12-01 cs.AI 79%

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models

Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang, Xiaojian Ma, Yitao Liang

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.AI

Comments update project page

2307.03172 2023-11-22 cs.CL 79%

Lost in the Middle: How Language Models Use Long Contexts

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments 18 pages, 16 figures. Accepted for publication in Transactions of the Association for Computational Linguistics (TACL), 2023

2311.10768 2023-11-21 cs.CL 79%

Memory Augmented Language Models through Mixture of Word Experts

Cicero Nogueira dos Santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David Uthus

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments 14 pages

2305.14788 2023-11-07 cs.CL 79%

Adapting Language Models to Compress Contexts

Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023; added results for Llama-2-7B model

2311.00475 2023-11-02 cs.CL 79%

Style Locality for Controllable Generation with kNN Language Models

Gilles Nawezi, Lucie Flek, Charles Welch

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments Accepted to TamingLLM Workshop at SIGDIAL 2023

2305.15025 2023-10-25 cs.CL 79%

Dior-CVAE: Pre-trained Language Models and Diffusion Priors for Variational Dialog Generation

Tianyu Yang, Thy Thy Tran, Iryna Gurevych

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

2310.15004 2023-10-24 cs.CL 79%

When Language Models Fall in Love: Animacy Processing in Transformer Language Models

Michael Hanna, Yonatan Belinkov, Sandro Pezzelle

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments To appear at EMNLP 2023

2310.06408 2023-10-24 cs.CL 79%

Humans and language models diverge when predicting repeating text

Aditya R. Vaidya, Javier Turek, Alexander G. Huth

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments To appear in the 26th Conference on Computational Natural Language Learning (CoNLL 2023). Code and data are available at https://github.com/HuthLab/lm-repeating-text

2310.07345 2023-10-12 cs.CL cs.SD eess.AS 79%

Investigating the Effect of Language Models in Sequence Discriminative Training for Neural Transducers

Zijian Yang, Wei Zhou, Ralf Schlüter, Hermann Ney

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments accepted at ASRU 2023

2211.07828 2023-06-13 cs.CL 79%

Adaptation Approaches for Nearest Neighbor Language Models

Rishabh Bhardwaj, George Polovets, Monica Sunkara

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments 10 pages, 4 figures

2305.11462 2023-05-22 cs.CL 79%

Extending Memory for Language Modelling

Anupiya Nugaliyadde

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

2305.09764 2023-05-18 cs.CL cs.SD eess.AS 79%

Application-Agnostic Language Modeling for On-Device ASR

Markus Nußbaum-Thom, Lyan Verwimp, Youssef Oualil

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments accepted for ACL 2023 industry track

2305.08286 2023-05-16 cs.SE cs.AI 79%

A Language Model of Java Methods with Train/Test Deduplication

Chia-Yi Su, Aakash Bansal, Vijayanta Jain, Sepideh Ghanavati, Collin McMillan

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.AI

Comments 4 pages + 2 references + 2 appendix. No captioned tables. Tool demonstration paper under review at ESEC/FSE 2023 Demonstration track

2305.03819 2023-05-09 cs.CL 79%

Adapting Transformer Language Models for Predictive Typing in Brain-Computer Interfaces

Shijia Liu, David A. Smith

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

2210.13569 2023-05-03 cs.CL 79%

Characterizing Verbatim Short-Term Memory in Neural Language Models

Kristijan Armeni, Christopher Honey, Tal Linzen

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments V2 corrects an issue with tokenization for one of the models (Wikitext-103 transformer). The relevant figures and the accompanying text were updated. This update does not affect conclusions which remain the same as in previous version

2212.10898 2023-03-02 cs.CL q-bio.NC 79%

Training language models to summarize narratives improves brain alignment

Khai Loong Aw, Mariya Toneva

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments ICLR 2023 (notable top 25%)

2302.06942 2023-02-15 cs.LG 79%

Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks

Joseph Renner, Pascal Denis, Rémi Gilleron, Angèle Brunellière

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.LG

Journal ref Conference of the European Chapter of the Association for Computational Linguistics, May 2023, Dubrovnik, Croatia

2301.00066 2023-01-03 cs.CL eess.AS 79%

Memory Augmented Lookup Dictionary based Language Modeling for Automatic Speech Recognition

Yukun Feng, Ming Tu, Rui Xia, Chuanzeng Huang, Yuxuan Wang

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments Submitted to ICASSP 2023

2203.00633 2022-12-07 cs.CL 79%

Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale

Laurent Sartran, Samuel Barrett, Adhiguna Kuncoro, Miloš Stanojević, Phil Blunsom, Chris Dyer

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments 17 pages, 5 figures, 2 tables and 1 algorithm. To appear in TACL, to be presented at EMNLP 2022

2212.02475 2022-12-06 cs.CL 79%

Meta-Learning Fast Weight Language Models

Kevin Clark, Kelvin Guu, Ming-Wei Chang, Panupong Pasupat, Geoffrey Hinton, Mohammad Norouzi

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2022 short paper

2208.09982 2022-08-23 cs.CL 79%

GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization

Qianqian Xie, Jimin Huang, Tulika Saha, Sophia Ananiadou

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments Accepted by COLING2022

2204.07341 2022-04-27 cs.CL 79%

LaMemo: Language Modeling with Look-Ahead Memory

Haozhe Ji, Rongsheng Zhang, Zhenyu Yang, Zhipeng Hu, Minlie Huang

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

Comments Accepted by NAACL 2022

2110.13032 2021-11-08 cs.CL 79%

Paradigm Shift in Language Modeling: Revisiting CNN for Modeling Sanskrit Originated Bengali and Hindi Language

Chowdhury Rafeed Rahman, MD. Hasibur Rahman, Mohammad Rafsan, Samiha Zakir, Mohammed Eunus Ali, Rafsanjani Muhammod

专题命中长上下文与记忆：language model(title,abstract)；分类 cs.CL

1. 长上下文与记忆 4771 篇

Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval

AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases

GenDistiller: Distilling Pre-trained Language Models based on an Autoregressive Generative Model

NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents

Emergent Word Order Universals from Cognitively-Motivated Language Models

TRAMS: Training-free Memory Selection for Long-range Language Modeling

On a Foundation Model for Operating Systems

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models

Lost in the Middle: How Language Models Use Long Contexts

Memory Augmented Language Models through Mixture of Word Experts

Adapting Language Models to Compress Contexts

Style Locality for Controllable Generation with kNN Language Models

Dior-CVAE: Pre-trained Language Models and Diffusion Priors for Variational Dialog Generation

When Language Models Fall in Love: Animacy Processing in Transformer Language Models

Humans and language models diverge when predicting repeating text

Investigating the Effect of Language Models in Sequence Discriminative Training for Neural Transducers

Adaptation Approaches for Nearest Neighbor Language Models

Extending Memory for Language Modelling

Application-Agnostic Language Modeling for On-Device ASR

A Language Model of Java Methods with Train/Test Deduplication

Adapting Transformer Language Models for Predictive Typing in Brain-Computer Interfaces

Characterizing Verbatim Short-Term Memory in Neural Language Models

Training language models to summarize narratives improves brain alignment

Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks

Memory Augmented Lookup Dictionary based Language Modeling for Automatic Speech Recognition

Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale

Meta-Learning Fast Weight Language Models

GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization

LaMemo: Language Modeling with Look-Ahead Memory

Paradigm Shift in Language Modeling: Revisiting CNN for Modeling Sanskrit Originated Bengali and Hindi Language