语言大模型 / LLM - arXivDaily 专题

2312.11658 2024-01-17 cs.CR cs.AI cs.SE 88%

Traces of Memorisation in Large Language Models for Code

Ali Al-Kaswan, Maliheh Izadi, Arie van Deursen

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Comments ICSE 2024 Research Track

2301.13779 2023-12-21 cs.PL cs.AI cs.SE 88%

FLAME: A small language model for spreadsheet formulas

Harshit Joshi, Abishai Ebenezer, José Cambronero, Sumit Gulwani, Aditya Kanade, Vu Le, Ivan Radiček, Gust Verbruggen

专题命中预训练与数据：language model(title,abstract)；small language model(title)；large language model(abstract)；分类 cs.AI

Comments Accepted to AAAI 2024

2311.06377 2023-11-14 cs.CL 88%

Heaps' Law in GPT-Neo Large Language Model Emulated Corpora

Uyen Lai, Gurjit S. Randhawa, Paul Sheridan

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 4 pages, 1 figure, 1 table, EVIA 2023

2308.15727 2023-11-07 cs.CL 88%

Quantifying and Analyzing Entity-level Memorization in Large Language Models

Zhenhong Zhou, Jiuyang Xiang, Chaomeng Chen, Sen Su

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 9 pages, 7 figures

2310.04027 2023-11-07 cs.CL q-fin.ST q-fin.TR 88%

Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models

Boyu Zhang, Hongyang Yang, Tianyu Zhou, Ali Babar, Xiao-Yang Liu

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ACM International Conference on AI in Finance (ICAIF) 2023

2310.13673 2023-11-01 cs.CL 88%

StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models

Sullam Jeoung, Yubin Ge, Jana Diesner

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023

2310.19998 2023-11-01 cs.CL cond-mat.dis-nn cond-mat.mes-hall cond-mat.mtrl-sci physics.app-ph 88%

Generative retrieval-augmented ontologic graph and multi-agent strategies for interpretive large language model-based materials design

Markus J. Buehler

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2310.16261 2023-10-26 cs.CL 88%

The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining

Ting-Rui Chiang, Dani Yogatama

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL

Comments EMNLP 2023

2310.12874 2023-10-24 cs.CL 88%

StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding

Cheng Jiayang, Lin Qiu, Tsz Ho Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, Zheng Zhang

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023 main conference

2310.13705 2023-10-24 cs.HC cs.AI 88%

Large language models in textual analysis for gesture selection

Laura B. Hensel, Nutchanon Yongsatianchot, Parisa Torshizi, Elena Minucci, Stacy Marsella

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

Journal ref 25th ACM International Conference on Multimodal Interaction. ICMI. 2023. 1-10

2310.10808 2023-10-18 cs.IR cs.AI 88%

If the Sources Could Talk: Evaluating Large Language Models for Research Assistance in History

Giselle Gonzalez Garcia, Christian Weilbach

专题命中预训练与数据：language model(title,abstract)；large language model(title)；LLM(abstract)；分类 cs.AI

Comments Will be published at CHR2023

2308.04948 2023-10-10 cs.CL 88%

Extrapolating Large Language Models to Non-English by Aligning Languages

Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2305.10321 2023-09-20 cs.CL cs.SD eess.AS 88%

Controllable Speaking Styles Using a Large Language Model

Atli Thor Sigurgeirsson, Simon King

专题命中预训练与数据：language model(title,abstract)；large language model(title)；LLM(abstract)；分类 cs.CL

Comments Submitted to ICASSP 2024

2309.09390 2023-09-19 cs.CL cs.SD eess.AS 88%

Augmenting text for spoken language understanding with Large Language Models

Roshan Sharma, Suyoun Kim, Daniel Lazar, Trang Le, Akshat Shrivastava, Kwanghoon Ahn, Piyush Kansal, Leda Sari, Ozlem Kalinli, Michael Seltzer

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Submitted to ICASSP 2024

2309.04516 2023-09-12 eess.AS cs.LG cs.SD 88%

End-to-End Speech Recognition and Disfluency Removal with Acoustic Language Model Pretraining

Saksham Bassi, Giulio Duregon, Siddhartha Jalagam, David Roth

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.LG

2309.00642 2023-09-06 cs.CL 88%

Extracting Mathematical Concepts with Large Language Models

Valeria de Paiva, Qiyue Gao, Pavel Kovalev, Lawrence S. Moss

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 13 pages, 4 figures, presented to the 14th MathUI Workshop 2023

2306.11702 2023-09-04 cs.DB cs.CL 88%

Lingua Manga: A Generic Large Language Model Centric System for Data Curation

Zui Chen, Lei Cao, Sam Madden

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 4 pages, 6 figures, VLDB 2023 Demo paper

2306.03950 2023-07-10 cs.CL 88%

MISGENDERED: Limits of Large Language Models in Understanding Pronouns

Tamanna Hossain, Sunipa Dev, Sameer Singh

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted at ACL 2023 as a long paper

2305.08283 2023-07-07 cs.CL 88%

From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models

Shangbin Feng, Chan Young Park, Yuhan Liu, Yulia Tsvetkov

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL

Comments ACL 2023

2306.11892 2023-06-22 cs.CL 88%

Exploring New Frontiers in Agricultural NLP: Investigating the Potential of Large Language Models for Food Applications

Saed Rezayi, Zhengliang Liu, Zihao Wu, Chandra Dhakal, Bao Ge, Haixing Dai, Gengchen Mai, Ninghao Liu, Chen Zhen, Tianming Liu, Sheng Li

专题命中预训练与数据：language model(title,abstract)；large language model(title)；pretraining(abstract)；分类 cs.CL

2205.06733 2023-06-12 cs.CL 88%

Arithmetic-Based Pretraining -- Improving Numeracy of Pretrained Language Models

Dominic Petrak, Nafise Sadat Moosavi, Iryna Gurevych

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL

Comments Published at StarSEM2023

2305.10036 2023-06-05 cs.CL cs.CY 88%

Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark

Wenjun Peng, Jingwei Yi, Fangzhao Wu, Shangxi Wu, Bin Zhu, Lingjuan Lyu, Binxing Jiao, Tong Xu, Guangzhong Sun, Xing Xie

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted by ACL 2023

2304.11158 2023-06-02 cs.CL 88%

Emergent and Predictable Memorization in Large Language Models

Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, Edward Raff

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2304.01373 2023-06-01 cs.CL 88%

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, Oskar van der Wal

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Code at https://github.com/EleutherAI/pythia

2305.15275 2023-05-25 cs.CL 88%

Self-Evolution Learning for Discriminative Language Model Pretraining

Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL

Comments Accepted to Findings of ACL2023

2305.15041 2023-05-25 cs.CL 88%

Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science

Veniamin Veselovsky, Manoel Horta Ribeiro, Akhil Arora, Martin Josifoski, Ashton Anderson, Robert West

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 8 pages

2305.12816 2023-05-23 cs.CL 88%

Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model

Xiao Wang, Weikang Zhou, Qi Zhang, Jie Zhou, Songyang Gao, Junzhe Wang, Menghan Zhang, Xiang Gao, Yunwen Chen, Tao Gui

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL

Comments Accepted by ACL2023

2305.02156 2023-05-04 cs.IR cs.CL 88%

Zero-Shot Listwise Document Reranking with a Large Language Model

Xueguang Ma, Xinyu Zhang, Ronak Pradeep, Jimmy Lin

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

2304.10447 2023-04-21 cs.CL 88%

Domain-specific Continued Pretraining of Language Models for Capturing Long Context in Mental Health

Shaoxiong Ji, Tianlin Zhang, Kailai Yang, Sophia Ananiadou, Erik Cambria, Jörg Tiedemann

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL

2205.10770 2022-11-04 cs.CL 88%

Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models

Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen Aghajanyan

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

1. 预训练与数据 12353 篇

Traces of Memorisation in Large Language Models for Code

FLAME: A small language model for spreadsheet formulas

Heaps' Law in GPT-Neo Large Language Model Emulated Corpora

Quantifying and Analyzing Entity-level Memorization in Large Language Models

Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models

StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models

Generative retrieval-augmented ontologic graph and multi-agent strategies for interpretive large language model-based materials design

The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining

StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding

Large language models in textual analysis for gesture selection

If the Sources Could Talk: Evaluating Large Language Models for Research Assistance in History

Extrapolating Large Language Models to Non-English by Aligning Languages

Controllable Speaking Styles Using a Large Language Model

Augmenting text for spoken language understanding with Large Language Models

End-to-End Speech Recognition and Disfluency Removal with Acoustic Language Model Pretraining

Extracting Mathematical Concepts with Large Language Models

Lingua Manga: A Generic Large Language Model Centric System for Data Curation

MISGENDERED: Limits of Large Language Models in Understanding Pronouns

From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models

Exploring New Frontiers in Agricultural NLP: Investigating the Potential of Large Language Models for Food Applications

Arithmetic-Based Pretraining -- Improving Numeracy of Pretrained Language Models

Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark

Emergent and Predictable Memorization in Large Language Models

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

Self-Evolution Learning for Discriminative Language Model Pretraining

Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science

Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model

Zero-Shot Listwise Document Reranking with a Large Language Model

Domain-specific Continued Pretraining of Language Models for Capturing Long Context in Mental Health

Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models