语言大模型 / LLM - arXivDaily 专题

2507.12930 2025-09-30 cs.CL cs.AI 84%

Making Language Model a Hierarchical Classifier

Yihong Wang, Zhonglin Jiang, Ningyuan Xi, Yue Zhao, Qingqing Gu, Xiyuan Chen, Hao Wu, Sheng Xu, Hange Zhou, Yong Chen, Luo Ji

机构 * Geely AI Lab（地平线AI实验室）； Tsinghua University（清华大学）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2509.18152 2025-09-24 cs.LG cs.AI 84%

WLFM: A Well-Logs Foundation Model for Multi-Task and Cross-Well Geological Interpretation

Zhenyu Qi, Qing Yu, Jichen Wang, Yun-Bo Zhao, Zerui Li, Wenjun Lv

机构 * Institute of Advanced Technology, University of Science and Technology of China（科学技术大学先进技术研究所）； Department of Automation, University of Science and Technology of China（科学技术大学自动化系）； Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合国家科学中心人工智能研究所）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2509.17317 2025-09-23 cs.CL cs.AI 84%

Scaling, Simplification, and Adaptation: Lessons from Pretraining on Machine-Translated Text

Dan John Velasco, Matthew Theodore Roque

机构 * Samsung R&D Institute Philippines（三星菲律宾研发院）

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Under review

2509.10369 2025-09-15 cs.LG cs.AI eess.SP q-bio.TO 84%

Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms

Gul Rukh Khattak, Konstantinos Patlatzoglou, Joseph Barker, Libor Pastika, Boroumand Zeidaabadi, Ahmed El-Medany, Hesham Aggour, Yixiu Liang, Antonio H. Ribeiro, Jeffrey Annis, Antonio Luiz Pinho Ribeiro, Junbo Ge, Daniel B. Kramer, Jonathan W. Waks, Evan Brittain, Nicholas Peters, Fu Siong Ng, Arunashis Sau

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Currently under review at npj Digital Medicine

2508.03682 2025-09-11 cs.LG cs.AI 84%

Self-Questioning Language Models

Lili Chen, Mihir Prabhudesai, Katerina Fragkiadaki, Hao Liu, Deepak Pathak

机构 * Carnegie Mellon University（卡内基梅隆大学）

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

2507.21509 2025-09-08 cs.CL cs.LG 84%

Persona Vectors: Monitoring and Controlling Character Traits in Language Models

Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, Jack Lindsey

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

2509.02046 2025-09-08 cs.LG cs.AI stat.ML 84%

Fantastic Pretraining Optimizers and Where to Find Them

Kaiyue Wen, David Hall, Tengyu Ma, Percy Liang

机构 * Stanford University（斯坦福大学）

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 108 pages, 8 figures, reproducible runs available at https://wandb.ai/marin-community/optimizer-scaling

2508.18598 2025-08-27 cs.CL cs.AI 84%

What do language models model? Transformers, automata, and the format of thought

Colin Klein

机构 * School of Philosophy（哲学学院）； The Australian National University（澳大利亚国立大学）

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2406.13069 2025-08-26 cs.CL cs.AI 84%

Evaluating $n$-Gram Novelty of Language Models Using Rusty-DAWG

William Merrill, Noah A. Smith, Yanai Elazar

机构 * New York University（纽约大学）； Allen Institute for AI（人工智能研究院）； University of Washington（华盛顿大学）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments To appear at EMNLP 2024

2508.15791 2025-08-25 cs.CL cs.AI 84%

InteChar: A Unified Oracle Bone Character List for Ancient Chinese Language Modeling

Xiaolei Diao, Zhihan Zhou, Lida Shi, Ting Wang, Ruihua Qi, Hao Xu, Daqian Shi

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

2508.13603 2025-08-20 cs.CL cs.AI 84%

Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLM

Dariia Puhach, Amir H. Payberah, Éva Székely

机构 * Department of Linguistics（语言学系）； Department of Speech, Music, and Hearing（语音、音乐与听觉系）

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Journal ref Interspeech 2025 (2025), 2058-2062

2508.11551 2025-08-19 stat.ML cs.AI cs.LG 84%

ADMIRE-BayesOpt: Accelerated Data MIxture RE-weighting for Language Models with Bayesian Optimization

Shengzhuang Chen, Xu Ouyang, Michael Arthur Leopold Pearce, Thomas Hartvigsen, Jonathan Richard Schwarz

机构 * Thomson Reuters Foundational Research & Imperial College London（汤姆逊·路透基础研究与帝国理工学院伦敦）； University of Virginia（弗吉尼亚大学）； Graphcore ； Thomson Reuters Foundational Research（汤姆逊·路透基础研究）

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

2507.07695 2025-08-01 cs.CL cs.AI 84%

KeyKnowledgeRAG (K^2RAG): An Enhanced RAG method for improved LLM question-answering capabilities

Hruday Markondapatnaikuni, Basem Suleiman, Abdelkarim Erradi, Shijing Chen

机构 * University of Sydney（悉尼大学）； University of New South Wales（新南威尔士大学）； Qatar University（卡塔尔大学）

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 21 pages, 14 figures

2507.11588 2025-07-24 q-bio.GN cs.AI cs.LG 84%

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics

Suyuan Zhao, Yizhen Luo, Ganbo Yang, Yan Zhong, Hao Zhou, Zaiqing Nie

机构 * Department of Computer Science and Tecnology, Tsinghua University（清华大学计算机科学与技术系）； Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）； School of Mathematical Sciences, Peking University（北京大学数学科学学院）； PharMolix Inc.（PharMolix公司）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Accpeted by ICML 2025

2507.14141 2025-07-22 eess.SP cs.AI cs.LG 84%

DIVER-0 : A Fully Channel Equivariant EEG Foundation Model

Danny Dongyeop Han, Ahhyun Lucy Lee, Taeyang Lee, Yonghyeon Gwon, Sebin Lee, Seongjin Lee, David Keetae Park, Shinjae Yoo, Jiook Cha, Chun Kee Chung

机构 * snu（首尔国立大学）； bnl（布鲁克海文国家实验室）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 11 pages, 1 figures, ICML 2025 Workshop on GenBio

2505.14523 2025-07-21 cs.CL cs.AI 84%

Exploring Graph Representations of Logical Forms for Language Modeling

Michael Sullivan

机构 * University at Buffalo（布法罗大学）； Saarland University（萨尔兰州大学）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments To be published in ACL 2025 Findings

2507.11966 2025-07-17 cs.CL cs.AI cs.CY 84%

Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation

Ziyu Ge, Gabriel Chua, Leanne Tan, Roy Ka-Wei Lee

专题命中预训练与数据：prompting(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2504.21018 2025-07-15 cs.CL cs.LG 84%

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization

Enes Özeren, Yihong Liu, Hinrich Schütze

机构 * LMU Munich（慕尼黑莱布尼茨大学）； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）

专题命中预训练与数据：LLM(title)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 18 pages, 3 figures, 15 tables. After ACL reviews: Corrected typos, Table 4 caption updated and the order of the results changed, numbers are unchanged. This paper will appear in ACL SRW 2025

2504.01450 2025-07-15 cs.LG cs.CL 84%

CASCADE Your Datasets for Cross-Mode Knowledge Retrieval of Language Models

Runlong Zhou, Yi Zhang

机构 * University of Washington（华盛顿大学）； Microsoft Research（微软研究院）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments COLM 2025

2501.12370 2025-07-04 cs.LG cs.AI 84%

Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models

Samira Abnar, Harshay Shah, Dan Busbridge, Alaaeldin Mohamed Elnouby Ali, Josh Susskind, Vimal Thilak

机构 * Apple（苹果公司）； MIT（麻省理工学院）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2507.01844 2025-07-03 cs.CL cs.LG 84%

Low-Perplexity LLM-Generated Sequences and Where To Find Them

Arthur Wuhrmann, Anastasiia Kucherenko, Andrei Kucharavy

机构 * École Polytechnique Fédérale de Lausanne（瑞士联邦理工学院）； Institute of Entrepreneurship and Management, HES-SO Valais-Wallis（企业家与管理学院）； Institute of Informatics, HES-SO Valais-Wallis（信息学院）

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Camera-ready version. Accepted to ACL 2025. 10 pages, 4 figures, 6 tables

2507.01045 2025-07-03 cs.LG cs.AI eess.SP 84%

Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals

Xiao Gu, Wei Tang, Jinpei Han, Veer Sangha, Fenglin Liu, Shreyank N Gowda, Antonio H. Ribeiro, Patrick Schwab, Kim Branson, Lei Clifton, Antonio Luiz P. Ribeiro, Zhangdaihong Liu, David A. Clifton

机构 * University of Oxford（牛津大学）； University of Nottingham（诺丁汉大学）； Uppsala University（乌普萨拉大学）； GlaxoSmithKline（葛兰素史克）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

2506.21803 2025-06-30 eess.SP cs.AI cs.LG 84%

From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining

Fuying Wang, Jiacheng Xu, Lequan Yu

机构 * School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China（计算与数据科学学院，香港大学，香港特别行政区，中国）

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments ICML 2025

2506.21071 2025-06-27 cs.LG cs.CL 84%

Enhancing LLM Tool Use with High-quality Instruction Data from Knowledge Graph

Jingwei Wang, Zai Zhang, Hao Qian, Chunjing Gan, Binbin Hu, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Bin Shi, Bo Dong

机构 * Ant Group, China（蚂蚁集团）； School of Computer Science and Technology, Xi’an Jiaotong University, China（西安交通大学计算机科学与技术学院）； School of Distance Education, Xi’an Jiaotong University, China（西安交通大学继续教育学院）

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 20 pages, 12 figures

2506.20249 2025-06-26 cs.AI cs.CL cs.MA 84%

Language Modeling by Language Models

Junyan Cheng, Peter Clark, Kyle Richardson

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

2502.14709 2025-06-23 cs.CL cs.LG 84%

Group-Level Data Selection for Efficient Pretraining

Zichun Yu, Fei Peng, Jie Lei, Arnold Overwijk, Wen-tau Yih, Chenyan Xiong

机构 * Language Technologies Institute, Carnegie Mellon University（语言技术研究所，卡内基梅隆大学）； Meta

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

2506.14761 2025-06-18 cs.CL cs.AI 84%

From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

Mathurin Videau, Badr Youbi Idrissi, Alessandro Leite, Marc Schoenauer, Olivier Teytaud, David Lopez-Paz

机构 * FAIR at Meta ； TAU, INRIA ； LISN, CNRS \& Université Paris-Saclay

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

2505.12442 2025-06-18 cs.CR cs.AI cs.CL 84%

IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems

Liwen Wang, Wenxuan Wang, Shuai Wang, Zongjie Li, Zhenlan Ji, Zongyi Lyu, Daoyuan Wu, Shing-Chi Cheung

机构 * The Hong Kong University of Science and Technology（香港科技大学）

专题命中预训练与数据：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2502.07827 2025-06-13 cs.LG cs.AI 84%

Implicit Language Models are RNNs: Balancing Parallelization and Expressivity

Mark Schöne, Babak Rahmani, Heiner Kremer, Fabian Falck, Hitesh Ballani, Jannes Gladrow

机构 * Chair of Highly-Parallel VLSI Systems and Neuro-Microelectronics , TUD Dresden University of Technology, Dresden, Germany（德累斯顿技术大学神经微电子学系）； Microsoft Research, Cambridge, United Kingdom（微软研究院）

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 25 pages, 12 figures, 7 tables

2505.05568 2025-06-12 cs.LG cs.AI cs.DB 84%

Griffin: Towards a Graph-Centric Relational Database Foundation Model

Yanbo Wang, Xiyuan Wang, Quan Gan, Minjie Wang, Qibin Yang, David Wipf, Muhan Zhang

机构 * Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）； Amazon Web Services（亚马逊网络服务）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Published at ICML 2025

1. 预训练与数据 12379 篇

Making Language Model a Hierarchical Classifier

WLFM: A Well-Logs Foundation Model for Multi-Task and Cross-Well Geological Interpretation

Scaling, Simplification, and Adaptation: Lessons from Pretraining on Machine-Translated Text

Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms

Self-Questioning Language Models

Persona Vectors: Monitoring and Controlling Character Traits in Language Models

Fantastic Pretraining Optimizers and Where to Find Them

What do language models model? Transformers, automata, and the format of thought

Evaluating $n$-Gram Novelty of Language Models Using Rusty-DAWG

InteChar: A Unified Oracle Bone Character List for Ancient Chinese Language Modeling

Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLM

ADMIRE-BayesOpt: Accelerated Data MIxture RE-weighting for Language Models with Bayesian Optimization

KeyKnowledgeRAG (K^2RAG): An Enhanced RAG method for improved LLM question-answering capabilities

SToFM: a Multi-scale Foundation Model for Spatial Transcriptomics

DIVER-0 : A Fully Channel Equivariant EEG Foundation Model

Exploring Graph Representations of Logical Forms for Language Modeling

Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization

CASCADE Your Datasets for Cross-Mode Knowledge Retrieval of Language Models

Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models

Low-Perplexity LLM-Generated Sequences and Where To Find Them

Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals

From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining

Enhancing LLM Tool Use with High-quality Instruction Data from Knowledge Graph

Language Modeling by Language Models

Group-Level Data Selection for Efficient Pretraining

From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems

Implicit Language Models are RNNs: Balancing Parallelization and Expressivity

Griffin: Towards a Graph-Centric Relational Database Foundation Model