语言大模型 / LLM - arXivDaily 专题

2504.12563 2025-11-06 cs.CL cs.AI cs.LG 87%

MetaSynth: Meta-Prompting-Driven Agentic Scaffolds for Diverse Synthetic Data Generation

Haris Riaz, Sourav Bhabesh, Vinayak Arannil, Miguel Ballesteros, Graham Horwood

专题命中预训练与数据：prompting(title,abstract)；LLM(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 33 pages, 17 figures. Findings of ACL 2025

Journal ref Findings of the Association for Computational Linguistics: ACL 2025, pp. 18770-18803, Vienna, Austria, 2025. Association for Computational Linguistics

2510.15990 2025-10-21 cs.LG cs.AI cs.CL 87%

Can GRPO Help LLMs Transcend Their Pretraining Origin?

Kangqi Ni, Zhen Tan, Zijie Liu, Pingzhi Li, Tianlong Chen

机构 * cs.unc.edu（北卡罗来纳大学教堂山分校计算机科学系）

专题命中预训练与数据：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2510.01631 2025-10-03 cs.LG cs.AI cs.CL 87%

Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls

Feiyang Kang, Newsha Ardalani, Michael Kuchnik, Youssef Emad, Mostafa Elhoushi, Shubhabrata Sengupta, Shang-Wen Li, Ramya Raghavendra, Ruoxi Jia, Carole-Jean Wu

机构 * FAIR at Meta（Meta 的 FAIR）

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a Main Conference paper at EMNLP 2025

2509.23543 2025-09-30 q-bio.GN cs.NE q-bio.MN 87%

Contrastive Learning Enhances Language Model Based Cell Embeddings for Low-Sample Single Cell Transcriptomics

Luxuan Zhang, Douglas Jiang, Qinglong Wang, Haoqi Sun, Feng Tian

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；foundation model(abstract)

Comments 14 pages, 4 figures, 2 tables

2410.14332 2025-08-14 cs.CV 87%

ViCToR: Improving Visual Comprehension via Token Reconstruction for Pretraining LMMs

Yin Xie, Kaicheng Yang, Peirou Liang, Xiang An, Yongle Zhao, Yumeng Wang, Ziyong Feng, Roy Miles, Ismail Elezi, Jiankang Deng

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 10 pages, 6 figures, 5 tables

2505.22964 2025-08-14 cs.CL cs.AI cs.LG 87%

Exploring Scaling Laws for EHR Foundation Models

Sheng Zhang, Qin Liu, Naoto Usuyama, Cliff Wong, Tristan Naumann, Hoifung Poon

机构 * Microsoft Research（微软研究院）； University of Southern California（南加州大学）

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2503.06312 2025-07-23 cs.CV 87%

DOFA-CLIP: Multimodal Vision-Language Foundation Models for Earth Observation

Zhitong Xiong, Yi Wang, Weikang Yu, Adam J Stewart, Jie Zhao, Nils Lehmann, Thomas Dujardin, Zhenghang Yuan, Pedram Ghamisi, Xiao Xiang Zhu

机构 * Technical University of Munich（慕尼黑技术大学）； Helmholtz-Zentrum Dresden-Rossendorf（德累斯顿-罗斯托克亥姆霍尔茨中心）

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments code & weights: https://github.com/xiong-zhitong/DOFA-CLIP

2411.10272 2025-05-27 cs.AI cs.CL cs.LG 87%

P$^2$ Law: Scaling Law for Post-Training After Model Pruning

Xiaodong Chen, Yuxuan Hu, Xiaokang Zhang, Yanling Wang, Cuiping Li, Hong Chen, Jing Zhang

机构 * Engineering Research Center of Database and Business Intelligence, MOE, China（数据库与商业智能工程研究中心，教育部，中国）； School of Information, Renmin University of China,Beijing, China（信息学院，中国人民大学，北京，中国）； Key Laboratory of Data Engineering and Knowledge Engineering, MOE, China（数据工程与知识工程重点实验室，教育部，中国）； Zhipu AI, China（智谱AI，中国）

专题命中预训练与数据：post-training(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted as Main of ACL2025

2411.01533 2025-05-20 cs.CL cs.AI cs.LG 87%

Enhancing LLM Evaluations: The Garbling Trick

William F. Bradley

机构 * Mirabolic Consulting（Mirabolic咨询公司）

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 15 pages, 4 figures

2504.12427 2025-04-18 cs.CL cs.AI cs.CY cs.LG 87%

Position: The Most Expensive Part of an LLM should be its Training Data

Nikhil Kandpal, Colin Raffel

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 8 pages, 3 figures

2406.01638 2025-04-01 cs.LG cs.AI cs.CL 87%

TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment

Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, Rui Zhao

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted as an Oral Presentation at AAAI 2025 (Main Technical Track)

2403.16952 2025-03-21 cs.CL cs.AI cs.LG 87%

Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance

Jiasheng Ye, Peiju Liu, Tianxiang Sun, Jun Zhan, Yunhua Zhou, Xipeng Qiu

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted by ICLR2025, camera ready version

2503.02304 2025-03-18 cs.CV 87%

A Token-level Text Image Foundation Model for Document Understanding

Tongkun Guan, Zining Wang, Pei Fu, Zhengtao Guo, Wei Shen, Kai Zhou, Tiezhu Yue, Chen Duan, Hao Sun, Qianyi Jiang, Junfeng Luo, Xiaokang Yang

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 23 pages

2502.12372 2025-02-19 cs.CL cs.AI cs.LG 87%

Factual Inconsistency in Data-to-Text Generation Scales Exponentially with LLM Size: A Statistical Validation

Joy Mahapatra, Soumyajit Roy, Utpal Garain

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 21 pages

2502.04419 2025-02-11 cs.LG cs.AI cs.CL 87%

Understanding and Mitigating the Bias Inheritance in LLM-based Data Augmentation on Downstream Tasks

Miaomiao Li, Hao Chen, Yang Wang, Tingyuan Zhu, Weijia Zhang, Kaijie Zhu, Kam-Fai Wong, Jindong Wang

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Technical report; 31 pages

2412.02906 2024-12-05 cs.SE cs.AI cs.CL cs.LG 87%

Does Few-Shot Learning Help LLM Performance in Code Synthesis?

Derek Xu, Tong Xie, Botao Xia, Haoyu Li, Yunsheng Bai, Yizhou Sun, Wei Wang

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2411.02791 2024-11-06 cs.CL cs.AI cs.IR cs.LG cs.NE stat.ML 87%

Language Models and Cycle Consistency for Self-Reflective Machine Translation

Jianqiao Wangni

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.14904 2024-10-29 cs.CR cs.AI cs.CL cs.LG 87%

Watermarking Makes Language Models Radioactive

Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, Teddy Furon

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at NeurIPS 2024 (Spotlight). Code at https://github.com/facebookresearch/radioactive-watermark - webpage at https://pierrefdz.github.io/publications/radioactive/

2406.10918 2024-10-21 cs.LG cs.AI cs.CL 87%

Multi-LLM QA with Embodied Exploration

Bhrij Patel, Vishnu Sashank Dorbala, Amrit Singh Bedi, Dinesh Manocha

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages, 9 Figures, 5 Tables

2406.02958 2024-10-21 cs.LG cs.AI cs.CL cs.CR cs.DC 87%

PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs

Charlie Hou, Akshat Shrivastava, Hongyuan Zhan, Rylan Conway, Trang Le, Adithya Sagar, Giulia Fanti, Daniel Lazar

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2024 (Oral). Latest revision corrects a discussion on concurrent work arXiv:2403.01749. We described their work as reliant on using closed-sourced models when in reality they also evaluate and use open source models. This has been corrected in this version

2408.04381 2024-08-09 cs.IR 87%

Understanding and Modeling Job Marketplace with Pretrained Language Models

Yaochen Zhu, Liang Wu, Binchi Zhang, Song Wang, Qi Guo, Liangjie Hong, Luke Simon, Jundong Li

专题命中预训练与数据：language model(title,abstract)；foundation model(abstract)；pretraining(abstract)；prompting(abstract)

Comments accepted by CIKM'24 applied research track

2408.00161 2024-08-09 cs.CL cs.AI cs.ET cs.LG 87%

Automatic Generation of Behavioral Test Cases For Natural Language Processing Using Clustering and Prompting

Ying Li, Rahul Singh, Tarun Joshi, Agus Sudjianto

专题命中预训练与数据：prompting(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2404.09624 2024-07-25 cs.CV 87%

AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception

Yipo Huang, Xiangfei Sheng, Zhichao Yang, Quan Yuan, Zhichao Duan, Pengfei Chen, Leida Li, Weisi Lin, Guangming Shi

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments Accepted by ACMMM24

2309.10952 2024-06-25 cs.CL cs.AI cs.LG 87%

LMDX: Language Model-based Document Information Extraction and Localization

Vincent Perot, Kai Kang, Florian Luisier, Guolong Su, Xiaoyu Sun, Ramya Sree Boppana, Zilong Wang, Zifeng Wang, Jiaqi Mu, Hao Zhang, Chen-Yu Lee, Nan Hua

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2406.13193 2024-06-21 cs.LG cs.AI cs.CL physics.chem-ph 87%

PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes

He Cao, Yanjun Shao, Zhiyuan Liu, Zijing Liu, Xiangru Tang, Yuan Yao, Yu Li

专题命中预训练与数据：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2404.01361 2024-04-03 cs.CL cs.AI cs.HC cs.LG 87%

LLM Attributor: Interactive Visual Attribution for LLM Generation

Seongmin Lee, Zijie J. Wang, Aishwarya Chakravarthy, Alec Helbling, ShengYun Peng, Mansi Phute, Duen Horng Chau, Minsuk Kahng

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 8 pages, 3 figures, For a video demo, see https://youtu.be/mIG2MDQKQxM

2310.05861 2024-04-03 cs.CL cs.AI cs.CV cs.LG 87%

Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models

Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

专题命中预训练与数据：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2024 camera-ready (23 pages), Code: https://github.com/archiki/RepARe

2403.14589 2024-04-02 cs.AI cs.CL cs.LG 87%

ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy

Zonghan Yang, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Yang Liu

专题命中预训练与数据：language agent(title,abstract)；foundation model(abstract)；prompting(abstract)；分类 cs.CL、cs.AI、cs.LG

2403.01308 2024-03-15 cs.CL cs.AI cs.LG 87%

VBART: The Turkish LLM

Meliksah Turker, Mehmet Erdi Ari, Aydin Han

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2403.06070 2024-03-12 cs.CV cs.HC 87%

Reframe Anything: LLM Agent for Open World Video Reframing

Jiawang Cao, Yongliang Wu, Weiheng Chi, Wenbo Zhu, Ziyue Su, Jay Wu

专题命中预训练与数据：LLM(title,abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments 14 pages, 6 figures

1. 预训练与数据 12365 篇

MetaSynth: Meta-Prompting-Driven Agentic Scaffolds for Diverse Synthetic Data Generation

Can GRPO Help LLMs Transcend Their Pretraining Origin?

Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls

Contrastive Learning Enhances Language Model Based Cell Embeddings for Low-Sample Single Cell Transcriptomics

ViCToR: Improving Visual Comprehension via Token Reconstruction for Pretraining LMMs

Exploring Scaling Laws for EHR Foundation Models

DOFA-CLIP: Multimodal Vision-Language Foundation Models for Earth Observation

P$^2$ Law: Scaling Law for Post-Training After Model Pruning

Enhancing LLM Evaluations: The Garbling Trick

Position: The Most Expensive Part of an LLM should be its Training Data

TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment

Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance

A Token-level Text Image Foundation Model for Document Understanding

Factual Inconsistency in Data-to-Text Generation Scales Exponentially with LLM Size: A Statistical Validation

Understanding and Mitigating the Bias Inheritance in LLM-based Data Augmentation on Downstream Tasks

Does Few-Shot Learning Help LLM Performance in Code Synthesis?

Language Models and Cycle Consistency for Self-Reflective Machine Translation

Watermarking Makes Language Models Radioactive

Multi-LLM QA with Embodied Exploration

PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs

Understanding and Modeling Job Marketplace with Pretrained Language Models

Automatic Generation of Behavioral Test Cases For Natural Language Processing Using Clustering and Prompting

AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception

LMDX: Language Model-based Document Information Extraction and Localization

PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes

LLM Attributor: Interactive Visual Attribution for LLM Generation

Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models

ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy

VBART: The Turkish LLM

Reframe Anything: LLM Agent for Open World Video Reframing