语言大模型 / LLM

Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation

Ruijie Xi, He Ba, Hao Yuan, Rishu Agrawal, Yuxin Tian, Ruoyan Kong, Arul Prakash

机构 * North Carolina State University（北卡罗来纳州立大学） ； Meta 纠错

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Finetuning a Weather Foundation Model with Lightweight Decoders for Unseen Physical Processes

Fanny Lehmann, Firat Ozdemir, Benedikt Soja, Torsten Hoefler, Siddhartha Mishra, Sebastian Schemm

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Approximating Language Model Training Data from Weights

John X. Morris, Junjie Oscar Yin, Woojeong Kim, Vitaly Shmatikov, Alexander M. Rush

机构 * Cornell University（康奈尔大学） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；SFT(abstract)；分类 cs.CL

Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning

Wassim Bouaziz, Mathurin Videau, Nicolas Usunier, El-Mahdi El-Mhamdi

机构 * Meta FAIR & CMAP（Meta FAIR及CMAP） ； École polytechnique Paris, France（巴黎高等理工学院） ； Meta FAIR & Université Paris Saclay Paris, France（Meta FAIR及巴黎萨克雷大学） ； Work done while at Meta FAIR（在Meta FAIR期间的工作） ； CMAP, École polytechnique Palaiseau, France（CMAP及巴黎高等理工学院Palaiseau分校） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.LG

Comments 18 pages, 12 figures

Assessing the Role of Data Quality in Training Bilingual Language Models

Skyler Seto, Maartje ter Hoeve, Maureen de Seyssel, David Grangier

机构 * Apple（苹果公司） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments 26 pages, 18 figures, 25 tables

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models

Lindia Tjuatja, Graham Neubig

专题命中 预训练与数据 ：language model(title,abstract)；post-training(abstract)；分类 cs.CL

Comments Accepted to ACL 2025 Main Conference

A Foundation Model for Spatial Proteomics

Muhammad Shaban, Yuzhou Chang, Huaying Qiu, Yao Yu Yeo, Andrew H. Song, Guillaume Jaume, Yuchen Wang, Luca L. Weishaupt, Tong Ding, Anurag Vaidya, Abdallah Lamane, Daniel Shao, Mohammed Zidane, Yunhao Bai, Paige McCallum, Shuli Luo, Wenrui Wu, Yang Wang, Precious Cramer, Chi Ngai Chan, Pierre Stephan, Johanna Schaffenrath, Jia Le Lee, Hendrik A. Michel, Caiwei Tian, Cristina Almagro-Perez, Sophia J. Wagner, Sharifa Sahai, Ming Y. Lu, Richard J. Chen, Andrew Zhang, Mark Edward M. Gonzales, Ahmad Makky, Jia-Ying Joey Lee, Hao Cheng, Nourhan El Ahmar, Sayed Matar, Maximilian Haist, Darci Phillips, Yuqi Tan, Garry P. Nolan, W. Richard Burack, Jacob D. Estes, Jonathan T. C. Liu, Toni K Choueiri, Neeraj Agarwal, Marc Barry, Scott J. Rodig, Long Phi Le, Georg Gerber, Christian M. Schürch, Fabian J. Theis, Youn H Kim, Joe Yeong, Sabina Signoretti, Brooke E. Howitt, Lit-Hsin Loo, Qin Ma, Sizun Jiang, Faisal Mahmood

专题命中 预训练与数据 ：foundation model(title,abstract)；post-training(abstract)；分类 cs.AI

Ola: Pushing the Frontiers of Omni-Modal Language Model

Zuyan Liu, Yuhao Dong, Jiahui Wang, Ziwei Liu, Winston Hu, Jiwen Lu, Yongming Rao

机构 * Tsinghua University（清华大学） ； Tencent Hunyuan Research（腾讯文睿实验室） ； S-Lab, NTU（国立科技大学S实验室） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Adaptive-VP: A Framework for LLM-Based Virtual Patients that Adapts to Trainees' Dialogue to Facilitate Nurse Communication Training

Keyeun Lee, Seolhee Lee, Esther Hehsun Kim, Yena Ko, Jinsu Eun, Dahee Kim, Hyewon Cho, Haiyi Zhu, Robert E. Kraut, Eunyoung Suh, Eun-mee Kim, Hajin Lim

机构 * hci+d Lab（hci+d实验室） ； Department of Communication（通信系） ； Department of Nursing（护理系） ； Human-Computer Interaction Institute（人机交互研究所） 纠错

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ACL 2025 Findings, 34 pages, 9 figures

Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models

Congcong Zhu, Xiaoyan Xu, Jiayue Han, Jingrun Chen

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Comments Accepted as a conference paper in ICML2025

Emergent LLM behaviors are observationally equivalent to data leakage

Christopher Barrie, Petter Törnberg

机构 * Christopher Barrie Assistant Professor of Sociology, New York University（纽约大学社会学助理教授） ； New York University（纽约大学） ； Petter Törnberg Institute for Logic, Language, and Computation, University of Amsterdam（阿姆斯特丹大学逻辑、语言和计算研究所） ； University of Amsterdam（阿姆斯特丹大学） 纠错

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation

Jingping Nie, Dung T. Tran, Karan Thakkar, Vasudha Kowtha, Jon Huang, Carlos Avendano, Erdrin Azemi, Vikramjit Mitra

机构 * Apple, USA（苹果公司） ； The University of North Carolina at Chapel Hill, USA（北卡罗来纳大学教堂山分校） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Comments 5 pages, Interspeech 2025 conference

A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints

Michael Munn, Susan Wei

专题命中 预训练与数据 ：pretraining(title,abstract)；foundation model(abstract)；分类 cs.LG

Comments Accepted as an ICML 2025 paper

Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning

Tong Li, Jiachuan Wang, Yongqi Zhang, Shuangyin Li, Lei Chen

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Department of Computer Science and Engineering（计算机科学与工程系） ； Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； Thrust of Data Science And Analytics（数据科学与分析方向） ； South China Normal University（华南师范大学） ； School of Computer Science（计算机科学学院） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted to KDD 2025. This is the author's version of the work

LangSAMP: Language-Script Aware Multilingual Pretraining

Yihong Liu, Haotian Ye, Chunlan Ma, Mingyang Wang, Hinrich Schütze

机构 * Center for Information and Language Processing, LMU Munich（信息与语言处理中心，慕尼黑莱茵河大学） ； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML）） ； Bosch Center for Artificial Intelligence（博世人工智能中心） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments ACL 2025

Breaking Language Barriers in Visual Language Models via Multilingual Textual Regularization

Iñigo Pikabea, Iñaki Lacunza, Oriol Pareras, Carlos Escolano, Aitor Gonzalez-Agirre, Javier Hernando, Marta Villegas

机构 * Barcelona Supercomputing Center（巴塞罗那超级计算中心） ； Universitat Politècnica de Catalunya（加泰罗尼亚理工大学） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；instruction tuning(abstract)；分类 cs.AI

Comments v2: Expanded model merging experiments. Fix duplicated subsection on limitations

Reformulation for Pretraining Data Augmentation

Xintong Hao, Ruijie Zhu, Ge Zhang, Ke Shen, Chenggang Li

机构 * ByteDance Seed（字节跳动种子） ； University of California, Santa Cruz（加州大学圣克ruz分校） 纠错

专题命中 预训练与数据 ：pretraining(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Dataset released https://huggingface.co/datasets/ByteDance-Seed/mga-fineweb-edu

Behind Maya: Building a Multilingual Vision Language Model

Nahid Alam, Karthik Reddy Kanjula, Surya Guthikonda, Timothy Chung, Bala Krishna S Vegesna, Abhipsha Das, Anthony Susevski, Ryan Sze-Yin Chan, S M Iftekhar Uddin, Shayekh Bin Islam, Roshan Santhosh, Snegha A, Drishti Sharma, Chen Liu, Isha Chaturvedi, Genta Indra Winata, Ashvanth. S, Snehanshu Mukherjee, Alham Fikri Aji

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted at VLMs4ALL CVPR 2025 Workshop; corrected workshop name spelling

AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection

Kai Hua, Steven Wu, Ge Zhang, Ke Shen

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 28 pages, 19 figures

Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection

Ziqing Fan, Siyuan Du, Shengchao Hu, Pingjie Wang, Li Shen, Ya Zhang, Dacheng Tao, Yanfeng Wang

机构 * Shanghai Jiao Tong University（上海交通大学） ； Shanghai AI Laboratory（上海人工智能实验室） ； Fudan University（复旦大学） ； Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.LG

Impact of Noise on LLM-Models Performance in Abstraction and Reasoning Corpus (ARC) Tasks with Model Temperature Considerations

Nikhil Khandalkar, Pavan Yadav, Krishna Shinde, Lokesh B. Ramegowda, Rajarshi Das

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 60 pages, 25 figures

Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training

Zhijun Wang, Jiahuan Li, Hao Zhou, Rongxiang Weng, Jingang Wang, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Exploring Expert Failures Improves LLM Agent Tuning

Li-Cheng Lan, Andrew Bai, Minhao Cheng, Cho-Jui Hsieh, Tianyi Zhou

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensing

Xuyang Li, Chenyu Li, Gemine Vivone, Danfeng Hong

机构 * Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院 aerospace information research institute） ； School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院） ； School of Mathematics and Statistics, Southeast University（东南大学数学与统计学院） ； Institute of Methodologies for Environmental Analysis, National Research Council, CNR-IMAA（国家研究理事会环境分析方法研究所） ； Graduate School of Frontier Sciences, the University of Tokyo（东京大学前沿科学研究生院） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

FACT: Foundation Model for Assessing Cancer Tissue Margins with Mass Spectrometry

Mohammad Farahmand, Amoon Jamzad, Fahimeh Fooladgar, Laura Connolly, Martin Kaufmann, Kevin Yi Mi Ren, John Rudan, Doug McKay, Gabor Fichtinger, Parvin Mousavi

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Journal ref International Journal of Computer Assisted Radiology and Surgery (2025)

Training state-of-the-art pathology foundation models with orders of magnitude less data

Mikhail Karasikov, Joost van Doorn, Nicolas Känzig, Melis Erdal Cesur, Hugo Mark Horlings, Robert Berke, Fei Tang, Sebastian Otálora

专题命中 预训练与数据 ：foundation model(title,abstract)；post-training(abstract)；分类 cs.LG

Comments 10 pages, 3 figures

Krutrim LLM: A Novel Tokenization Strategy for Multilingual Indic Languages with Petabyte-Scale Data Processing

Rahul Kumar, Shubham Kakde, Divyansh Rajput, Daud Ibrahim, Rishabh Nahata, Pidathala Sowjanya, Deepak Kumarr, Gautam Bhargava, Chandra Khatri

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Historical Ink: 19th Century Latin American Spanish Newspaper Corpus with LLM OCR Correction

Laura Manrique-Gómez, Tony Montes, Arturo Rodríguez-Herrera, Rubén Manrique

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Journal ref ACL, Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities, pages 132-139, 2024

CFunModel: A "Funny" Language Model Capable of Chinese Humor Generation and Processing

Zhenghan Yu, Xinyu Hu, Xiaojun Wan

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 9 pages

SeLIP: Similarity Enhanced Contrastive Language Image Pretraining for Multi-modal Head MRI

Zhiyang Liu, Dong Yang, Minghao Zhang, Hanyu Sun, Hong Wu, Huiying Wang, Wen Shen, Chao Chai, Shuang Xia

专题命中 预训练与数据 ：pretraining(title,abstract)；foundation model(abstract)；分类 cs.AI