语言大模型 / LLM - arXivDaily 专题

2503.03962 2025-06-05 cs.CL 79%

On the Acquisition of Shared Grammatical Representations in Bilingual Language Models

Catherine Arnett, Tyler A. Chang, James A. Michaelov, Benjamin K. Bergen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 9 pages, 5 figures. Accepted at ACL 2025

2506.01172 2025-06-03 cs.CL 79%

The Inverse Scaling Effect of Pre-Trained Language Model Surprisal Is Not Due to Data Leakage

Byung-Doh Oh, Hongao Zhu, William Schuler

机构 * Center for Data Science（数据科学中心）； School of Foreign Languages（外国语言学院）； Department of Linguistics（语言学系）

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL Findings 2025; results with Natural Stories alignment issue corrected (commit 4700daa)

2402.15734 2025-06-03 cs.LG stat.ML 79%

Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning

Wuyang Chen, Jialin Song, Pu Ren, Shashank Subramanian, Dmitriy Morozov, Michael W. Mahoney

机构 * Simon Fraser University（西蒙弗雷泽大学）； Lawrence Berkeley National Laboratory（伯克利国家实验室）； International Computer Science Institute（国际计算机科学研究所）； University of California, Berkeley（加州大学伯克利分校）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2502.20864 2025-06-02 cs.CL 79%

Do Language Models Understand Honorific Systems in Javanese?

Mohammad Rifqi Farhansyah, Iwan Darmawan, Adryan Kusumawardhana, Genta Indra Winata, Alham Fikri Aji, Derry Tanti Wijaya

机构 * Institut Teknologi Bandung（技术科技大学）； Monash Indonesia（莫纳什印尼分校）； Boston University（波士顿大学）； Capital One ； MBZUAI

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments ACL 2025 - Main Conference

2502.14350 2025-05-29 cs.DB cs.LG 79%

Optimize Cardinality Estimation Model Pretraining by Simplifying the Training Datasets

Boyang Fang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Preliminary analysis uncovered data quality concerns in our experimental dataset

2410.17573 2025-05-28 cs.LG cs.CR 79%

Securing Federated Learning against Backdoor Threats with Foundation Model Integration

Xiaohuan Bi, Xi Li

机构 * Renmin University of China（中国人民大学）； The University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments This paper has been accepted to IEEE IRI 2025

2505.21137 2025-05-28 cs.CL cs.SD eess.AS 79%

Scaling and Prompting for Improved End-to-End Spoken Grammatical Error Correction

Mengjie Qian, Rao Ma, Stefano Bannò, Kate M. Knill, Mark J. F. Gales

机构 * University of CambridgeUK（剑桥大学）

专题命中预训练与数据：prompting(title)；foundation model(abstract)；分类 cs.CL

Comments submitted to Interspeech

2505.13089 2025-05-28 cs.CL 79%

Systematic Generalization in Language Models Scales with Information Entropy

Sondre Wold, Lucas Georges Gabriel Charpentier, Étienne Simon

机构 * Language Technology Group University of Oslo（奥斯陆大学语言技术组）

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2025: Findings

2505.20052 2025-05-27 cs.LG q-bio.QM 79%

Ankh3: Multi-Task Pretraining with Sequence Denoising and Completion Enhances Protein Representations

Hazem Alsamkary, Mohamed Elshaffei, Mohamed Elkerdawy, Ahmed Elnaggar

机构 * Proteinea Inc.（Proteinea公司）

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.LG

Comments 8 pages, 0 figures

2310.13312 2025-05-27 cs.CL 79%

Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models

Jaeyoung Choe, Keonwoong Noh, Nayeon Kim, Seyun Ahn, Woohwan Jung

机构 * Department of Applied Artificial Intelligence, Hanyang University（应用人工智能系，翰阳大学）； Division of Computer Science, Hanyang University（计算机科学系，翰阳大学）； Ramply Inc.（Ramply公司）

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023 (Findings)

2505.17631 2025-05-26 cs.IR cs.AI 79%

BehaveGPT: A Foundation Model for Large-scale User Behavior Modeling

Jiahui Gong, Jingtao Ding, Fanjin Meng, Chen Yang, Hong Chen, Zuojian Wang, Haisheng Lu, Yong Li

专题命中预训练与数据：foundation model(title)；pretraining(abstract)；分类 cs.AI

Comments 22 pages, 8 figures, 5 tables

2410.00871 2025-05-26 cs.CV cs.AI 79%

MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining

Yunze Liu, Li Yi

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Journal ref Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2025

2505.15559 2025-05-22 cs.SD cs.AI eess.AS 79%

Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes

Zixun Guo, Simon Dixon

机构 * Centre for Digital Music（数字音乐中心）； Queen Mary University of London（伦敦大学玛丽女王学院）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2505.12711 2025-05-21 cs.CV cs.AI 79%

Any-to-Any Learning in Computational Pathology via Triplet Multimodal Pretraining

Qichen Sun, Zhengrui Guo, Rui Peng, Hao Chen, Jinzhuo Wang

机构 * Peking University（北京大学）； The Hong Kong University of Science and Technology（香港科技大学）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2505.13466 2025-05-21 cs.AI 79%

AgentSGEN: Multi-Agent LLM in the Loop for Semantic Collaboration and GENeration of Synthetic Data

Vu Dinh Xuan, Hao Vo, David Murphy, Hoang D. Nguyen

机构 * University of Information Technology, VNU–HCM（越南胡志明市信息技术大学）； University College Cork（科尔克大学）

专题命中预训练与数据：LLM(title,abstract)；分类 cs.AI

2502.05505 2025-05-21 cs.LG cs.CR cs.CV stat.ML 79%

Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model

Zinan Lin, Tadas Baltrusaitis, Wenyu Wang, Sergey Yekhanin

机构 * Microsoft Research Redmond（微软研究院红mond分部）； Microsoft Cambridge（微软剑桥分部）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Published in: (1) ICLR 2025 Workshop on Data Problems, (2) ICLR 2025 Workshop on Synthetic Data

2502.08949 2025-05-20 cs.LG 79%

DICE: Device-level Integrated Circuits Encoder with Graph Contrastive Pretraining

Sungyoung Lee, Ziyi Wang, Seunggeun Kim, Taekyun Lee, Yao Lai, David Z. Pan

机构 * Electrical and Computer Engineering, The University of Texas at Austin（得克萨斯大学奥斯汀分校电子与计算机工程系）； Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）； Computer Science, The University of Hong Kong（香港大学计算机科学系）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2505.10848 2025-05-20 cs.LG 79%

Foundation model for mass spectrometry proteomics

Justin Sanders, Melih Yilmaz, Jacob H. Russell, Wout Bittremieux, William E. Fondrie, Nicholas M. Riley, Sewoong Oh, William Stafford Noble

机构 * Paul G. Allen School of Computer Science and Engineering, University of Washington（保罗·G·阿伦计算机科学与工程学院，华盛顿大学）； Department of Chemistry, University of Washington（华盛顿大学化学系）； Adrem Data Lab, University of Antwerp（阿德雷姆数据实验室，安特卫普大学）； Talus Bioscience（塔卢斯生物科学）； Department of Genome Sciences, University of Washington（华盛顿大学基因组科学系）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2305.15560 2025-05-20 cs.CV cs.CR cs.LG 79%

Differentially Private Synthetic Data via Foundation Model APIs 1: Images

Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, Sergey Yekhanin

机构 * Microsoft Research（微软研究院）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Published in ICLR 2024

2505.03844 2025-05-13 eess.IV cs.AI cs.CV 79%

From Spaceborne to Airborne: SAR Image Synthesis Using Foundation Models for Multi-Scale Adaptation

Solene Debuysere, Nicolas Trouve, Nathan Letheule, Olivier Leveque, Elise Colin

机构 * ONERA - DEMR ； Université Paris Saclay（巴黎萨克雷大学）； ONERA - DTIS

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2405.16731 2025-05-12 cs.LG cs.NE 79%

Pretraining with Random Noise for Fast and Robust Learning without Weight Transport

Jeonghwan Cheon, Sang Wan Lee, Se-Bum Paik

机构 * Department of Brain and Cognitive Sciences（脑科学与认知科学系）； Graduate School of Data Science（数据科学研究生院）； Kim Jaechul Graduate School of AI（金在哲人工智能研究生院）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Journal ref Advances in Neural Information Processing Systems 37, 13748-13768, 2024

2505.03426 2025-05-07 cs.CV cs.AI 79%

Phenotype-Guided Generative Model for High-Fidelity Cardiac MRI Synthesis: Advancing Pretraining and Clinical Applications

Ziyu Li, Yujian Hu, Zhengyao Ding, Yiheng Mao, Haitao Li, Fan Yi, Hongkun Zhang, Zhengxing Huang

机构 * Zhejiang University（浙江大学）； The First Affiliated Hospital of Zhejiang University School of Medicine（浙江大学医学院附属第一医院）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2408.06150 2025-05-06 cs.CL physics.chem-ph q-bio.BM 79%

LipidBERT: A Lipid Language Model Pre-trained on METiS de novo Lipid Library

Tianhao Yu, Cai Yao, Zhuorui Sun, Feng Shi, Lin Zhang, Kangjie Lyu, Xuan Bai, Andong Liu, Xicheng Zhang, Jiali Zou, Wenshou Wang, Chris Lai, Kai Wang

机构 * METiS Pharmaceuticals（METiS制药公司）

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2504.17315 2025-04-25 cs.CV cs.AI 79%

DIMT25@ICDAR2025: HW-TSC's End-to-End Document Image Machine Translation System Leveraging Large Vision-Language Model

Zhanglin Wu, Tengfei Song, Ning Xie, Weidong Zhang, Pengfei Li, Shuang Wu, Chong Li, Junhao Zhu, Hao Yang

机构 * Huawei Translation Service Center（华为翻译服务中心）

专题命中预训练与数据：language model(title,abstract)；分类 cs.AI

Comments 7 pages, 1 figures, 2 tables

2412.02856 2025-04-23 cs.CV cs.LG 79%

Is Large-Scale Pretraining the Secret to Good Domain Generalization?

Piotr Teterwak, Kuniaki Saito, Theodoros Tsiligkaridis, Bryan A. Plummer, Kate Saenko

机构 * Boston University（波士顿大学）； MIT Lincoln Laboratory（麻省理工学院林肯实验室）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted at ICLR 2025

2504.15432 2025-04-23 cs.CL 79%

Feeding LLM Annotations to BERT Classifiers at Your Own Risk

Yucheng Lu, Kazimier Smith

机构 * New York University（纽约大学）

专题命中预训练与数据：LLM(title,abstract)；分类 cs.CL

2401.10375 2025-04-22 cs.CR cs.DC cs.LG 79%

Foundation Models in Federated Learning: Assessing Backdoor Vulnerabilities

Xi Li, Chen Wu, Jiaqi Wang

机构 * University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）； Meta ； The Pennsylvania State University（宾夕法尼亚州立大学）

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Xi Li and Chen Wu are equal contribution. The corresponding author is Jiaqi Wang. This paper has been accepted by IJCNN 2025

2408.11054 2025-04-18 cs.CV cs.AI 79%

Near, far: Patch-ordering enhances vision foundation models' scene understanding

Valentinos Pariza, Mohammadreza Salehi, Gertjan Burghouts, Francesco Locatello, Yuki M. Asano

专题命中预训练与数据：foundation model(title)；pretraining(abstract)；分类 cs.AI

Comments Accepted at ICLR25. The webpage is accessible at: https://vpariza.github.io/NeCo/

2504.11054 2025-04-16 cs.LG 79%

Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models

Andrea Tirinzoni, Ahmed Touati, Jesse Farebrother, Mateusz Guzek, Anssi Kanervisto, Yingchen Xu, Alessandro Lazaric, Matteo Pirotta

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Published at ICLR 2025

2504.02116 2025-04-04 cs.CL 79%

Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive ziji

Xiulin Yang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref COLING 2025

1. 预训练与数据 12393 篇

On the Acquisition of Shared Grammatical Representations in Bilingual Language Models

The Inverse Scaling Effect of Pre-Trained Language Model Surprisal Is Not Due to Data Leakage

Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning

Do Language Models Understand Honorific Systems in Javanese?

Optimize Cardinality Estimation Model Pretraining by Simplifying the Training Datasets

Securing Federated Learning against Backdoor Threats with Foundation Model Integration

Scaling and Prompting for Improved End-to-End Spoken Grammatical Error Correction

Systematic Generalization in Language Models Scales with Information Entropy

Ankh3: Multi-Task Pretraining with Sequence Denoising and Completion Enhances Protein Representations

Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models

BehaveGPT: A Foundation Model for Large-scale User Behavior Modeling

MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining

Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes

Any-to-Any Learning in Computational Pathology via Triplet Multimodal Pretraining

AgentSGEN: Multi-Agent LLM in the Loop for Semantic Collaboration and GENeration of Synthetic Data

Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model

DICE: Device-level Integrated Circuits Encoder with Graph Contrastive Pretraining

Foundation model for mass spectrometry proteomics

Differentially Private Synthetic Data via Foundation Model APIs 1: Images

From Spaceborne to Airborne: SAR Image Synthesis Using Foundation Models for Multi-Scale Adaptation

Pretraining with Random Noise for Fast and Robust Learning without Weight Transport

Phenotype-Guided Generative Model for High-Fidelity Cardiac MRI Synthesis: Advancing Pretraining and Clinical Applications

LipidBERT: A Lipid Language Model Pre-trained on METiS de novo Lipid Library

DIMT25@ICDAR2025: HW-TSC's End-to-End Document Image Machine Translation System Leveraging Large Vision-Language Model

Is Large-Scale Pretraining the Secret to Good Domain Generalization?

Feeding LLM Annotations to BERT Classifiers at Your Own Risk

Foundation Models in Federated Learning: Assessing Backdoor Vulnerabilities

Near, far: Patch-ordering enhances vision foundation models' scene understanding

Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models

Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive ziji