语言大模型 / LLM - arXivDaily 专题

2010.12547 2021-04-13 cs.CL 83%

Multilingual BERT Post-Pretraining Alignment

Lin Pan, Chung-Wei Hang, Haode Qi, Abhishek Shah, Saloni Potdar, Mo Yu

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments Accepted at NAACL2021

2011.08539 2020-11-18 cs.CL 83%

MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretraining

Wei Zhu

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

2011.04946 2020-11-11 cs.CL 83%

When Do You Need Billions of Words of Pretraining Data?

Yian Zhang, Alex Warstadt, Haau-Sing Li, Samuel R. Bowman

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 10 pages, 6 figures

2011.03203 2020-11-09 cs.CL 83%

Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining

Grigorii Guz, Patrick Huber, Giuseppe Carenini

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 10 pages, 1 figure, COLING 2020

2010.02480 2020-10-30 cs.CL 83%

Pretrained Language Model Embryology: The Birth of ALBERT

Cheng-Han Chiang, Sung-Feng Huang, Hung-yi Lee

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2020, short paper

2010.09403 2020-10-20 cs.CL 83%

Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation

Dušan Variš, Ondřej Bojar

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments ACL-SRW 2019 (camera-ready)

2009.07610 2020-10-07 cs.CL 83%

Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT

Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments EMNLP 2020, main conference

2009.11462 2020-09-29 cs.CL 83%

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, Noah A. Smith

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Findings in EMNLP 2020

2007.11648 2020-07-24 cs.CL 83%

Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models

Mittul Singh, Peter Smit, Sami Virpioja, Mikko Kurimo

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2005.13482 2020-05-28 cs.CL 83%

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, Phil Blunsom

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 17 pages, 6 tables, 2 figures. AK and LK contributed equally

2004.11026 2020-04-24 cs.CL 83%

QURIOUS: Question Generation Pretraining for Text Generation

Shashi Narayan, Gonçalo Simoes, Ji Ma, Hannah Craighead, Ryan Mcdonald

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 9 pages

1909.02339 2020-04-21 cs.CL 83%

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

Anne Lauscher, Ivan Vulić, Edoardo Maria Ponti, Anna Korhonen, Goran Glavaš

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

1912.10165 2019-12-24 cs.CL 83%

Zero-shot Text Classification With Generative Language Models

Raul Puri, Bryan Catanzaro

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

1912.09637 2019-12-23 cs.CL 83%

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model

Wenhan Xiong, Jingfei Du, William Yang Wang, Veselin Stoyanov

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted to ICLR 2020

1909.01066 2019-09-05 cs.CL 83%

Language Models as Knowledge Bases?

Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, Sebastian Riedel

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments accepted at EMNLP 2019

1906.02940 2019-07-30 cs.LG cs.CV eess.IV stat.ML 83%

Selfie: Self-supervised Pretraining for Image Embedding

Trieu H. Trinh, Minh-Thang Luong, Quoc V. Le

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.LG

1907.11692 2019-07-29 cs.CL 83%

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

1904.00585 2019-05-20 cs.CL 83%

Using Similarity Measures to Select Pretraining Data for NER

Xiang Dai, Sarvnaz Karimi, Ben Hachey, Cecile Paris

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments NAACL 2019

1902.07938 2019-02-22 cs.CL 83%

Pretrained language model transfer on neural named entity recognition in Indonesian conversational texts

Rezka Leonandya, Fariz Ikhwantri

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted in CICLing 2019

1901.09128 2019-01-29 cs.CL 83%

Language Model Pre-training for Hierarchical Document Representations

Ming-Wei Chang, Kristina Toutanova, Kenton Lee, Jacob Devlin

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

1809.10040 2019-01-08 cs.CL 83%

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Kelly W. Zhang, Samuel R. Bowman

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Journal ref Blackbox NLP Workshop, EMNLP 2018

1210.8440 2012-11-01 cs.CL 83%

Large Scale Language Modeling in Automatic Speech Recognition

Ciprian Chelba, Dan Bikel, Maria Shugrina, Patrick Nguyen, Shankar Kumar

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

cs/0108023 2009-11-30 cs.CL cs.IR 83%

Information Extraction Using the Structured Language Model

Ciprian Chelba, Milind Mahajan

专题命中预训练与数据：language model(title,abstract)；SLM(abstract)；分类 cs.CL

Comments EMNLP'01, Pittsburgh; 8 pages

Journal ref EMNLP/NAACL 2001 Conference Proceedings

cs/0001023 2009-11-30 cs.CL 83%

Structured Language Modeling for Speech Recognition

Ciprian Chelba, Frederick Jelinek

专题命中预训练与数据：language model(title,abstract)；SLM(abstract)；分类 cs.CL

Comments 4 pages + 2 pages of ERRATA

Journal ref Proceedings of NLDB'99, Klagenfurt, Austria

2606.11387 2026-06-11 cs.CL cs.AI cs.LG 新提交 83%

Small Experiments, Cheaper Decisions: A Case Study in Staged Promotion for Micro-Pretraining

小实验，更经济的决策：微预训练中分阶段提升的案例研究

Felipe Chavarro Polania

机构 * Hewlett Packard Enterprise（慧与科技公司）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结研究微预训练中分阶段提升协议，通过固定预算筛选配置，在Windows A100和Linux L40S上验证，发现早期排名不稳定，但最终协议以144 GPU小时找到最优配置，成本低于全量筛选。

Comments 14 pages, 5 figures; 12-hour dual-host micro-pretraining promotion study; source package includes curated ancillary artifacts

2602.17881 2026-02-23 cs.CL cs.AI cs.LG 83%

Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

理解语言模型中转向向量的不可靠性：几何预测器与线性近似的局限性

Joschka Braun

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG；foundation model(comments)

AI总结该研究探讨了语言模型中转向向量的可靠性问题，发现训练数据和行为表示对转向效果有显著影响，提出几何预测器和非线性近似方法以提升稳定性。

Comments Master's Thesis, University of Tübingen. 89 pages, 34 figures. Portions of this work were published at the ICLR 2025 Workshop on Foundation Models in the Wild (see arXiv:2505.22637)

2505.24167 2025-06-02 cs.CV 83%

Pretraining Deformable Image Registration Networks with Random Images

Junyu Chen, Shuwen Wei, Yihao Liu, Aaron Carass, Yong Du

机构 * Johns Hopkins University（约翰霍普金斯大学）； Vanderbilt University（范德比尔特大学）

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)

Comments Accepted by MIDL 2025. Code available at https://github.com/junyuchen245/Pretraining_Image_Registration_DNNs

2505.22533 2026-08-19 cs.LG cs.AI quant-ph 版本更新 82%

TabularQGAN: A quantum generative model for tabular data synthesis

TabularQGAN：一种用于表格数据合成的量子生成模型

Pallavi Bhardwaj, Caitlin Jones, Lasse Dierich, Aleksandar Vučković

机构 * SAP SE（SAP公司）； BASF Digital Solutions（巴斯夫数字解决方案）； QUTAC Quantum Technology and Application Consortium（QUTAC量子技术与应用联盟）； Technical University of Munich CIT（慕尼黑技术大学CIT）； Ludwig-Maximilians-Universität München（慕尼黑路德维希-马克西米利安大学）； Merck KGaA（默克公司）

专题命中预训练与数据：LLM(summary_cn,abstract)；分类 cs.AI、cs.LG

AI总结本文提出新型量子生成对抗网络TabularQGAN，针对异构表格数据建模，在MIMIC-III等数据集上与经典及LLM方法对比，取得有竞争力甚至领先的合成性能，验证了其泛化能力。

Comments 19 pages,8 figures and 4 tables

Journal ref Sci. Rep. 16, 23555 (2026)1-19

2608.10154 2026-08-12 cs.CL cs.AI 新提交 82%

Multimodal Item Parameter Estimation using Simulated Response Probabilitie

基于模拟响应概率的多模态题目参数估计

Christopher Ormerod, YoungKoung Kim

机构 * College Board（大学理事会）

专题命中预训练与数据：LLM(abstract,abstract_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本研究基于Qwen3.5微调多模态大语言模型，利用含图像与文本的多项选择题训练语料，通过学习学生能力相关的选择概率，实现了题目参数的准确估计。

Comments Submitted and Accepted for AIME-Con 2026

2608.07316 2026-08-10 cs.CL cs.AI cs.SI 新提交 82%

Natural Language Processing Psychometrics

自然语言处理心理测量学

Edoardo Sebastiano De Duro, Emma Franchino, Massimo Stella

机构 * University of Trento（特伦托大学）

专题命中预训练与数据：LLM(summary_cn,abstract)；分类 cs.CL、cs.AI

AI总结该研究将文本心理预测视为心理测量问题，用9种LLM角色完成问卷，结合多特征构建RF模型，解释了SWLS等的方差，还能区分角色、分类临床与对照参与者，明确了NLP心理测量的区分方法。

1. 预训练与数据 12393 篇

Multilingual BERT Post-Pretraining Alignment

MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretraining

When Do You Need Billions of Words of Pretraining Data?

Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining

Pretrained Language Model Embryology: The Birth of ALBERT

Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation

Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

QURIOUS: Question Generation Pretraining for Text Generation

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

Zero-shot Text Classification With Generative Language Models

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model

Language Models as Knowledge Bases?

Selfie: Self-supervised Pretraining for Image Embedding

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Using Similarity Measures to Select Pretraining Data for NER

Pretrained language model transfer on neural named entity recognition in Indonesian conversational texts

Language Model Pre-training for Hierarchical Document Representations

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Large Scale Language Modeling in Automatic Speech Recognition

Information Extraction Using the Structured Language Model

Structured Language Modeling for Speech Recognition

Small Experiments, Cheaper Decisions: A Case Study in Staged Promotion for Micro-Pretraining

Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

Pretraining Deformable Image Registration Networks with Random Images

TabularQGAN: A quantum generative model for tabular data synthesis

Multimodal Item Parameter Estimation using Simulated Response Probabilitie

Natural Language Processing Psychometrics