语言大模型 / LLM - arXivDaily 专题

2010.09403 2020-10-20 cs.CL 83%

Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation

Dušan Variš, Ondřej Bojar

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments ACL-SRW 2019 (camera-ready)

2009.07610 2020-10-07 cs.CL 83%

Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT

Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments EMNLP 2020, main conference

2009.11462 2020-09-29 cs.CL 83%

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, Noah A. Smith

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Findings in EMNLP 2020

2007.11648 2020-07-24 cs.CL 83%

Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models

Mittul Singh, Peter Smit, Sami Virpioja, Mikko Kurimo

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2005.13482 2020-05-28 cs.CL 83%

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, Phil Blunsom

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 17 pages, 6 tables, 2 figures. AK and LK contributed equally

2004.11026 2020-04-24 cs.CL 83%

QURIOUS: Question Generation Pretraining for Text Generation

Shashi Narayan, Gonçalo Simoes, Ji Ma, Hannah Craighead, Ryan Mcdonald

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments 9 pages

1909.02339 2020-04-21 cs.CL 83%

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

Anne Lauscher, Ivan Vulić, Edoardo Maria Ponti, Anna Korhonen, Goran Glavaš

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

1912.10165 2019-12-24 cs.CL 83%

Zero-shot Text Classification With Generative Language Models

Raul Puri, Bryan Catanzaro

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

1912.09637 2019-12-23 cs.CL 83%

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model

Wenhan Xiong, Jingfei Du, William Yang Wang, Veselin Stoyanov

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted to ICLR 2020

1909.01066 2019-09-05 cs.CL 83%

Language Models as Knowledge Bases?

Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, Sebastian Riedel

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments accepted at EMNLP 2019

1906.02940 2019-07-30 cs.LG cs.CV eess.IV stat.ML 83%

Selfie: Self-supervised Pretraining for Image Embedding

Trieu H. Trinh, Minh-Thang Luong, Quoc V. Le

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.LG

1907.11692 2019-07-29 cs.CL 83%

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

1904.00585 2019-05-20 cs.CL 83%

Using Similarity Measures to Select Pretraining Data for NER

Xiang Dai, Sarvnaz Karimi, Ben Hachey, Cecile Paris

专题命中预训练与数据：pretraining(title,abstract)；language model(abstract)；分类 cs.CL

Comments NAACL 2019

1902.07938 2019-02-22 cs.CL 83%

Pretrained language model transfer on neural named entity recognition in Indonesian conversational texts

Rezka Leonandya, Fariz Ikhwantri

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments Accepted in CICLing 2019

1901.09128 2019-01-29 cs.CL 83%

Language Model Pre-training for Hierarchical Document Representations

Ming-Wei Chang, Kristina Toutanova, Kenton Lee, Jacob Devlin

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

1809.10040 2019-01-08 cs.CL 83%

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Kelly W. Zhang, Samuel R. Bowman

专题命中预训练与数据：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

Journal ref Blackbox NLP Workshop, EMNLP 2018

1210.8440 2012-11-01 cs.CL 83%

Large Scale Language Modeling in Automatic Speech Recognition

Ciprian Chelba, Dan Bikel, Maria Shugrina, Patrick Nguyen, Shankar Kumar

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；分类 cs.CL

cs/0108023 2009-11-30 cs.CL cs.IR 83%

Information Extraction Using the Structured Language Model

Ciprian Chelba, Milind Mahajan

专题命中预训练与数据：language model(title,abstract)；SLM(abstract)；分类 cs.CL

Comments EMNLP'01, Pittsburgh; 8 pages

Journal ref EMNLP/NAACL 2001 Conference Proceedings

cs/0001023 2009-11-30 cs.CL 83%

Structured Language Modeling for Speech Recognition

Ciprian Chelba, Frederick Jelinek

专题命中预训练与数据：language model(title,abstract)；SLM(abstract)；分类 cs.CL

Comments 4 pages + 2 pages of ERRATA

Journal ref Proceedings of NLDB'99, Klagenfurt, Austria

2606.11387 2026-06-11 cs.CL cs.AI cs.LG 新提交 83%

Small Experiments, Cheaper Decisions: A Case Study in Staged Promotion for Micro-Pretraining

小实验，更经济的决策：微预训练中分阶段提升的案例研究

Felipe Chavarro Polania

机构 * Hewlett Packard Enterprise（慧与科技公司）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结研究微预训练中分阶段提升协议，通过固定预算筛选配置，在Windows A100和Linux L40S上验证，发现早期排名不稳定，但最终协议以144 GPU小时找到最优配置，成本低于全量筛选。

Comments 14 pages, 5 figures; 12-hour dual-host micro-pretraining promotion study; source package includes curated ancillary artifacts

2602.17881 2026-02-23 cs.CL cs.AI cs.LG 83%

Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

理解语言模型中转向向量的不可靠性：几何预测器与线性近似的局限性

Joschka Braun

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG；foundation model(comments)

AI总结该研究探讨了语言模型中转向向量的可靠性问题，发现训练数据和行为表示对转向效果有显著影响，提出几何预测器和非线性近似方法以提升稳定性。

Comments Master's Thesis, University of Tübingen. 89 pages, 34 figures. Portions of this work were published at the ICLR 2025 Workshop on Foundation Models in the Wild (see arXiv:2505.22637)

2505.24167 2025-06-02 cs.CV 83%

Pretraining Deformable Image Registration Networks with Random Images

Junyu Chen, Shuwen Wei, Yihao Liu, Aaron Carass, Yong Du

机构 * Johns Hopkins University（约翰霍普金斯大学）； Vanderbilt University（范德比尔特大学）

专题命中预训练与数据：pretraining(title,abstract)；foundation model(abstract)

Comments Accepted by MIDL 2025. Code available at https://github.com/junyuchen245/Pretraining_Image_Registration_DNNs

2505.22533 2026-08-19 cs.LG cs.AI quant-ph 版本更新 82%

TabularQGAN: A quantum generative model for tabular data synthesis

TabularQGAN：一种用于表格数据合成的量子生成模型

Pallavi Bhardwaj, Caitlin Jones, Lasse Dierich, Aleksandar Vučković

机构 * SAP SE（SAP公司）； BASF Digital Solutions（巴斯夫数字解决方案）； QUTAC Quantum Technology and Application Consortium（QUTAC量子技术与应用联盟）； Technical University of Munich CIT（慕尼黑技术大学CIT）； Ludwig-Maximilians-Universität München（慕尼黑路德维希-马克西米利安大学）； Merck KGaA（默克公司）

专题命中预训练与数据：LLM(summary_cn,abstract)；分类 cs.AI、cs.LG

AI总结本文提出新型量子生成对抗网络TabularQGAN，针对异构表格数据建模，在MIMIC-III等数据集上与经典及LLM方法对比，取得有竞争力甚至领先的合成性能，验证了其泛化能力。

Comments 19 pages,8 figures and 4 tables

Journal ref Sci. Rep. 16, 23555 (2026)1-19

2608.10154 2026-08-12 cs.CL cs.AI 新提交 82%

Multimodal Item Parameter Estimation using Simulated Response Probabilitie

基于模拟响应概率的多模态题目参数估计

Christopher Ormerod, YoungKoung Kim

机构 * College Board（大学理事会）

专题命中预训练与数据：LLM(abstract,abstract_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本研究基于Qwen3.5微调多模态大语言模型，利用含图像与文本的多项选择题训练语料，通过学习学生能力相关的选择概率，实现了题目参数的准确估计。

Comments Submitted and Accepted for AIME-Con 2026

2608.07316 2026-08-10 cs.CL cs.AI cs.SI 新提交 82%

Natural Language Processing Psychometrics

自然语言处理心理测量学

Edoardo Sebastiano De Duro, Emma Franchino, Massimo Stella

机构 * University of Trento（特伦托大学）

专题命中预训练与数据：LLM(summary_cn,abstract)；分类 cs.CL、cs.AI

AI总结该研究将文本心理预测视为心理测量问题，用9种LLM角色完成问卷，结合多特征构建RF模型，解释了SWLS等的方差，还能区分角色、分类临床与对照参与者，明确了NLP心理测量的区分方法。

2601.18904 2026-08-10 cs.SD cs.AI cs.CL 版本更新 82%

MetaSICL: Globalizing Auditory LLMs for Underserved Speakers and Languages via Meta Speech In-Context Learning

MetaSICL: 通过元语音上下文学习适应听觉大语言模型

Haolong Zheng, Siyin Wang, Zengrui Jin, Mark Hasegawa-Johnson

机构 * University of Illinois Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）； Tsinghua University（清华大学）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

AI总结提出MetaSICL方法，利用高资源语音数据通过元学习增强听觉大语言模型的上下文学习能力，在低资源场景下优于直接微调。

2608.03372 2026-08-05 cs.CL cs.AI 新提交 82%

FACTWASH: Catching AI Rewrites That Wash Hearsay into Fact

FACTWASH：捕捉将传闻洗成事实的AI改写

Alex Kwon

专题命中预训练与数据：LLM(summary_cn,abstract)；分类 cs.CL、cs.AI

AI总结本研究提出开源工具factwash，用于捕捉AI将传闻洗成事实的改写，明确否定线索检测F1达0.91，LLM见证者可提升线索检测召回率，在mem0 2.0.7上标记8个模糊传闻写入中的5个。

Comments 15 pages, 3 figures. Code and data: https://github.com/collapseindex/factwash

2607.25135 2026-07-29 cs.AI cs.LG 新提交 82%

ScalableRAG: High-Quality RAG at Zero Ingestion Cost

ScalableRAG：零摄入成本下的高质量检索增强生成

Hilaf Hasson, Aditya Chakravarty, Jayant Thomas, Krishna Gogineni

机构 * Cohesity（科赫思）

专题命中预训练与数据：LLM(summary_cn,abstract)；分类 cs.AI、cs.LG

AI总结研究提出零摄入成本的ScalableRAG及有限摄入的改进版本，通过维护工作区实现即时聚合推理，在六个语料库测试中表现出色，平均准确率大幅超越基线，还通过固定LLM调用次数等进一步提升大规模时的准确率。

2605.09504 2026-07-24 cs.CR cs.AI cs.LG 版本更新 82%

AI Security Policy Should Assess Systems, Not Only Models

位置：AI安全政策应针对系统，而非模型

Michael A. Riegler, Inga Strümke

机构 * AI Safety Department（人工智能安全部门）； SimulaMet and OsloMet（SimulaMet和奥斯陆计量）

专题命中预训练与数据：LLM(summary_cn,abstract)；分类 cs.AI、cs.LG

AI总结本文提出swarm-attack框架，展示通过协调轻量级LLM代理发现系统漏洞和绕过安全机制的可能性，证明在低成本硬件上可实现零成本安全测试。

2607.10534 2026-07-14 cs.AI cs.CR cs.LG 新提交 82%

Cross-Layer Misalignment Detection in Agent Skills: A Progressive Loading-Aware Contrastive Learning Approach

智能体技能中的跨层错位检测：一种渐进式加载感知对比学习方法

Chengjun Zhang, Yang Gao, Jianna Hur, Jingjing Zhang, Sagar Samtani

机构 * Indiana University Bloomington（印第安纳大学布卢明顿分校）

专题命中预训练与数据：LLM(abstract,abstract_cn)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

AI总结研究智能体技能跨层错位问题，提出渐进式加载感知分层对比学习框架PL-HCL，通过建模技能分层结构和学习跨层一致性来检测错位，使用相关语料库与挑战集，提升宏F1指标，为用户和运营商提供筛选工具与设计原则。

Comments 10 pages, 5 pages supplemental. Accepted at the KDD 2026 Workshop on Evaluation and Trustworthiness of Agentic AI

1. 预训练与数据 12379 篇

Unsupervised Pretraining for Neural Machine Translation Using Elastic Weight Consolidation

Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

Effects of Language Relatedness for Cross-lingual Transfer Learning in Character-Based Language Models

Syntactic Structure Distillation Pretraining For Bidirectional Encoders

QURIOUS: Question Generation Pretraining for Text Generation

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

Zero-shot Text Classification With Generative Language Models

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model

Language Models as Knowledge Bases?

Selfie: Self-supervised Pretraining for Image Embedding

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Using Similarity Measures to Select Pretraining Data for NER

Pretrained language model transfer on neural named entity recognition in Indonesian conversational texts

Language Model Pre-training for Hierarchical Document Representations

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

Large Scale Language Modeling in Automatic Speech Recognition

Information Extraction Using the Structured Language Model

Structured Language Modeling for Speech Recognition

Small Experiments, Cheaper Decisions: A Case Study in Staged Promotion for Micro-Pretraining

Understanding Unreliability of Steering Vectors in Language Models: Geometric Predictors and the Limits of Linear Approximations

Pretraining Deformable Image Registration Networks with Random Images

TabularQGAN: A quantum generative model for tabular data synthesis

Multimodal Item Parameter Estimation using Simulated Response Probabilitie

Natural Language Processing Psychometrics

MetaSICL: Globalizing Auditory LLMs for Underserved Speakers and Languages via Meta Speech In-Context Learning

FACTWASH: Catching AI Rewrites That Wash Hearsay into Fact

ScalableRAG: High-Quality RAG at Zero Ingestion Cost

AI Security Policy Should Assess Systems, Not Only Models

Cross-Layer Misalignment Detection in Agent Skills: A Progressive Loading-Aware Contrastive Learning Approach