语言大模型 / LLM

Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task Learning

Zehui Li, Vallijah Subasri, Yifei Shen, Dongsheng Li, Yiren Zhao, Guy-Bart Stan, Caihua Shan

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision

Yunyi Zhang, Ruozhen Yang, Xueqiang Xu, Rui Li, Jinfeng Xiao, Jiaming Shen, Jiawei Han

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted to WWW 2025 Research Track

BTS: Harmonizing Specialized Experts into a Generalist LLM

Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan, Suchin Gururangan, Mike Lewis

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs

Zixuan Hu, Yongxian Wei, Li Shen, Chun Yuan, Dacheng Tao

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Crystal: Illuminating LLM Abilities on Language and Code

Tianhua Tao, Junbo Li, Bowen Tan, Hongyi Wang, William Marshall, Bhargav M Kanakiya, Joel Hestness, Natalia Vassilieva, Zhiqiang Shen, Eric P. Xing, Zhengzhong Liu

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Published as a conference paper at COLM 2024

Language Agents Meet Causality -- Bridging LLMs and Causal World Models

John Gkountouras, Matthias Lindemann, Phillip Lippe, Efstratios Gavves, Ivan Titov

专题命中 预训练与数据 ：language agent(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Project page: https://j0hngou.github.io/LLMCWM/

OpenGraph: Towards Open Graph Foundation Models

Lianghao Xia, Ben Kao, Chao Huang

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by EMNLP'2024

A multimodal LLM for the non-invasive decoding of spoken text from brain recordings

Youssef Hmamouche, Ismail Chihab, Lahoucine Kdouri, Amal El Fallah Seghrouchni

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments 15 pages, 4 figures

dnaGrinder: a lightweight and high-capacity genomic foundation model

Qihang Zhao, Chi Zhang, Weixiong Zhang

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Source-Aware Training Enables Knowledge Attribution in Language Models

Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, Hao Peng

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments COLM '24

IGOT: Information Gain Optimized Tokenizer on Domain Adaptive Pretraining

Dawei Feng, Yihai Zhang, Zhixuan Xu

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web

Yibo Yan, Haomin Wen, Siru Zhong, Wei Chen, Haodong Chen, Qingsong Wen, Roger Zimmermann, Yuxuan Liang

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by The Web Conference 2024

Learning to Deliver: a Foundation Model for the Montreal Capacitated Vehicle Routing Problem

Samuel J. K. Chin, Matthias Winkenbach, Akash Srivastava

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

Biao Zhang, Zhongtao Liu, Colin Cherry, Orhan Firat

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments ICLR24

She had Cobalt Blue Eyes: Prompt Testing to Create Aligned and Sustainable Language Models

Veronica Chatrath, Oluwanifemi Bamgbose, Shaina Raza

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

Comments Accepted as Oral at the AAAI 2nd Workshop on Sustainable AI

Proving Test Set Contamination in Black Box Language Models

Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, Tatsunori B. Hashimoto

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；prompting(abstract)

Skywork: A More Open Bilingual Foundation Model

Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Haihua Yang, Biye Li, Cheng Cheng, Weiwei Lü, Rui Hu, Chenxia Li, Liu Yang, Xilin Luo, Xuejie Wu, Lunan Liu, Wenjun Cheng, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Lei Lin, Xiaokun Wang, Yutuan Ma, Chuanhai Dong, Yanqi Sun, Yifu Chen, Yongyi Peng, Xiaojuan Liang, Shuicheng Yan, Han Fang, Yahui Zhou

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization

Cheng Deng, Tianhang Zhang, Zhongmou He, Yi Xu, Qiyuan Chen, Yuanyuan Shi, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin, Junxian He

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；instruction tuning(abstract)

The Wisdom of Hindsight Makes Language Models Better Instruction Followers

Tianjun Zhang, Fangchen Liu, Justin Wong, Pieter Abbeel, Joseph E. Gonzalez

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；RLHF(abstract)

General Intelligence Requires Reward-based Pretraining

Seungwook Han, Jyothish Pari, Samuel J. Gershman, Pulkit Agrawal

机构 * Improbable AI Lab, MIT（Improbable AI Lab，麻省理工学院） ； Department of Psychology（心理学系） ； Center for Brain Science, Harvard University（哈佛大学脑科学中心） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments https://improbableai.notion.site/General-Intelligence-Requires-Reward-Based-Pretraining-2023b66e4cf580d3ab44c7860b75d25f?pvs=73

SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge

Yuxiang Mei, Yuang Zheng, Dongxing Xu, Yanhua Long

机构 * Shanghai Normal University（上海师范大学） ； Unisound AI Technology Co., Ltd.（Unisound人工智能技术有限公司） 纠错

专题命中 预训练与数据 ：SLM(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by Interspeech 2025 MLC-SLM workshop

CoGate: Confidence-Gated Co-Decoding for Secure Code Generation

CoGate：用于安全代码生成的置信门控协同解码

Minghao Hu, Lannan Luo, Allen Roush, Phillip Howard

专题命中 预训练与数据 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

AI总结 针对现有协同解码未考虑专家模型置信度的问题，提出CoGate方法，在多LLM后端和代码生成基准上优于CoSec+，在CWEval的Func-Sec@10指标最高提升12.6%。

Auditing pretraining contamination in single-cell foundation model benchmarks

单细胞基础模型基准中的预训练污染审计

Sarwan Ali

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)

AI总结 研究单细胞基础模型基准中的预训练污染问题，提出scContam审计框架，结合指纹信号与成员推理攻击。应用于多个基准和模型，发现部分基准有污染，且MIA-scFM的AUROC与模型容量数据比有关，证明预训练审计可行且应伴随基准报告。

The CRAFT principles for the responsible use of large language models in policymaking

政策制定中负责任使用大语言模型的CRAFT原则

Willem Fourie, Gray Manicom, Tanya de Villiers-Botha

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)

AI总结 研究政策制定中如何利用大语言模型，提出CRAFT原则，即控制、严谨、问责、公平和透明，以在管理风险的同时充分利用大语言模型改善政策制定过程。

Alternative routes to universal diversity scaling in component systems: from proteomes to large language models

组件系统中通用多样性标度的替代路径：从蛋白质组到大语言模型

Andrea Mazzolini, Leonardo Agasso, Filippo Valle, Michele Caselle, Marco Cosentino Lagomarsino, Matteo Osella

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)

AI总结 研究复杂组件系统中多样性标度定律的涌现条件，发现创新驱动增长模型与潜在异质性模型均能产生相同的宏观定律，但无法唯一确定生成机制。

Understanding the (In)Security of Vibe-Coded Applications

理解“氛围编码”应用的安全性（与非安全性）

Junquan Deng, Zhiyu Fan, Ruijie Meng

专题命中 预训练与数据 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

AI总结 本文系统研究了通过“氛围编码”范式开发的应用的安全性，发现其存在占位符逻辑、未过滤输入和秘密泄露等重复性漏洞模式，根源在于AI代理的系统性局限，且即使改进LLM能力也无法消除根本风险。

Formal-Method-Guided Vibe Coding: Closing the Verification Loop on AI-Generated Safety-Critical Software Through Model-Driven Engineering

形式化方法引导的Vibe编码：通过模型驱动工程关闭AI生成安全关键软件的验证循环

Ran Wei, Le Zhu, Haochi Wang, Jim Woodcock, Fang Yan, Simon Foster, Xiangyang Ji

专题命中 预训练与数据 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

AI总结 提出Forge管道，结合Vibe编码与模型驱动工程，通过多种形式化验证（Dafny、CSP、Isabelle）迭代修正LLM生成的Java代码，生成符合安全标准的验证证据。

Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretraining

Brain-DiT：一种基于元数据条件预训练的通用多状态fMRI基础模型

Junfeng Xia, Wenhao Ye, Xuanye Pan, Xinke Shen, Mo Wang, Quanying Liu

机构 * Department of Biomedical Engineering, Southern University of Science and Technology, China（南方科技大学生物医学工程系，中国） ； School of Biomedical Engineering, Shenzhen University, China（深圳大学生物医学工程学院，中国） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)

AI总结 提出Brain-DiT，一种基于扩散Transformer和元数据条件预训练的通用多状态fMRI基础模型，在24个数据集上预训练，通过生成式预训练学习多尺度表示，在7个下游任务中优于重建和对齐方法。

Scaling Laws for Moral Machine Judgment in Large Language Models

大语言模型中道德机器判断的扩展规律

Kazuhiro Takemoto

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)

AI总结 研究通过评估75种大语言模型配置，发现模型规模与人类偏好距离呈幂律关系，扩展推理模型在较小规模时表现更优，为价值判断的扩展规律研究提供依据。

Comments 12 pages, 4 figures, 3 tables

Journal ref R Soc Open Sci. (2026) 13 (6): 260202

Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis

多智能体大语言模型能否识别其同类？角色约束政治分析中的笔迹风格指纹识别

Juergen Dietrich

机构 * Democracy Intelligence gGmbH 纠错

专题命中 预训练与数据 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 研究多智能体LLM在政治分析中能否通过笔迹风格识别模型家族，提出SD-CV协议，T5模型在五类归属任务中达到F1=0.991，证明提示级匿名化无法消除模型身份信号。

Comments 24 pages, 3 figures