语言大模型 / LLM

Mixture-of-Personas Language Models for Population Simulation

Ngoc Bui, Hieu Trung Nguyen, Shantanu Kumar, Julian Theodore, Weikang Qiu, Viet Anh Nguyen, Rex Ying

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

The Lucie-7B LLM and the Lucie Training Dataset: Open resources for multilingual language generation

Olivier Gouvert, Julie Hunter, Jérôme Louradour, Christophe Cerisara, Evan Dufraisse, Yaya Sy, Laura Rivière, Jean-Pierre Lorré, OpenLLM-France community

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；foundation model(abstract)

GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs

Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, Siliang Tang

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Accepted to WWW'25

Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task Learning

Zehui Li, Vallijah Subasri, Yifei Shen, Dongsheng Li, Yiren Zhao, Guy-Bart Stan, Caihua Shan

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision

Yunyi Zhang, Ruozhen Yang, Xueqiang Xu, Rui Li, Jinfeng Xiao, Jiaming Shen, Jiawei Han

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted to WWW 2025 Research Track

BTS: Harmonizing Specialized Experts into a Generalist LLM

Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan, Suchin Gururangan, Mike Lewis

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs

Zixuan Hu, Yongxian Wei, Li Shen, Chun Yuan, Dacheng Tao

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Crystal: Illuminating LLM Abilities on Language and Code

Tianhua Tao, Junbo Li, Bowen Tan, Hongyi Wang, William Marshall, Bhargav M Kanakiya, Joel Hestness, Natalia Vassilieva, Zhiqiang Shen, Eric P. Xing, Zhengzhong Liu

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Published as a conference paper at COLM 2024

Language Agents Meet Causality -- Bridging LLMs and Causal World Models

John Gkountouras, Matthias Lindemann, Phillip Lippe, Efstratios Gavves, Ivan Titov

专题命中 预训练与数据 ：language agent(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Project page: https://j0hngou.github.io/LLMCWM/

OpenGraph: Towards Open Graph Foundation Models

Lianghao Xia, Ben Kao, Chao Huang

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by EMNLP'2024

A multimodal LLM for the non-invasive decoding of spoken text from brain recordings

Youssef Hmamouche, Ismail Chihab, Lahoucine Kdouri, Amal El Fallah Seghrouchni

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments 15 pages, 4 figures

dnaGrinder: a lightweight and high-capacity genomic foundation model

Qihang Zhao, Chi Zhang, Weixiong Zhang

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Source-Aware Training Enables Knowledge Attribution in Language Models

Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, Hao Peng

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments COLM '24

IGOT: Information Gain Optimized Tokenizer on Domain Adaptive Pretraining

Dawei Feng, Yihai Zhang, Zhixuan Xu

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web

Yibo Yan, Haomin Wen, Siru Zhong, Wei Chen, Haodong Chen, Qingsong Wen, Roger Zimmermann, Yuxuan Liang

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by The Web Conference 2024

Learning to Deliver: a Foundation Model for the Montreal Capacitated Vehicle Routing Problem

Samuel J. K. Chin, Matthias Winkenbach, Akash Srivastava

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

Biao Zhang, Zhongtao Liu, Colin Cherry, Orhan Firat

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments ICLR24

She had Cobalt Blue Eyes: Prompt Testing to Create Aligned and Sustainable Language Models

Veronica Chatrath, Oluwanifemi Bamgbose, Shaina Raza

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

Comments Accepted as Oral at the AAAI 2nd Workshop on Sustainable AI

Proving Test Set Contamination in Black Box Language Models

Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, Tatsunori B. Hashimoto

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；prompting(abstract)

Skywork: A More Open Bilingual Foundation Model

Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Haihua Yang, Biye Li, Cheng Cheng, Weiwei Lü, Rui Hu, Chenxia Li, Liu Yang, Xilin Luo, Xuejie Wu, Lunan Liu, Wenjun Cheng, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Lei Lin, Xiaokun Wang, Yutuan Ma, Chuanhai Dong, Yanqi Sun, Yifu Chen, Yongyi Peng, Xiaojuan Liang, Shuicheng Yan, Han Fang, Yahui Zhou

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization

Cheng Deng, Tianhang Zhang, Zhongmou He, Yi Xu, Qiyuan Chen, Yuanyuan Shi, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin, Junxian He

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；instruction tuning(abstract)

The Wisdom of Hindsight Makes Language Models Better Instruction Followers

Tianjun Zhang, Fangchen Liu, Justin Wong, Pieter Abbeel, Joseph E. Gonzalez

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；RLHF(abstract)

General Intelligence Requires Reward-based Pretraining

Seungwook Han, Jyothish Pari, Samuel J. Gershman, Pulkit Agrawal

机构 * Improbable AI Lab, MIT（Improbable AI Lab，麻省理工学院） ； Department of Psychology（心理学系） ； Center for Brain Science, Harvard University（哈佛大学脑科学中心） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments https://improbableai.notion.site/General-Intelligence-Requires-Reward-Based-Pretraining-2023b66e4cf580d3ab44c7860b75d25f?pvs=73

SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge

Yuxiang Mei, Yuang Zheng, Dongxing Xu, Yanhua Long

机构 * Shanghai Normal University（上海师范大学） ； Unisound AI Technology Co., Ltd.（Unisound人工智能技术有限公司） 纠错

专题命中 预训练与数据 ：SLM(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by Interspeech 2025 MLC-SLM workshop

BrainNorm: A Foundation Model that knows Normal via Semantic Atlas Pretraining

BrainNorm：一种通过语义图谱预训练了解正常状态的基础模型

Madhumitha Venkatesh, Shanawaj S Madarkar, Konda Reddy Mopuri

机构 * Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)

AI总结 该研究提出BrainNorm基础模型，经约6.6万例T1w sMRI数据训练，通过语义图谱预训练学习规范表征，在多类下游任务上泛化性能优异，其识别的神经退行性偏差与临床病理吻合。

Know3D: Prompting 3D Generation with Knowledge from Vision-Language Models

Know3D: 通过视觉-语言模型的知识提示3D生成

Wenyue Chen, Wenjue Chen, Peng Li, Qinghe Wang, Xu Jia, Heliang Zheng, Rongfei Jia, Yuan Liu, Ronggang Wang

机构 * Peking University（北京大学） ； Math Magic（数学魔术） ； The Hong Kong University of Science and Technology（香港科学与技术大学） ； Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology（广东省超高清沉浸媒体技术重点实验室） ； Dalian University of Technology（大连理工大学） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；prompting(title)；large language model(abstract)

AI总结 本文提出Know3D框架，利用多模态大语言模型的知识提升3D生成的可控性，通过潜在状态注入实现语言可控的后视图生成，改进3D生成的几何细节与一致性。

Comments ECCV2026 page: this https URL (https://xishuxishu.github.io/Know3D.github.io/)

CoGate: Confidence-Gated Co-Decoding for Secure Code Generation

CoGate：用于安全代码生成的置信门控协同解码

Minghao Hu, Lannan Luo, Allen Roush, Phillip Howard

专题命中 预训练与数据 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

AI总结 针对现有协同解码未考虑专家模型置信度的问题，提出CoGate方法，在多LLM后端和代码生成基准上优于CoSec+，在CWEval的Func-Sec@10指标最高提升12.6%。

Auditing pretraining contamination in single-cell foundation model benchmarks

单细胞基础模型基准中的预训练污染审计

Sarwan Ali

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)

AI总结 研究单细胞基础模型基准中的预训练污染问题，提出scContam审计框架，结合指纹信号与成员推理攻击。应用于多个基准和模型，发现部分基准有污染，且MIA-scFM的AUROC与模型容量数据比有关，证明预训练审计可行且应伴随基准报告。

The CRAFT principles for the responsible use of large language models in policymaking

政策制定中负责任使用大语言模型的CRAFT原则

Willem Fourie, Gray Manicom, Tanya de Villiers-Botha

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)

AI总结 研究政策制定中如何利用大语言模型，提出CRAFT原则，即控制、严谨、问责、公平和透明，以在管理风险的同时充分利用大语言模型改善政策制定过程。

Alternative routes to universal diversity scaling in component systems: from proteomes to large language models

组件系统中通用多样性标度的替代路径：从蛋白质组到大语言模型

Andrea Mazzolini, Leonardo Agasso, Filippo Valle, Michele Caselle, Marco Cosentino Lagomarsino, Matteo Osella

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)

AI总结 研究复杂组件系统中多样性标度定律的涌现条件，发现创新驱动增长模型与潜在异质性模型均能产生相同的宏观定律，但无法唯一确定生成机制。