语言大模型 / LLM

ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool Learning

ToolACE-R: 基于模型的迭代训练与自适应细化用于工具学习

Xingshan Zeng, Weiwen Liu, Xu Huang, Zezhong Wang, Lingzhi Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruiming Tang, Qun Liu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 ToolACE-R通过基于模型的迭代训练和自适应细化，提升工具学习的效率和性能。

Comments Accepted by AAAI2026

Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers

激活 oracle：训练和评估 LLM 作为通用目的激活解释器

Adam Karvonen, James Chua, Clément Dumas, Kit Fraser-Taliente, Subhash Kantamneni, Julian Minder, Euan Ong, Arnab Sen Sharma, Daniel Wen, Owain Evans, Samuel Marks

机构 * MATS ； Truthful AI ； EPFL（瑞士联邦理工学院） ； ENS Paris-Saclay（巴黎-萨克雷大学） ； Northeastern University（东北大学） ； Anthropic 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 本文提出了一种通用的 LLM 激活解释器，通过多样化训练在多个下游任务中表现出色，能恢复模型内部信息。

Comments 36 pages

Artificial or Just Artful? Do LLMs Bend the Rules in Programming?

人工还是艺术？LLMs在编程中是否会打破规则？

Oussama Ben Sghaier, Kevin Delcourt, Houari Sahraoui

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；prompting(abstract)

AI总结 本文研究了LLMs在不同提示条件下如何调整代码生成策略，发现测试可见性显著影响性能，测试驱动的细化成为主要适应策略。

Generative Retrieval with Few-shot Indexing

基于少样本索引的生成检索

Arian Askari, Chuan Meng, Mohammad Aliannejadi, Zhaochun Ren, Evangelos Kanoulas, Suzan Verberne

机构 * Leiden University（莱顿大学） ； The University of Edinburgh（爱丁堡大学） ； University of Amsterdam（阿姆斯特丹大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 本文提出了一种无需训练的少样本索引生成检索框架，通过提示LLM生成文档标识符来提升检索性能。

Comments Accepted for publication at the 48th European Conference on Information Retrieval (ECIR 2026)

Harnessing Object Grounding for Time-Sensitive Video Understanding

利用物体接地提升时间敏感视频理解

Tz-Ying Wu, Sharath Nittur Sridhar, Subarna Tripathi

机构 * Intel（英特尔公司） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

AI总结 本文提出 GO-Tokenizer 以提升视频大型语言模型的时间敏感视频理解能力，通过实时编码紧凑的物体信息，提高模型性能并减少噪声影响。

Comments Accepted to WACV 2026

PCMind-2.1-Kaiyuan-2B Technical Report

PCMind-2.1-Kaiyuan-2B 技术报告

Kairong Luo, Zhenbo Sun, Xinyu Shi, Shengqi Chen, Bowen Yu, Yunyi Chen, Chenyi Dang, Hengtao Tao, Hui Wang, Fangming Liu, Kaifeng Lyu, Wenguang Chen

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 PCMind-2.1-Kaiyuan-2B通过开源20亿参数模型提升资源受限下的训练效率与效果，采用分位数数据基准、战略选择性重复和多领域课程训练等方法，实现与先进开源模型相当的性能。

Apertus: Democratizing Open and Compliant LLMs for Global Language Environments

Apertus：民主化开放且合规的大型语言模型以适应全球语言环境

Project Apertus, Alejandro Hernández-Cano, Alexander Hägele, Allen Hao Huang, Angelika Romanou, Antoni-Joan Solergibert, Barna Pasztor, Bettina Messmer, Dhia Garbaya, Eduard Frank Ďurech, Ido Hakimi, Juan García Giraldo, Mete Ismayilzada, Negar Foroutan, Skander Moalla, Tiancheng Chen, Vinko Sabolčec, Yixuan Xu, Michael Aerni, Badr AlKhamissi, Inés Altemir Mariñas, Mohammad Hossein Amani, Matin Ansaripour, Ilia Badanin, Harold Benoit, Emanuela Boros, Nicholas Browning, Fabian Bösch, Maximilian Böther, Niklas Canova, Camille Challier, Clement Charmillot, Jonathan Coles, Jan Deriu, Arnout Devos, Lukas Drescher, Daniil Dzenhaliou, Maud Ehrmann, Dongyang Fan, Simin Fan, Silin Gao, Miguel Gila, María Grandury, Diba Hashemi, Alexander Hoyle, Jiaming Jiang, Mark Klein, Andrei Kucharavy, Anastasiia Kucherenko, Frederike Lübeck, Roman Machacek, Theofilos Manitaras, Andreas Marfurt, Kyle Matoba, Simon Matrenok, Henrique Mendonça, Fawzi Roberto Mohamed, Syrielle Montariol, Luca Mouchel, Sven Najem-Meyer, Jingwei Ni, Gennaro Oliva, Matteo Pagliardini, Elia Palme, Andrei Panferov, Léo Paoletti, Marco Passerini, Ivan Pavlov, Auguste Poiroux, Kaustubh Ponkshe, Nathan Ranchin, Javi Rando, Mathieu Sauser, Jakhongir Saydaliev, Muhammad Ali Sayfiddinov, Marian Schneider, Stefano Schuppli, Marco Scialanga, Andrei Semenov, Kumar Shridhar, Raghav Singhal, Anna Sotnikova, Alexander Sternfeld, Ayush Kumar Tarun, Paul Teiletche, Jannis Vamvas, Xiaozhe Yao, Hao Zhao, Alexander Ilic, Ana Klimovic, Andreas Krause, Caglar Gulcehre, David Rosenthal, Elliott Ash, Florian Tramèr, Joost VandeVondele, Livio Veraldi, Martin Rajman, Thomas Schulthess, Torsten Hoefler, Antoine Bosselut, Martin Jaggi, Imanol Schlag

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 Apertus通过开放且合规的大型语言模型，解决数据合规性和多语言表示问题，提供透明的开发资源以促进全球语言环境的应用。

Cluster-based Adaptive Retrieval: Dynamic Context Selection for RAG Applications

Yifan Xu, Vipul Gupta, Rohit Aggarwal, Varsha Mahadevan, Bhaskar Krishnamachari

机构 * University of Southern California（美国南加州大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

PixCLIP: Achieving Fine-grained Visual Language Understanding via Any-granularity Pixel-Text Alignment Learning

Yicheng Xiao, Yu Chen, Haoxuan Ma, Jiale Hong, Caorui Li, Lingxiang Wu, Haiyun Guo, Jinqiao Wang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Languages are Modalities: Cross-Lingual Alignment via Encoder Injection

Rajan Agarwal, Aarush Gupta

机构 * University of Waterloo（多伦多大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 14 pages, 3 Figures

MisSynth: Improving MISSCI Logical Fallacies Classification with Synthetic Data

Mykhailo Poliakov, Nadiya Shvai

机构 * National University of Kyiv-Mohyla Academy（基辅-莫希拉学院国家大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation

Feng Xiong, Hongling Xu, Yifei Wang, Runxi Cheng, Yong Wang, Xiangxiang Chu

机构 * Alibaba Group（阿里巴巴集团） ； University of Chinese Academy of Sciences（中国科学院大学） ； Tsinghua University（清华大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Reinforcement Learning on Pre-Training Data

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Evander Yang, Kun Li, Haoyuan Wu, Jiajia Wu, Zihao Zheng, Chenchen Zhang, Kun Shi, Kyrierl Deng, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang Xu, Zheng Fang, Shuai Li, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bo Chao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Wayyt Wang, Bo Zhou, Di Wang

机构 * LLM Department, Tencent（腾讯大模型部门） ； HunYuan Infra Team（文心一言基础设施团队） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Work in progress

Understanding Emergent In-Context Learning from a Kernel Regression Perspective

Chi Han, Ziqi Wang, Han Zhao, Heng Ji

机构 * Siebel School of Computing and Data Science（计算与数据科学学院） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Transactions on Machine Learning Research (TMLR 2025)

SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs

Yuanyang Yin, Yaqi Zhao, Yajie Zhang, Yuanxing Zhang, Ke Lin, Jiahao Wang, Xin Tao, Pengfei Wan, Wentao Zhang, Feng Zhao

机构 * University of Science and Technology of China（中国科学技术大学） ； Peking University（北京大学） ； Kuaishou Technology（快手科技） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

DriveQA: Passing the Driving Knowledge Test

Maolin Wei, Wanzhou Liu, Eshed Ohn-Bar

机构 * Boston University（波士顿大学） ； Washington University in St. Louis（华盛顿大学圣路易斯分校） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Accepted by ICCV 2025. Project page: https://driveqaiccv.github.io/

Small LLMs Do Not Learn a Generalizable Theory of Mind via Reinforcement Learning

Sneheel Sarangi, Hanan Salam

机构 * NYU Abu Dhabi（纽约大学阿布扎比分校） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding

Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, Radha Poovendran

机构 * Microsoft GenAI（微软生成人工智能） ； University of Washington（华盛顿大学） ； The University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by ACL 2025. Codes and Data: https://kodcode-ai.github.io/

Lightweight Safety Guardrails via Synthetic Data and RL-guided Adversarial Training

Aleksei Ilin, Gor Matevosyan, Xueying Ma, Vladimir Eremin, Suhaa Dada, Muqun Li, Riyaaz Shaik, Haluk Noyan Tokgozoglu

机构 * Apple（苹果公司） 纠错

专题命中 预训练与数据 ：LLM(abstract)；language model(abstract)；small language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Language-Unlocked ViT (LUViT): Empowering Self-Supervised Vision Transformers with LLMs

Selim Kuzucu, Muhammad Ferjad Naeem, Anna Kukleva, Federico Tombari, Bernt Schiele

机构 * Max Planck Institute for Informatics（马克斯·普朗克信息研究所） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 26 pages, 6 figures

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size

Soufiane Hayou, Liyuan Liu

机构 * Simons Institute UC Berkeley（Simons研究所加州大学伯克利分校） ； Microsoft Research（微软研究院） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments TD,LR: How to set the learning rate for emebdding layer in LLMs?

RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answering

Soroosh Tayebi Arasteh, Mahshad Lotfinia, Keno Bressem, Robert Siepmann, Lisa Adams, Dyke Ferber, Christiane Kuhl, Jakob Nikolas Kather, Sven Nebelung, Daniel Truhn

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in Radiology: Artificial Intelligence

Journal ref Radiology: Artificial Intelligence, (2025), 7(4):e240476

Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective

Daniel Franzen, Jan Disselhoff, David Hartmann

机构 * Johannes Gutenberg University Mainz（莱茵河畔明斯特大学） ； Lambda, Inc.（Lambda公司） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2025 camera-ready; 15 pages, 6 figures, 5 tables

Focus On This, Not That! Steering LLMs with Adaptive Feature Specification

Tom A. Lamb, Adam Davies, Alasdair Paren, Philip H. S. Torr, Francesco Pinto

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 36pages, 19 figures

LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback

Thai Hoang, Kung-Hsiang Huang, Shirley Kokane, Jianguo Zhang, Zuxin Liu, Ming Zhu, Jake Grigsby, Tian Lan, Michael S Ryoo, Chien-Sheng Wu, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong, Juan Carlos Niebles

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments LAM Simulator framework for agentic data generation

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective

Seungwook Han, Jinyeop Song, Jeff Gore, Pulkit Agrawal

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments https://charming-centaur-089.notion.site/Emergence-and-Effectiveness-of-Task-Vectors-in-In-Context-Learning-An-Encoder-Decoder-Perspective-2054664a1d59814f8401cded3332fce4

Agentic Knowledgeable Self-awareness

Shuofei Qiao, Zhisong Qiu, Baochang Ren, Xiaobin Wang, Xiangyuan Ru, Ningyu Zhang, Xiang Chen, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2025

Each Graph is a New Language: Graph Learning with LLMs

Huachi Zhou, Jiahe Du, Chuang Zhou, Chang Yang, Yilin Xiao, Yuxuan Xie, Xiao Huang

机构 * First Author Affiliation（第一作者机构） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

ProgRM: Build Better GUI Agents with Progress Rewards

Danyang Zhang, Situo Zhang, Ziyue Yang, Zichen Zhu, Zihan Zhao, Ruisheng Cao, Lu Chen, Kai Yu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Data Doping or True Intelligence? Evaluating the Transferability of Injected Knowledge in LLMs

Essa Jan, Moiz Ali, Muhammad Saram Hassan, Fareed Zaffar, Yasir Zaki

机构 * Lahore University of Management Sciences（拉瓦尔大学管理科学学院） ； New York University Abu Dhabi（纽约大学阿布扎克分校） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 4 pages, 1 figure