语言大模型 / LLM

Online Rubrics Elicitation from Pairwise Comparisons

MohammadHossein Rezaei, Robert Vacareanu, Zihao Wang, Clinton Wang, Bing Liu, Yunzhong He, Afra Feyza Akyürek

专题命中 后训练与偏好优化 ：LLM(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation

Ziniu Li, Congliang Chen, Tianyun Yang, Tian Ding, Ruoyu Sun, Ge Zhang, Wenhao Huang, Zhi-Quan Luo

机构 * The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Shenzhen Research Institute of Big Data（深圳大数据研究院） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

From Faithfulness to Correctness: Generative Reward Models that Think Critically

Qiyao Ma, Yunsheng Shi, Hongtao Tian, Chao Wang, Weiming Chang, Ting Yao

机构 * University of California, Davis（加州大学戴维斯分校） ； WeChat, Tencent（微信、腾讯） ； Tsinghua University（清华大学） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Causally-Enhanced Reinforcement Policy Optimization

Xiangqi Wang, Yue Huang, Yujun Zhou, Xiaonan Luo, Kehan Guo, Xiangliang Zhang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Reinforcement learning publication of 24 pages

HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation

Ling Yang, Xinchen Zhang, Ye Tian, Chenming Shang, Minghao Xu, Wentao Zhang, Bin Cui

机构 * Peking University（北京大学） ； Tsinghua University（清华大学） ； Mila - Québec AI Institute（魁北克人工智能研究所） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments NeurIPS 2025. Code: https://github.com/Gen-Verse/HermesFlow

BaseReward: A Strong Baseline for Multimodal Reward Model

Yi-Fan Zhang, Haihua Yang, Huanyu Zhang, Yang Shi, Zezhou Chen, Haochen Tian, Chaoyou Fu, Haotian Wang, Kai Wu, Bo Cui, Xu Wang, Jianfei Pan, Haotian Wang, Zhang Zhang, Liang Wang

机构 * ByteDance（字节跳动） ； CASIA（中国科学院自动化研究所） ； NJU（南京大学） ； PKU（北京大学） ； THU（清华大学） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)

Posterior-GRPO: Rewarding Reasoning Processes in Code Generation

Lishui Fan, Yu Zhang, Mouxiang Chen, Zhongxin Liu

机构 * College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室） ； Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

ArGen: Auto-Regulation of Generative AI via GRPO and Policy-as-Code

Kapil Madan

机构 * Principled Evolution（原则进化） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 53 pages, 7 figures, 8 tables. Open-source implementation available at: https://github.com/Principled-Evolution/argen-demo. Work explores the integration of policy-as-code for AI alignment, with a case study in culturally-nuanced, ethical AI using Dharmic principles

DCPO: Dynamic Clipping Policy Optimization

Shihui Yang, Chengfeng Dou, Peidong Guo, Kai Lu, Qiang Ju, Fei Deng, Rihui Xin

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Kwai Keye-VL 1.5 Technical Report

Biao Yang, Bin Wen, Boyang Ding, Changyi Liu, Chenglong Chu, Chengru Song, Chongling Rao, Chuan Yi, Da Li, Dunju Zang, Fan Yang, Guorui Zhou, Guowang Zhang, Han Shen, Hao Peng, Haojie Ding, Hao Wang, Haonan Fan, Hengrui Ju, Jiaming Huang, Jiangxia Cao, Jiankang Chen, Jingyun Hua, Kaibing Chen, Kaiyu Jiang, Kaiyu Tang, Kun Gai, Muhao Wei, Qiang Wang, Ruitao Wang, Sen Na, Shengnan Zhang, Siyang Mao, Sui Huang, Tianke Zhang, Tingting Gao, Wei Chen, Wei Yuan, Xiangyu Wu, Xiao Hu, Xingyu Lu, Yi-Fan Zhang, Yiping Yang, Yulong Chen, Zeyi Lu, Zhenhua Wu, Zhixin Ling, Zhuoran Yang, Ziming Li, Di Xu, Haixuan Gao, Hang Li, Jing Wang, Lejian Ren, Qigen Hu, Qianqian Wang, Shiyao Wang, Xinchen Luo, Yan Li, Yuhang Hu, Zixing Zhang

机构 * Keye Team, Kuaishou Group（快手集团Keye团队） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；post-training(abstract)

Comments Github page: https://github.com/Kwai-Keye/Keye

Agentic AI Empowered Multi-UAV Trajectory Optimization in Low-Altitude Economy Networks

Feibo Jiang, Li Dong, Xitao Pan, Kezhi Wang, Cunhua Pan

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Ovis2.5 Technical Report

Shiyin Lu, Yang Li, Yu Xia, Yuwei Hu, Shanshan Zhao, Yanqing Ma, Zhichao Wei, Yinglun Li, Lunhao Duan, Jianshan Zhao, Yuxuan Han, Haijun Li, Wanying Chen, Junke Tang, Chengkun Hou, Zhixing Du, Tianli Zhou, Wenjie Zhang, Huping Ding, Jiahe Li, Wen Li, Gui Hu, Yiliang Gu, Siran Yang, Jiamang Wang, Hailong Sun, Yibo Wang, Hui Sun, Jinlong Huang, Yuping He, Shengze Shi, Weihong Zhang, Guodong Zheng, Junpeng Jiang, Sensen Gao, Yi-Feng Wu, Sijia Chen, Yuhui Chen, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang

机构 * Alibaba Group（阿里巴巴集团） 纠错

专题命中 后训练与偏好优化 ：instruction tuning(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Controlling Multimodal LLMs via Reward-guided Decoding

Oscar Mañas, Pierluca D'Oro, Koustuv Sinha, Adriana Romero-Soriano, Michal Drozdzal, Aishwarya Agrawal

机构 * Mila - Quebec AI Institute（魁北克AI研究院） ； Université de Montréal（蒙特利尔大学） ； McGill University（麦吉尔大学） ； Meta FAIR ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published at ICCV 2025

MiGrATe: Mixed-Policy GRPO for Adaptation at Test-Time

Peter Phan, Dhruv Agarwal, Kavitha Srinivas, Horst Samulowitz, Pavan Kapanipathi, Andrew McCallum

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Eyes Will Shut: A Vision-Based Next GPS Location Prediction Model by Reinforcement Learning from Visual Map Feed Back

Ruixing Zhang, Yang Zhang, Tongyu Zhu, Leilei Sun, Weifeng Lv

专题命中 后训练与偏好优化 ：LLM(abstract)；language model(abstract)；SFT(abstract)

InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling

Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, Min Dou, Kai Chen, Wenhai Wang, Yu Qiao, Yali Wang, Limin Wang

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； Nanjing University（南京大学） ； Shanghai Innovation Institute（上海创新研究院） ； Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments technical report

Seedream 3.0 Technical Report

Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, Wei Liu, Yichun Shi, Shiqi Sun, Yu Tian, Zhi Tian, Peng Wang, Rui Wang, Xuanda Wang, Xun Wang, Ye Wang, Guofeng Wu, Jie Wu, Xin Xia, Xuefeng Xiao, Zhonghua Zhai, Xinyu Zhang, Qi Zhang, Yuwei Zhang, Shijia Zhao, Jianchao Yang, Weilin Huang

机构 * ByteDance（字节跳动） 纠错

专题命中 后训练与偏好优化 ：foundation model(abstract)；post-training(abstract)；SFT(abstract)

Comments Seedream 3.0 Technical Report

Reinforcement Learning from Human Feedback with High-Confidence Safety Constraints

Yaswanth Chittepu, Blossom Metevier, Will Schwarzer, Austin Hoag, Scott Niekum, Philip S. Thomas

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校） 纠错

专题命中 后训练与偏好优化 ：language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 20 pages, 6 figures, 4 tables, Second Reinforcement Learning Conference (RLC 2025)

ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding

Yiyang Zhou, Yangfan He, Yaofeng Su, Siwei Han, Joel Jang, Gedas Bertasius, Mohit Bansal, Huaxiu Yao

专题命中 后训练与偏好优化 ：language model(abstract)；SFT(abstract)；preference optimization(abstract)

Comments 31 pages, 18 figures

Do Not Let Low-Probability Tokens Over-Dominate in RL for LLMs

Zhihe Yang, Xufang Luo, Zilong Wang, Dongqi Han, Zhiyuan He, Dongsheng Li, Yunjian Xu

机构 * The Chinese University of Hong Kong（香港中文大学） ； Microsoft Research Asia（微软亚洲研究院） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 24 pages, 12 figures

Principled Data Selection for Alignment: The Hidden Risks of Difficult Examples

Chengqian Gao, Haonan Li, Liu Liu, Zeke Xie, Peilin Zhao, Zhiqiang Xu

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ICML 2025

Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception

Yuanchen Wu, Lu Zhang, Hang Yao, Junlong Du, Ke Yan, Shouhong Ding, Yunsheng Wu, Xiaoqiang Li

机构 * School of Computer Engineering & Science, Shanghai University（上海大学计算机工程与科学学院） ； Tencent YouTu Lab（腾讯优图实验室） 纠错

专题命中 后训练与偏好优化 ：language model(abstract)；post-training(abstract)；preference optimization(abstract)

Comments Accepted to CVPR 2025

Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots

Erfan Shayegani, G M Shahariar, Sara Abdali, Lei Yu, Nael Abu-Ghazaleh, Yue Dong

专题命中 后训练与偏好优化 ：language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Optimizing Retrieval Strategies for Financial Question Answering Documents in Retrieval-Augmented Generation Systems

Sejong Kim, Hyunseo Song, Hyunwoo Seo, Hyunjun Kim

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments 15 pages, 3 figures, 11 tables. Accepted at ICLR 2025 Workshop on Advances in Financial AI. Code available at https://github.com/seohyunwoo-0407/GAR

Efficient Safety Retrofitting Against Jailbreaking for LLMs

Dario Garcia-Gasulla, Adrian Tormos, Anna Arias-Duart, Daniel Hinjos, Oscar Molina-Sedano, Ashwin Kumar Gururajan, Maria Eugenia Cardello

专题命中 后训练与偏好优化 ：LLM(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

Adaptive Segment-level Reward: Bridging the Gap Between Action and Reward Space in Alignment

Yanshi Li, Shaopan Xiong, Gengru Chen, Xiaoyang Li, Yijia Luo, Xingyuan Bu, Yingshui Tan, Wenbo Su, Bo Zheng

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Reward Models Identify Consistency, Not Causality

Yuhui Xu, Hanze Dong, Lei Wang, Caiming Xiong, Junnan Li

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages

DAMA: Data- and Model-aware Alignment of Multi-modal LLMs

Jinda Lu, Junkang Wu, Jinghan Li, Xiaojun Jia, Shuo Wang, YiFan Zhang, Junfeng Fang, Xiang Wang, Xiangnan He

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)

Inferring from Logits: Exploring Best Practices for Decoding-Free Generative Candidate Selection

Mingyu Derek Ma, Yanna Ding, Zijie Huang, Jianxi Gao, Yizhou Sun, Wei Wang

专题命中 后训练与偏好优化 ：language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

"What's Happening"- A Human-centered Multimodal Interpreter Explaining the Actions of Autonomous Vehicles

Xuewen Luo, Fan Ding, Ruiqi Chen, Rishikesh Panda, Junnyong Loo, Shuyun Zhang

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments This paper has been accepted for presentation at WACV Workshop HAVI 2025