语言大模型 / LLM

DeepSeek: Paradigm Shifts and Technical Evolution in Large AI Models

Luolin Xiong, Haofen Wang, Xi Chen, Lu Sheng, Yun Xiong, Jingping Liu, Yanghua Xiao, Huajun Chen, Qing-Long Han, Yang Tang

机构 * Key Laboratory of Smart Manufacturing in Energy Chemical Process, East China University of Science and Technology（东华大学能源化学过程智能制造重点实验室） ； School of Information Science and Engineering, East China University of Science and Technology（东华大学信息科学与工程学院） ； Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所认知与复杂系统决策智能实验室） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Research Institute of Intelligent Complex Systems, Fudan University（复旦大学智能复杂系统研究院） ； School of Science, Computing and Engineering Technologies, Swinburne University of Technology（斯威本科技大学科学、计算与工程技术学院） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Self-Improving Model Steering

Rongyi Zhu, Yuhui Wang, Tanqiu Jiang, Jiacheng Liang, Ting Wang

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 16 pages, 9 figures

Skywork-R1V3 Technical Report

Wei Shen, Jiangbo Pei, Yi Peng, Xuchen Song, Yang Liu, Jian Peng, Haofeng Sun, Yunzhuo Hao, Peiyu Wang, Jianhao Zhang, Yahui Zhou

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL

First Return, Entropy-Eliciting Explore

Tianyu Zheng, Tianshun Xing, Qingshui Gu, Taoran Liang, Xingwei Qu, Xin Zhou, Yizhi Li, Zhoufutu Wen, Chenghua Lin, Wenhao Huang, Qian Liu, Ge Zhang, Zejun Ma

机构 * The University of Manchester（曼彻斯特大学） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

A Comprehensive Graph Framework for Question Answering with Mode-Seeking Preference Alignment

Quanwei Tang, Sophia Yat Mei Lee, Junshuang Wu, Dong Zhang, Shoushan Li, Erik Cambria, Guodong Zhou

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments acl 2025 findings

Robust Reward Modeling via Causal Rubrics

Pragya Srivastava, Harman Singh, Rahul Madhavan, Gandharv Patil, Sravanti Addepalli, Arun Suggala, Rengarajan Aravamudhan, Soumya Sharma, Anirban Laha, Aravindan Raghuveer, Karthikeyan Shanmugam, Doina Precup

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

SAGE:Specification-Aware Grammar Extraction for Automated Test Case Generation with LLMs

Aditi, Hyunwoo Park, Sicheol Sung, Yo-Sub Han, Sang-Ki Ko

机构 * University of Seoul, South Korea（首尔大学，韩国） ； Yonsei University, South Korea（延世大学，韩国） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Reinforcing Code Generation: Improving Text-to-SQL with Execution-Based Learning

Atharv Kulkarni, Vivek Srikumar

机构 * University of Utah（犹他大学） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Under review at EMNLP 2025

TO-GATE: Clarifying Questions and Summarizing Responses with Trajectory Optimization for Eliciting Human Preference

Yulin Dou, Jiangming Liu

机构 * School of Information Science and Engineering, Yunnan University, China（云南大学信息科学与工程学院） ； Yunnan Key Laboratory of Intelligent Systems and Computing, China（云南省智能系统与计算重点实验室） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

RewardBench 2: Advancing Reward Model Evaluation

Saumya Malik, Valentina Pyatkin, Sander Land, Jacob Morrison, Noah A. Smith, Hannaneh Hajishirzi, Nathan Lambert

机构 * Allen Institute for Artificial Intelligence（艾伦人工智能研究所） ； University of Washington（华盛顿大学） ； Cohere 纠错

专题命中 后训练与偏好优化 ：language model(abstract)；post-training(abstract)；RLHF(abstract)；分类 cs.CL

Comments Data, models, and leaderboard available at https://huggingface.co/collections/allenai/reward-bench-2-683d2612a4b3e38a3e53bb51

Curiosity-Driven Reinforcement Learning from Human Feedback

Haoran Sun, Yekun Chai, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang

机构 * Baidu Inc.（百度公司） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL

Comments ACL 2025

Speculative Reward Model Boosts Decision Making Ability of LLMs Cost-Effectively

Jiawei Gu, Shangsong Liang

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ACL2025 Oral (Industry Track)

MPO: Multilingual Safety Alignment via Reward Gap Optimization

Weixiang Zhao, Yulin Hu, Yang Deng, Tongtong Wu, Wenxuan Zhang, Jiahe Guo, An Zhang, Yanyan Zhao, Bing Qin, Tat-Seng Chua, Ting Liu

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL

Comments To Appear at ACL 2025 (Main)

From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment

Jing Ye, Lu Xiang, Yaping Zhang, Chengqing Zong

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments 27 pages

SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimization

Minghan Chen, Guikun Chen, Wenguan Wang, Yi Yang

机构 * ReLER Lab, CCAI, Zhejiang University（ReLER实验室、中国人工智能学会、浙江大学） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments On going project

Probabilistic Uncertain Reward Model

Wangtao Sun, Xiang Cheng, Xing Yu, Haotian Xu, Zhao Yang, Shizhu He, Jun Zhao, Kang Liu

机构 * The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China（认知与复杂系统决策智能实验室，自动化研究所，中国科学院，北京，中国） ； School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China（人工智能学院，中国科学院大学，北京，中国） ； Xiaohongshu Inc（小红书公司） ； Meituan Inc（美团公司） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.LG

Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approach

Shannon Lodoen, Alexi Orchard

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI

Comments 10 pages, 1 figure, Accepted version

Drift: Decoding-time Personalized Alignments with Implicit User Preferences

Minbeom Kim, Kang-il Lee, Seongho Joo, Hwaran Lee, Thibaut Thonet, Kyomin Jung

机构 * Seoul National University（首尔国立大学） ； Sogang University（ソガン大学） ； NAVER AI Lab（NAVER AI实验室） ； NAVER Labs Europe（NAVER欧洲实验室） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL

Comments 19 pages, 6 figures

(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts

Minghao Wu, Jiahao Xu, Yulin Yuan, Gholamreza Haffari, Longyue Wang, Weihua Luo, Kaifu Zhang

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments To appear at TACL

Governance Challenges in Reinforcement Learning from Human Feedback: Evaluator Rationality and Reinforcement Stability

Dana Alsagheer, Abdulrahman Kamal, Mohammad Kamal, Weidong Shi

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.AI

ADAPT: Actively Discovering and Adapting to Preferences for any Task

Maithili Patel, Xavier Puig, Ruta Desai, Roozbeh Mottaghi, Sonia Chernova, Joanne Truong, Akshara Rai

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

R-PRM: Reasoning-Driven Process Reward Modeling

Shuaijie She, Junxiao Liu, Yifeng Liu, Jiajun Chen, Xin Huang, Shujian Huang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments The project is available at https://github.com/NJUNLP/R-PRM

GAPO: Learning Preferential Prompt through Generative Adversarial Policy Optimization

Zhouhong Gu, Xingzhou Chen, Xiaoran Shi, Tao Wang, Suhang Zheng, Tianyu Li, Hongwei Feng, Yanghua Xiao

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learning

Ke Ji, Yixin Lian, Linxu Li, Jingsheng Gao, Weiyuan Li, Bin Dai

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL

Comments 18 pages, 4 figures

CE-U: Cross Entropy Unlearning

Bo Yang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.LG

The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedback

Ruitao Chen, Liwei Wang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.LG

X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale

Haoran Xu, Kenton Murray, Philipp Koehn, Hieu Hoang, Akiko Eriguchi, Huda Khayrallah

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Published as a conference paper at ICLR 2025 (spotlight)

Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicity

Rheeya Uppaal, Apratim Dey, Yiting He, Yiqiao Zhong, Junjie Hu

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted to ICLR 2025

YINYANG-ALIGN: Benchmarking Contradictory Objectives and Proposing Multi-Objective Optimization based DPO for Text-to-Image Alignment

Amitava Das, Yaswanth Narsupalli, Gurpreet Singh, Vinija Jain, Vasu Sharma, Suranjana Trivedy, Aman Chadha, Amit Sheth

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；preference optimization(abstract)；分类 cs.AI

PILAF: Optimal Human Preference Sampling for Reward Modeling

Yunzhen Feng, Ariel Kwiatkowski, Kunhao Zheng, Julia Kempe, Yaqi Duan

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.LG