语言大模型 / LLM

Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization

Jingfeng Guo, Jian Liu, Jinnan Chen, Shiwei Mao, Changrong Hu, Puhua Jiang, Junlin Yu, Jing Xu, Qi Liu, Lixin Xu, Zhuo Chen, Chunchao Guo

机构 * South China University of Technology（南方科技大学） ； Hong Kong University of Science and Technology（香港科技大学） ； National University of Singapore（国立新加坡大学） ； Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院） ； University of Science and Technology of China（中国科学技术大学） ； Beijing Normal University（北京师范大学） ； Tencent Hunyuan（腾讯文盈） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；post-training(abstract)

From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models

Mingkang Zhu, Xi Chen, Bei Yu, Hengshuang Zhao, Jiaya Jia

机构 * The Chinese University of Hong Kong（香港中文大学） ； The University of Hong Kong（香港大学） ； The Hong Kong University of Science and Technology（香港科学与技术大学） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization

Tao Zhang, Cheng Da, Kun Ding, Huan Yang, Kun Jin, Yan Li, Tingting Gao, Di Zhang, Shiming Xiang, Chunhong Pan

机构 * MAIS, CASIA（中国科学院自动化研究所MAIS实验室） ； Kuaishou Technology（快手科技） ； School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；language model(abstract)

Comments NeurIPS 2025

GSPR: Aligning LLM Safeguards as Generalizable Safety Policy Reasoners

Haoran Li, Yulin Chen, Jingru Zeng, Hao Peng, Huihao Jing, Wenbin Hu, Xi Yang, Ziqian Zeng, Sirui Han, Yangqiu Song

专题命中 后训练与偏好优化 ：LLM(title)；large language model(abstract)；language model(abstract)

Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination Mitigation

Jihao Gu, Yingyao Wang, Meng Cao, Pi Bu, Jun Song, Yancheng He, Shilong Li, Bo Zheng

机构 * Taobao & Tmall Group of Alibaba（淘宝与天猫集团） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；language model(abstract)

GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data

Shengliang Deng, Mi Yan, Songlin Wei, Haixin Ma, Yuxin Yang, Jiayi Chen, Zhiqi Zhang, Taoyu Yang, Xuheng Zhang, Wenhao Zhang, Heming Cui, Zhizheng Zhang, He Wang

专题命中 后训练与偏好优化 ：foundation model(title,abstract)；post-training(abstract)

Orthogonal Finetuning for Direct Preference Optimization

Chenxu Yang, Ruipeng Jia, Naibin Gu, Zheng Lin, Siyuan Chen, Chao Pang, Weichong Yin, Yu Sun, Hua Wu, Weiping Wang

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation

Chao Liang, Jianwen Jiang, Wang Liao, Jiaqi Yang, Zerong zheng, Weihong Zeng, Han Liang

机构 * ByteDance（字节跳动） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；post-training(abstract)

Comments Homepage: https://alignhuman.github.io/

RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation

Kaiqu Liang, Haimin Hu, Ryan Liu, Thomas L. Griffiths, Jaime Fernández Fisac

机构 * Princeton University（普林斯顿大学） 纠错

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 27 pages, 18 figures

On the Robustness of Reward Models for Language Model Alignment

Jiwoo Hong, Noah Lee, Eunki Kim, Guijin Son, Woojin Chung, Aman Gupta, Shao Tang, James Thorne

专题命中 后训练与偏好优化 ：language model(title)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2025

Atyaephyra at SemEval-2025 Task 4: Low-Rank Negative Preference Optimization

Jan Bronec, Jindřich Helcl

机构 * Charles University, Faculty of Mathematics and Physics（查理大学数学与物理学院） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 8 pages, 3 figures, accepted to SemEval workshop proceedings at ACL 2025

SUDO: Enhancing Text-to-Image Diffusion Models with Self-Supervised Direct Preference Optimization

Liang Peng, Boxi Wu, Haoran Cheng, Yibo Zhao, Xiaofei He

机构 * FABU Inc.（FABU公司） ； Zhejiang University（浙江大学） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；SFT(abstract)

Aligning Multimodal LLM with Human Preference: A Survey

Tao Yu, Yi-Fan Zhang, Chaoyou Fu, Junkang Wu, Jinda Lu, Kun Wang, Xingyu Lu, Yunhang Shen, Guibin Zhang, Dingjie Song, Yibo Yan, Tianlong Xu, Qingsong Wen, Zhang Zhang, Yan Huang, Liang Wang, Tieniu Tan

专题命中 后训练与偏好优化 ：LLM(title)；large language model(abstract)；language model(abstract)

Comments Project page: https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Alignment

PMANet: Malicious URL detection via post-trained language model guided multi-level feature attention network

Ruitong Liu, Yanbin Wang, Haitao Xu, Zhan Qin, Fan Zhang, Yiwei Liu, Zheng Cao

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(abstract)

Comments 18 pages, 8 figures

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key

Zhihe Yang, Xufang Luo, Dongqi Han, Yunjian Xu, Dongsheng Li

专题命中 后训练与偏好优化 ：language model(title,abstract)；preference optimization(abstract)

Comments Accepted by CVPR 2025

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference

Mingqi Gao, Yixin Liu, Xinyu Hu, Xiaojun Wan, Jonathan Bragg, Arman Cohan

专题命中 后训练与偏好优化 ：LLM(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Findings of NAACL 2025

AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation

Jingkun An, Yinghao Zhu, Zongjian Li, Enshen Zhou, Haoran Feng, Xijie Huang, Bohua Chen, Yemin Shi, Chengwei Pan

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；language model(abstract)

Comments Accepted by AAAI-2025

Improving Context-Aware Preference Modeling for Language Models

Silviu Pitis, Ziang Xiao, Nicolas Le Roux, Alessandro Sordoni

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024. 10 pages (29 with references and appendix)

Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimization

Xiaoxue Gao, Chen Zhang, Yiming Chen, Huayun Zhang, Nancy F. Chen

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；LLM(abstract)

Comments 5 pages

CogVLM2: Visual Language Models for Image and Video Understanding

Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, Lei Zhao, Zhuoyi Yang, Xiaotao Gu, Xiaohan Zhang, Guanyu Feng, Da Yin, Zihan Wang, Ji Qi, Xixuan Song, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Yuxiao Dong, Jie Tang

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(abstract)

CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs

Yassine Ouali, Adrian Bulat, Brais Martinez, Georgios Tzimiropoulos

专题命中 后训练与偏好优化 ：language model(title,abstract)；preference optimization(abstract)

Comments Accepted at ECCV 2024

Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization

Yihan Du, Anna Winnicki, Gal Dalal, Shie Mannor, R. Srikant

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.CL、cs.AI、cs.LG

RLHF and IIA: Perverse Incentives

Wanqiao Xu, Shi Dong, Xiuyuan Lu, Grace Lam, Zheng Wen, Benjamin Van Roy

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.CL、cs.AI、cs.LG

DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales

Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, Zhongzhu Zhou, Michael Wyatt, Molly Smith, Lev Kurilenko, Heyang Qin, Masahiro Tanaka, Shuai Che, Shuaiwen Leon Song, Yuxiong He

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 14 pages, 7 figures

Love First, Know Later: Persona-Based Romantic Compatibility Through LLM Text World Engines

先爱后知：基于人设的浪漫兼容性通过LLM文本世界引擎

Haoyang Shang, Zhengyang Yan, Xuan Liu

机构 * BreathingCORE 纠错

专题命中 后训练与偏好优化 ：LLM(title,abstract)；分类 cs.CL、cs.LG

AI总结 本文提出一种基于LLM文本世界引擎的浪漫兼容性匹配方法，通过模拟互动预测人类偏好，实现个性化匹配系统。

Comments NeurIPS 2025 Workshop: First Workshop on LLM Persona Modeling (Oral)

Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback

Yifu Yuan, Jianye Hao, Yi Ma, Zibin Dong, Hebin Liang, Jinyi Liu, Zhixin Feng, Kai Zhao, Yan Zheng

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at ICLR 2024. The website is available at https://uni-rlhf.github.io/

SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning

SAPO：用于智能体强化学习的单回滚自回归策略优化

Dayang Liang, Lang Feng, Bo An, Yunlong Liu

机构 * Xiamen University（厦门大学） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract_cn)；large language model(abstract)；language model(abstract)；post-training(abstract)

AI总结 本研究针对智能体强化学习现有方法的三大局限，提出SAPO框架，其共享策略与价值函数的自回归主干，结合广义优势估计器，在ALFWorld、WebShop任务上性能优于PPO和GRPO，内存与计算效率更高。

Bridging Network Fragmentation: A Semantic-Augmented DRL Framework for UAV-aided VANETs

弥合网络碎片化：一种语义增强的深度强化学习框架用于无人机辅助的VANETs

Gaoxiang Cao, Wenke Yuan, Huasen He, Yunpeng Hou, Xiaofeng Jiang, Shuangwu Chen, Jian Yang

机构 * Department of Automation, University of Science & Technology of China（中国科学技术大学自动化系） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract,abstract_cn)；large language model(abstract)；language model(abstract)；分类 cs.AI

AI总结 本文提出语义增强深度强化学习框架，通过语义推理优化无人机在VANETs中的部署，提升网络连通性与效率。

Comments Revised version. This update includes substantial improvements to the methodology, ablation studies, temporal robustness analysis, and cross-city generalization experiments. Submitted to IEEE Transactions on Cognitive Communications and Networking. 15 pages, 14 figures

Agent Safety Should Be a Runtime Contract

智能体安全应成为运行时契约

Albus W. Ng, Yi Han, Jusheng Zhang, Wenhao Wang

专题命中 后训练与偏好优化 ：LLM(abstract,abstract_cn)；RLHF(abstract,abstract_cn)；分类 cs.AI

AI总结 该研究指出将AI安全仅在训练阶段植入的范式不足，提出智能体安全应是兼具预防与证据层面的运行时契约，通过四类公开证据支撑该立场并给出研究议程。

Mind the Gap: Structure-Aware Consistency in Preference Learning

注意间隙：在偏好学习中的结构感知一致性

Mehryar Mohri, Yutao Zhong

机构 * Google Research（谷歌研究） ； Courant Institute of Mathematical Sciences（数学科学学院） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract,abstract_cn)；large language model(abstract)；language model(abstract)；分类 cs.LG

AI总结 本文提出结构感知H一致性框架，通过引入SA-DPO目标函数，改进偏好学习中的一致性问题，证明重尾 surrogate 在容量受限模型中具有更好的一致性保证。

Comments ICML 2026