语言大模型 / LLM

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference

Wenjie Qiu, Yi-Chen Li, Xuqin Zhang, Tianyi Zhang, Yihang Zhang, Zongzhang Zhang, Yang Yu

专题命中 后训练与偏好优化 ：LLM(title)；language model(abstract)；分类 cs.CL、cs.LG

Policy Filtration for RLHF to Mitigate Noise in Reward Models

Chuheng Zhang, Wei Shen, Li Zhao, Xuyun Zhang, Xiaolong Xu, Wanchun Dou, Jiang Bian

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.AI、cs.LG

Comments ICML2025

BadReward: Clean-Label Poisoning of Reward Models in Text-to-Image RLHF

Kaiwen Duan, Hongwei Yao, Yufei Chen, Ziyun Li, Tong Qiao, Zhan Qin, Cong Wang

机构 * City University of Hong Kong（香港城市大学） ； Zhejiang University（浙江大学） ； KTH Royal Institute of Technology（皇家理工学院） ； Hangzhou Dianzi University（杭州电子科技大学） 纠错

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.AI、cs.LG

Improving Protein Sequence Design through Designability Preference Optimization

Fanglei Xue, Andrew Kubaney, Zhichun Guo, Joseph K. Min, Ge Liu, Yi Yang, David Baker

机构 * University of Technology Sydney（技术科技大学） ； University of Washington（华盛顿大学） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Zhejiang University（浙江大学） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI、cs.LG

Can RLHF be More Efficient with Imperfect Reward Models? A Policy Coverage Perspective

Jiawei Huang, Bingcong Li, Christoph Dann, Niao He

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.AI、cs.LG

Comments 36 Pages; ICML 2025

Boost Your Human Image Generation Model via Direct Preference Optimization

Sanghyeon Na, Yonggyu Kim, Hyunjoon Lee

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted to CVPR 2025 as a highlight paper

VARP: Reinforcement Learning from Vision-Language Model Feedback with Agent Regularized Preferences

Anukriti Singh, Amisha Bhaskar, Peihong Yu, Souradip Chakraborty, Ruthwik Dasyam, Amrit Bedi, Pratap Tokekar

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments 8 pages

ROCM: RLHF on consistency models

Shivanshu Shekhar, Tong Zhang

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.AI、cs.LG

Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks

Rylan Schaeffer, Punit Singh Koura, Binh Tang, Ranjan Subramanian, Aaditya K Singh, Todor Mihaylov, Prajjwal Bhargava, Lovish Madaan, Niladri S. Chatterji, Vedanuj Goswami, Sergey Edunov, Dieuwke Hupkes, Sanmi Koyejo, Sharan Narang

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.CL、cs.LG

FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHF

Flint Xiaofeng Fan, Cheston Tan, Yew-Soon Ong, Roger Wattenhofer, Wei-Tsang Ooi

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.AI、cs.LG

Comments Updated for AAMAS 2025 camera-ready. This preprint represents the full version of the paper, including all proofs, experimental details, and additional discussions

DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization

Zhenglin Zhou, Xiaobo Xia, Fan Ma, Hehe Fan, Yi Yang, Tat-Seng Chua

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL、cs.LG

Comments 20 pages, 12 figures

How good is my story? Towards quantitative metrics for evaluating LLM-generated XAI narratives

Timour Ichmoukhamedov, James Hinns, David Martens

专题命中 后训练与偏好优化 ：LLM(title,abstract)；分类 cs.CL、cs.AI

Radiology Report Generation via Multi-objective Preference Optimization

Ting Xiao, Lei Shi, Peng Liu, Zhe Wang, Chenjia Bai

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by AAAI 2025

On the Relationship between Truth and Political Bias in Language Models

Suyash Fulay, William Brannon, Shrestha Mohanty, Cassandra Overney, Elinor Poole-Dayan, Deb Roy, Jad Kabbara

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024

Journal ref Proc. EMNLP (2024) 9004-9018

Reinforcement Learning for Sequence Design Leveraging Protein Language Models

Jithendaraa Subramanian, Shivakanth Sujit, Niloy Irtisam, Umong Sain, Riashat Islam, Derek Nowrouzezahrai, Samira Ebrahimi Kahou

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments 22 pages, 7 figures, 4 tables

A Theoretical Framework for Partially Observed Reward-States in RLHF

Chinmaya Kausik, Mirco Mutti, Aldo Pacchiano, Ambuj Tewari

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.AI、cs.LG

Comments 64 pages. 14 pages for main paper, 50 pages for references + appendix

Accelerating Direct Preference Optimization with Prefix Sharing

Franklin Wang, Sumanth Hegde

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL、cs.LG

Comments To appear in NeurIPS 2024 in the Fine-Tuning in Machine Learning Workshop

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment

Yiju Guo, Ganqu Cui, Lifan Yuan, Ning Ding, Zexu Sun, Bowen Sun, Huimin Chen, Ruobing Xie, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 main conference

Pixel-Space Post-Training of Latent Diffusion Models

Christina Zhang, Simran Motwani, Matthew Yu, Ji Hou, Felix Juefei-Xu, Sam Tsai, Peter Vajda, Zijian He, Jialiang Wang

专题命中 后训练与偏好优化 ：post-training(title,abstract)；分类 cs.AI、cs.LG

Clinical Reading Comprehension with Encoder-Decoder Models Enhanced by Direct Preference Optimization

Md Sultan Al Nahian, Ramakanth Kavuluru

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL、cs.LG

Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization

Navonil Majumder, Chia-Yu Hung, Deepanway Ghosal, Wei-Ning Hsu, Rada Mihalcea, Soujanya Poria

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at ACM MM 2024

WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences

Yujie Lu, Dongfu Jiang, Wenhu Chen, William Yang Wang, Yejin Choi, Bill Yuchen Lin

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments link: https://hf.co/spaces/WildVision/vision-arena

Are PPO-ed Language Models Hackable?

Suraj Anand, David Getzen

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 4 figures

Learning to Watermark LLM-generated Text via Reinforcement Learning

Xiaojun Xu, Yuanshun Yao, Yang Liu

专题命中 后训练与偏好优化 ：LLM(title,abstract)；分类 cs.AI、cs.LG

RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignment

Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, Yuandong Tian

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments ICLR 2024

Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning

Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, David Lindner

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments Presented at International Conference on Learning Representations (ICLR) 2024

Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solution

Nuo Xu, Jun Zhao, Can Zu, Sixian Li, Lu Chen, Zhihao Zhang, Rui Zheng, Shihan Dou, Wenjuan Qin, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.CL、cs.LG

Can language agents be alternatives to PPO? A Preliminary Empirical Study On OpenAI Gym

Junjie Sheng, Zixiao Huang, Chuyun Shen, Wenhao Li, Yun Hua, Bo Jin, Hongyuan Zha, Xiangfeng Wang

专题命中 后训练与偏好优化 ：language agent(title,abstract)；分类 cs.CL、cs.AI

Is RLHF More Difficult than Standard RL?

Yuanhao Wang, Qinghua Liu, Chi Jin

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted for presentation at NeurIPS 2023; 29 pages

The Trickle-down Impact of Reward (In-)consistency on RLHF

Lingfeng Shen, Sihao Chen, Linfeng Song, Lifeng Jin, Baolin Peng, Haitao Mi, Daniel Khashabi, Dong Yu

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.CL、cs.LG