语言大模型 / LLM

Self-Rewarding Language Models

Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, Jason Weston

专题命中 后训练与偏好优化 ：language model(title,abstract)；LLM(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

Comments ICML 2024

A Shared Low-Rank Adaptation Approach to Personalized RLHF

Renpu Liu, Peng Wang, Donghao Li, Cong Shen, Jing Yang

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at AISTATS 2025

Mitigating Reward Over-Optimization in RLHF via Behavior-Supported Regularization

Juntao Dai, Taiye Chen, Yaodong Yang, Qian Zheng, Gang Pan

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at ICLR 2025

Human Preferences for Constructive Interactions in Language Model Alignment

Yara Kyrychenko, Jon Roozenbeek, Brandon Davidson, Sander van der Linden, Ramit Debnath

专题命中 后训练与偏好优化 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 1 Figure, 1 Table, 11 pages

Robust Preference Optimization through Reward Model Distillation

Adam Fisch, Jacob Eisenstein, Vicky Zayats, Alekh Agarwal, Ahmad Beirami, Chirag Nagpal, Pete Shaw, Jonathan Berant

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL、cs.LG

Evaluating Defences against Unsafe Feedback in RLHF

Domenic Rosati, Giles Edkins, Harsh Raj, David Atanasov, Subhabrata Majumdar, Janarthanan Rajendran, Frank Rudzicz, Hassan Sajjad

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation

Shuo Tang, Xianghe Pang, Zexi Liu, Bohan Tang, Rui Ye, Tian Jin, Xiaowen Dong, Yanfeng Wang, Siheng Chen

专题命中 后训练与偏好优化 ：post-training(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Ranking Unraveled: Recipes for LLM Rankings in Head-to-Head AI Combat

Roland Daynauth, Christopher Clarke, Krisztian Flautner, Lingjia Tang, Jason Mars

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

$f$-PO: Generalizing Preference Optimization with $f$-divergence Minimization

Jiaqi Han, Mingjian Jiang, Yuxuan Song, Stefano Ermon, Minkai Xu

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；LLM(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Journal ref AISTATS 2025

Language Models Largely Exhibit Human-like Constituent Ordering Preferences

Ada Defne Tur, Gaurav Kamath, Siva Reddy

专题命中 后训练与偏好优化 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments NAACL 2025 Main Conference

Improving Factual Consistency of News Summarization by Contrastive Preference Optimization

Huawen Feng, Yan Fan, Xiong Liu, Ting-En Lin, Zekun Yao, Yuchuan Wu, Fei Huang, Yongbin Li, Qianli Ma

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

The Crucial Role of Samplers in Online Direct Preference Optimization

Ruizhe Shi, Runlong Zhou, Simon S. Du

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments ICLR accepted

Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both

Abhijnan Nath, Changsoo Jung, Ethan Seefried, Nikhil Krishnaswamy

专题命中 后训练与偏好优化 ：language model(title)；LLM(abstract)；RLHF(abstract)；preference optimization(abstract)

LiPO: Listwise Preference Optimization through Learning-to-Rank

Tianqi Liu, Zhen Qin, Junru Wu, Jiaming Shen, Misha Khalman, Rishabh Joshi, Yao Zhao, Mohammad Saleh, Simon Baumgartner, Jialu Liu, Peter J. Liu, Xuanhui Wang

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments Accepted at NAACL 2025

More RLHF, More Trust? On The Impact of Preference Alignment On Trustworthiness

Aaron J. Li, Satyapriya Krishna, Himabindu Lakkaraju

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Context-DPO: Aligning Language Models for Context-Faithfulness

Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang, Shenghua Liu

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes

Katarzyna Kobalczyk, Claudio Fanconi, Hao Sun, Mihaela van der Schaar

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Solving the Inverse Alignment Problem for Efficient RLHF

Shambhavi Krishna, Aishwarya Sahoo

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；LLM(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Multi-Response Preference Optimization with Augmented Ranking Dataset

Hansle Gwon, Imjin Ahn, Young-Hak Kim, Sanghyun Park, Tae Joon Jun

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Value Imprint: A Technique for Auditing the Human Values Embedded in RLHF Datasets

Ike Obi, Rohan Pant, Srishti Shekhar Agrawal, Maham Ghazanfar, Aaron Basiletti

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；LLM(abstract)；language model(abstract)；分类 cs.AI、cs.LG

ReMoDetect: Reward Models Recognize Aligned LLM's Generations

Hyunseok Lee, Jihoon Tack, Jinwoo Shin

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Published as a conference proceeding for NeurIPS 2024

Towards Reliable Alignment: Uncertainty-aware RLHF

Debangshu Banerjee, Aditya Gopalan

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Reward Difference Optimization For Sample Reweighting In Offline RLHF

Shiqi Wang, Zhengze Zhang, Rui Zhao, Fei Tan, Cam Tu Nguyen

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 findings

2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision

Shilong Li, Yancheng He, Hui Huang, Xingyuan Bu, Jiaheng Liu, Hangyu Guo, Weixun Wang, Jihao Gu, Wenbo Su, Bo Zheng

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments The first four authors contributed equally, 25 pages

Aligning CodeLLMs with Direct Preference Optimization

Yibo Miao, Bofei Gao, Shanghaoran Quan, Junyang Lin, Daoguang Zan, Jiaheng Liu, Jian Yang, Tianyu Liu, Zhijie Deng

专题命中 后训练与偏好优化 ：preference optimization(title)；large language model(abstract)；language model(abstract)；post-training(abstract)

Boosting Deductive Reasoning with Step Signals In RLHF

Jialian Li, Yipin Zhang, Wei Shen, Yuzi Yan, Jian Xie, Dong Yan

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

$β$-DPO: Direct Preference Optimization with Dynamic $β$

Junkang Wu, Yuexiang Xie, Zhengyi Yang, Jiancan Wu, Jinyang Gao, Bolin Ding, Xiang Wang, Xiangnan He

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2024

Investigating on RLHF methodology

Alexey Kutalev, Sergei Markoff

专题命中 后训练与偏好优化 ：RLHF(title)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments 23 pages, 6 figures, 6 tables

Model-based Preference Optimization in Abstractive Summarization without Human Feedback

Jaepill Choi, Kyubyung Chae, Jiwoo Song, Yohan Jo, Taesup Kim

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2024

Cost-Effective Online Multi-LLM Selection with Versatile Reward Models

Xiangxiang Dai, Jin Li, Xutong Liu, Anqi Yu, John C. S. Lui

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 32 pages, 14 figures, conference