语言大模型 / LLM

IterQR: An Iterative Framework for LLM-based Query Rewrite in e-Commercial Search System

Shangyu Chen, Xinyu Jia, Yingfei Zhang, Shuai Zhang, Xiang Li, Wei Lin

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning

Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, Jinqiao Wang

专题命中 后训练与偏好优化 ：language model(title,abstract)；pretraining(abstract)；post-training(abstract)；preference optimization(abstract)

Comments Project in development. Github: https://github.com/jefferyZhan/Griffon/tree/master/Vision-R1

IPO: Your Language Model is Secretly a Preference Classifier

Shivank Garg, Ayush Singh, Shweta Singh, Paras Chopra

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；preference optimization(abstract)

A Survey of Direct Preference Optimization

Shunyu Liu, Wenkai Fang, Zetian Hu, Junjie Zhang, Yang Zhou, Kongcheng Zhang, Rongcheng Tu, Ting-En Lin, Fei Huang, Mingli Song, Yongbin Li, Dacheng Tao

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Aligning to What? Limits to RLHF Based Alignment

Logan Barnhart, Reza Akbarian Bafghi, Stephen Becker, Maziar Raissi

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Language Model Personalization via Reward Factorization

Idan Shenfeld, Felix Faltings, Pulkit Agrawal, Aldo Pacchiano

专题命中 后训练与偏好优化 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；RLHF(abstract)

Active Learning for Direct Preference Optimization

Branislav Kveton, Xintong Li, Julian McAuley, Ryan Rossi, Jingbo Shang, Junda Wu, Tong Yu

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective

Ruichen Shao, Bei Li, Gangao Liu, Yang Chen, Xiang Zhou, Jingang Wang, Xunliang Cai, Peng Li

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments Accepted by ICLR 2025

Triple Preference Optimization: Achieving Better Alignment using a Single Step Optimization

Amir Saeidi, Shivanshu Verma, Aswin RRV, Kashif Rasul, Chitta Baral

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models

Xin Zhou, Yiwen Guo, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中 后训练与偏好优化 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；preference optimization(abstract)

AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic

Nathaniel R. Robinson, Shahd Abdelmoneim, Kelly Marchisio, Sebastian Ruder

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Comments Pre-print

JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs

Hongyi Li, Jiawei Ye, Jie Wu, Tianjie Yan, Chu Wang, Zhixin Li

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by AAAI 2025

Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration

Avinandan Bose, Zhihan Xiong, Aadirupa Saha, Simon Shaolei Du, Maryam Fazel

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence

Junru Lu, Jiazheng Li, Siyu An, Meng Zhao, Yulan He, Di Yin, Xing Sun

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments EMNLP 2024 Main, Final Version

Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models

Somanshu Singla, Zhen Wang, Tianyang Liu, Abdullah Ashfaq, Zhiting Hu, Eric P. Xing

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(abstract)；SFT(abstract)；RLHF(abstract)

Comments EMNLP 2024 Main

PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment

Dongxu Liu, Bing Xu, Yinzhuo Chen, Bufan Xu, Wenpeng Lu, Muyun Yang, Tiejun Zhao

专题命中 后训练与偏好优化 ：LLM(title)；large language model(abstract)；language model(abstract)；post-training(abstract)

Reward Modeling with Weak Supervision for Language Models

Ben Hauptvogel, Malte Ostendorff, Georg Rehm, Sebastian Möller

专题命中 后训练与偏好优化 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；RLHF(abstract)

Learning to Ask Informative Questions: Enhancing LLMs with Preference Optimization and Expected Information Gain

Davide Mazzaccara, Alberto Testoni, Raffaella Bernardi

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted to EMNLP 2024 (Findings)

Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study

Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, Yi Wu

专题命中 后训练与偏好优化 ：LLM(title)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments 16 pages, 2 figures, 14 tables

Journal ref ICML 2024

Enabling Language Models to Implicitly Learn Self-Improvement

Ziqi Wang, Le Hou, Tianjian Lu, Yuexin Wu, Yunxuan Li, Hongkun Yu, Heng Ji

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；prompting(abstract)

Comments Accepted at ICLR 2024. 28 pages, 5 figures, 4 tables

Intermediate direct preference optimization

Atsushi Kojima

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

New Desiderata for Direct Preference Optimization

Xiangkun Hu, Tong He, David Wipf

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Bayesian Reward Models for LLM Alignment

Adam X. Yang, Maxime Robeyns, Thomas Coste, Zhengyan Shi, Jun Wang, Haitham Bou-Ammar, Laurence Aitchison

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback

Timon Ziegenbein, Gabriella Skitalinskaya, Alireza Bayat Makou, Henning Wachsmuth

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Compass: Large Multilingual Language Model for South-east Asia

Sophia Maria

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(abstract)；SFT(abstract)；preference optimization(abstract)

Investigating Regularization of Self-Play Language Models

Reda Alami, Abdalgader Abubaker, Mastane Achab, Mohamed El Amine Seddik, Salem Lahlou

专题命中 后训练与偏好优化 ：language model(title,abstract)；LLM(abstract)；RLHF(abstract)；preference optimization(abstract)

RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, Tat-Seng Chua

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments Accepted by CVPR 2024

Balancing Enhancement, Harmlessness, and General Capabilities: Enhancing Conversational LLMs with Direct RLHF

Chen Zheng, Ke Sun, Hang Wu, Chenguang Xi, Xun Zhou

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Value Augmented Sampling for Language Model Alignment and Personalization

Seungwook Han, Idan Shenfeld, Akash Srivastava, Yoon Kim, Pulkit Agrawal

专题命中 后训练与偏好优化 ：language model(title,abstract)；LLM(abstract,comments)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Website: https://sites.google.com/view/llm-vas

MCTS-KBQA: Monte Carlo Tree Search with Information Gain Rewards for Knowledge Base Question Answering

MCTS-KBQA：基于信息增益奖励的知识库问答蒙特卡洛树搜索

Guanming Xiong, Haochen Li, Zonghong Dai, Liqiang Wen, Wen Zhao

专题命中 后训练与偏好优化 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 该研究针对基于LLM的KBQA中MCTS应用的奖励设计与计算成本问题，提出Fast MCTS方法，以信息增益奖励替代中间状态终端展开，在四个KBQA基准上提升了准确率-成本权衡。

Comments Accepted to CIKM 2026