语言大模型 / LLM

Linear Preference Optimization: Decoupled Gradient Control via Absolute Regularization

Rui Wang, Qianguo Sun, Chao Song, Junlong Wu, Tianrong Chen, Zhiyun Zeng, Yu Li

机构 * International Digital Economy Academy（国际数字经济学院） ； Emdoor Collaborative Laboratory（Emdoor协同实验室） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.LG

ASAP: Unsupervised Post-training with Label Distribution Shift Adaptive Learning Rate

Heewon Park, Mugon Joe, Miru Kim, Minhae Kwon

机构 * Soongsil University（顺斯大学） 纠错

专题命中 后训练与偏好优化 ：post-training(title,abstract)；分类 cs.LG

Comments 5 pages, 3 figures, accepted for ACM CIKM 2025

AddressVLM: Cross-view Alignment Tuning for Image Address Localization using Large Vision-Language Models

Shixiong Xu, Chenghao Zhang, Lubin Fan, Yuan Zhou, Bin Fan, Shiming Xiang, Gaofeng Meng, Jieping Ye

机构 * State Key Laboratory of Multimodal Artificial Intelligence Systems（多模态人工智能系统国家重点实验室） ； CASIA（中国科学院自动化所） ； Alibaba Cloud（阿里云） ； School of Intelligence Science and Technology（智能科学与技术学院） ； CAIR（中国科学院香港创新研究院） 纠错

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.AI

Policy Learning from Large Vision-Language Model Feedback without Reward Modeling

Tung M. Luu, Donghoon Lee, Younghwan Lee, Chang D. Yoo

机构 * School of Electrical Engineering, KAIST（韩国科学技术院电子工程学院） 纠错

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.LG

Comments Accepted to IROS 2025

Unlearning of Knowledge Graph Embedding via Preference Optimization

Jiajun Liu, Wenjun Ke, Peng Wang, Yao He, Ziyu Shang, Guozheng Li, Zijie Xu, Ke Ji

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI

Learning to Translate Ambiguous Terminology by Preference Optimization on Post-Edits

Nathaniel Berger, Johannes Eschbach-Dymanus, Miriam Exel, Matthias Huck, Stefan Riezler

机构 * Computational Linguistics &（计算语言学） ； IWR, Heidelberg University（海德堡大学IWR） ； SAP SE（SAP公司） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL

BOPO: Neural Combinatorial Optimization via Best-anchored and Objective-guided Preference Optimization

Zijun Liao, Jinbiao Chen, Debing Wang, Zizhen Zhang, Jiahai Wang

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.LG

Comments This paper has been accepted by ICML 2025

Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence

Shangbin Feng, Zifeng Wang, Yike Wang, Sayna Ebrahimi, Hamid Palangi, Lesly Miculicich, Achin Kulshrestha, Nathalie Rauschmayr, Yejin Choi, Yulia Tsvetkov, Chen-Yu Lee, Tomas Pfister

专题命中 后训练与偏好优化 ：LLM(title,abstract)；分类 cs.CL

Comments ICML 2025

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation

Juncheol Shin, Minsang Seok, Seonggon Kim, Eunhyeok Park

专题命中 后训练与偏好优化 ：post-training(title,abstract)；分类 cs.LG

Comments ICML 2025. Code: https://github.com/ewsn1593/HDRQ

Square$χ$PO: Differentially Private and Robust $χ^2$-Preference Optimization in Offline Direct Alignment

Xingyu Zhou, Yulian Wu, Wenqian Weng, Francesco Orabona

专题命中 后训练与偏好优化 ：preference optimization(title)；language model(abstract)；分类 cs.LG

Learning a Pessimistic Reward Model in RLHF

Yinglun Xu, Hangoo Kang, Tarun Suresh, Yuxuan Wan, Gagandeep Singh

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.LG

ProDS: Preference-oriented Data Selection for Instruction Tuning

Wenya Guo, Zhengkun Zhang, Xumeng Liu, Ying Zhang, Ziyu Lu, Haoze Zhu, Xubo Liu, Ruxue Yan

机构 * College of Computer Science, Nankai University（南开大学计算机科学学院） ； Baidu Inc.（百度公司） 纠错

专题命中 后训练与偏好优化 ：instruction tuning(title)；preference optimization(abstract)；分类 cs.LG

Towards Self-Improvement of Diffusion Models via Group Preference Optimization

Renjie Chen, Wenfeng Lin, Yichen Zhang, Jiangchuan Wei, Boyuan Liu, Chao Feng, Jiao Ran, Mingyu Guo

机构 * ByteDance Douyin Content Group（字节跳动抖音内容团队） 纠错

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI

Online Bandit Learning with Offline Preference Data for Improved RLHF

Akhil Agnihotri, Rahul Jain, Deepak Ramachandran, Zheng Wen

机构 * University of Southern California（南加州大学） ； Google DeepMind（谷歌DeepMind） ； USC（南加州大学） 纠错

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.LG

Preference Optimization for Combinatorial Optimization Problems

Mingjun Pan, Guanquan Lin, You-Wei Luo, Bin Zhu, Zhien Dai, Lijun Sun, Chun Yuan

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.LG

Comments This paper has been accepted by ICML 2025

FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback

Liqiang Jing, Xinya Du

机构 * Department of Computer Science, The University of Texas at Dallas（计算机科学系，德克萨斯大学达拉斯分校） 纠错

专题命中 后训练与偏好优化 ：language model(title,abstract)；分类 cs.CL

Pack-PTQ: Advancing Post-training Quantization of Neural Networks by Pack-wise Reconstruction

Changjun Li, Runqing Jiang, Zhuo Song, Pengpeng Yu, Ye Zhang, Yulan Guo

机构 * Shenzhen Campus, Sun Yat-sen University（中山大学深圳校区） 纠错

专题命中 后训练与偏好优化 ：post-training(title,abstract)；分类 cs.AI

PTQ4VM: Post-Training Quantization for Visual Mamba

Younghyun Cho, Changhun Lee, Seonggon Kim, Eunhyeok Park

专题命中 后训练与偏好优化 ：post-training(title,abstract)；分类 cs.LG

Comments Accepted at WACV 2025 (oral presentation)

De Novo Molecular Design Enabled by Direct Preference Optimization and Curriculum Learning

Junyu Hou

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.LG

One Framework to Rule Them All: Unifying RL-Based and RL-Free Methods in RLHF

Xin Cai

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.LG

RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization

Hanyang Zhao, Genta Indra Winata, Anirban Das, Shi-Xiong Zhang, David D. Yao, Wenpin Tang, Sambit Sahu

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI

UltraGen: Extremely Fine-grained Controllable Generation via Attribute Reconstruction and Global Preference Optimization

Longfei Yun, Letian Peng, Jingbo Shang

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.CL

Direct Preference Optimization-Enhanced Multi-Guided Diffusion Model for Traffic Scenario Generation

Seungjun Yu, Kisung Kim, Daejung Kim, Haewook Han, Jinhan Lee

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.LG

Sharp Analysis for KL-Regularized Contextual Bandits and RLHF

Heyang Zhao, Chenlu Ye, Quanquan Gu, Tong Zhang

专题命中 后训练与偏好优化 ：RLHF(title,abstract)；分类 cs.LG

Preference Optimization via Contrastive Divergence: Your Reward Model is Secretly an NLL Estimator

Zhuotong Chen, Fang Liu, Xuan Zhu, Yanjun Qi, Mohammad Ghavamzadeh

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI

Data Generation for Hardware-Friendly Post-Training Quantization

Lior Dikstein, Ariel Lapid, Arnon Netzer, Hai Victor Habi

专题命中 后训练与偏好优化 ：post-training(title,abstract)；分类 cs.LG

DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning

Utsav Singh, Souradip Chakraborty, Wesley A. Suttle, Brian M. Sadler, Vinay P Namboodiri, Amrit Singh Bedi

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.LG

Comments This paper is subsumed by a later paper of ours: arXiv:2411.00361

Enhancing Audiovisual Speech Recognition through Bifocal Preference Optimization

Yihan Wu, Yichen Lu, Yifan Peng, Xihua Wang, Ruihua Song, Shinji Watanabe

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI

Comments Accepted by AAAI 2025

Empathetic Response in Audio-Visual Conversations Using Emotion Preference Optimization and MambaCompressor

Yeonju Kim, Se Jin Park, Yong Man Ro

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.AI

Forward KL Regularized Preference Optimization for Aligning Diffusion Policies

Zhao Shan, Chenyou Fan, Shuang Qiu, Jiyuan Shi, Chenjia Bai

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；分类 cs.LG