语言大模型 / LLM - arXivDaily 专题

2505.18531 2025-05-27 cs.AI cs.CV 85%

Generative RLHF-V: Learning Principles from Multi-modal Human Preference

Jiayi Zhou, Jiaming Ji, Boyuan Chen, Jiapeng Sun, Wenqi Chen, Donghai Hong, Sirui Han, Yike Guo, Yaodong Yang

机构 * Peking University（北京大学）； Hong Kong University of Science and Technology（香港科技大学）； University College London（伦敦大学学院）

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 9 pages, 8 figures

2501.18101 2025-05-23 cs.CL 85%

Diverse Preference Optimization

Jack Lanchantin, Angelica Chen, Shehzaad Dhuliawala, Ping Yu, Jason Weston, Sainbayar Sukhbaatar, Ilia Kulikov

机构 * Meta

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL

2407.06443 2025-04-29 cs.AI 85%

Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment

Qizhang Feng, Siva Rajesh Kasa, Santhosh Kumar Kasa, Hyokun Yun, Choon Hui Teo, Sravan Babu Bodapati

机构 * Amazon Inc.（亚马逊公司）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2504.15804 2025-04-23 cs.AR cs.AI 85%

Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback

Ning Wang, Bingkun Yao, Jie Zhou, Yuchen Hu, Xi Wang, Nan Guan, Zhe Jiang

机构 * City University of Hong Kong（香港城市大学）； Southeast University（东南大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2504.13122 2025-04-18 cs.CV cs.LG 85%

VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models

Haojian Huang, Haodong Chen, Shengqiong Wu, Meng Luo, Jinlan Fu, Xinya Du, Hanwang Zhang, Hao Fei

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Code and Data: https://github.com/HaroldChen19/VistaDPO

2411.08923 2025-03-27 cs.CV cs.LG 85%

Aligning Visual Contrastive learning models via Preference Optimization

Amirabbas Afzali, Borna Khodabandeh, Ali Rasekh, Mahyar JafariNodeh, Sepehr kazemi, Simon Gottschalk

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；RLHF(abstract)；分类 cs.LG

2503.18454 2025-03-25 cs.CV cs.LG 85%

InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment

Yunhong Lu, Qichao Wang, Hengyuan Cao, Xierui Wang, Xiaoyin Xu, Min Zhang

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Accepted by CVPR2025

2503.14189 2025-03-19 cs.CL cs.CV 85%

Towards Harmless Multimodal Assistants with Blind Preference Optimization

Yongqi Li, Lu Yang, Jian Wang, Runyang You, Wenjie Li, Liqiang Nie

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2503.05079 2025-03-10 cs.LG 85%

On a Connection Between Imitation Learning and RLHF

Teng Xiao, Yige Yuan, Mingxiao Li, Zhengyu Chen, Vasant G Honavar

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments ICLR 2025

2410.09724 2025-03-04 cs.CL 85%

Taming Overconfidence in LLMs: Reward Calibration in RLHF

Jixuan Leng, Chengsong Huang, Banghua Zhu, Jiaxin Huang

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2410.17146 2025-03-03 cs.LG cs.CV 85%

LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging

Ke Wang, Nikolaos Dimitriadis, Alessandro Favero, Guillermo Ortiz-Jimenez, Francois Fleuret, Pascal Frossard

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；RLHF(abstract)；分类 cs.LG

Comments The first two authors contributed equally to this work. Accepted at ICLR 2025. Project website: https://lines-merging.github.io

2408.07471 2025-02-19 cs.CL 85%

Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization

Yuxin Jiang, Bo Huang, Yufei Wang, Xingshan Zeng, Liangyou Li, Yasheng Wang, Xin Jiang, Lifeng Shang, Ruiming Tang, Wei Wang

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 20 pages, 9 figures, 12 tables. Accepted at ICLR 2025

2502.11520 2025-02-18 cs.CL 85%

AURORA:Automated Training Framework of Universal Process Reward Models via Ensemble Prompting and Reverse Verification

Xiaoyu Tan, Tianchu Yao, Chao Qu, Bin Li, Minghao Yang, Dakuan Lu, Haozhe Wang, Xihe Qiu, Wei Chu, Yinghui Xu, Yuan Qi

专题命中后训练与偏好优化：prompting(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Under Review

2410.02743 2025-02-18 cs.CL 85%

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions

Yekun Chai, Haoran Sun, Huang Fang, Shuohuan Wang, Yu Sun, Hua Wu

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2501.18457 2025-02-11 cs.CL 85%

CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering

Yumeng Wang, Zhiyuan Fan, Qingyun Wang, May Fung, Heng Ji

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted by NAACL 2025

2501.16629 2025-01-29 cs.CL cs.CV 85%

CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs

Jinlan Fu, Shenzhen Huangfu, Hao Fei, Xiaoyu Shen, Bryan Hooi, Xipeng Qiu, See-Kiong Ng

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by ICLR 2025

2411.02337 2025-01-28 cs.CL 85%

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning

Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, Yuxiao Dong

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Published as a conference paper at ICLR 2025

2412.20996 2024-12-31 cs.CL 85%

Plug-and-Play Training Framework for Preference Optimization

Jingyuan Ma, Rui Li, Zheng Li, Lei Sha, Zhifang Sui

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 12 pages, 9 figures

2404.14285 2024-12-31 cs.RO cs.AI 85%

LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robots

Dongge Han, Trevor McInroe, Adam Jelley, Stefano V. Albrecht, Peter Bell, Amos Storkey

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments COLING 2025

2411.15708 2024-11-26 cs.CL 85%

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training

Xiaoye Qu, Daize Dong, Xuyang Hu, Tong Zhu, Weigao Sun, Yu Cheng

专题命中后训练与偏好优化：post-training(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Technical report,13 pages

2410.04834 2024-10-28 cs.CL 85%

As Simple as Fine-tuning: LLM Alignment via Bidirectional Negative Feedback Loss

Xin Mao, Feng-Lin Li, Huimin Xu, Wei Zhang, Wang Chen, Anh Tuan Luu

专题命中后训练与偏好优化：LLM(title,abstract)；RLHF(abstract)；preference optimization(abstract)；分类 cs.CL

Comments 20 pages, 9 figures

2407.19829 2024-10-28 cs.IR cs.AI 85%

Generative Retrieval with Preference Optimization for E-commerce Search

Mingming Li, Huimu Wang, Zuxu Chen, Guangtao Nie, Yiming Qiu, Guoyu Tang, Lin Liu, Jingwei Zhuo

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2410.15651 2024-10-22 cs.LG 85%

Understanding and Alleviating Memory Consumption in RLHF for LLMs

Jin Zhou, Hanmei Yang, Steven, Tang, Mingcan Xiang, Hui Guan, Tongping Liu

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2410.15096 2024-10-22 cs.AI 85%

GDPO: Learning to Directly Align Language Models with Diversity Using GFlowNets

Oh Joon Kwon, Daiki E. Matsunaga, Kee-Eung Kim

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；preference optimization(abstract)；分类 cs.AI

Journal ref EMNLP 2024

2406.19949 2024-10-15 cs.CL 85%

Calibrating LLMs with Preference Optimization on Thought Trees for Generating Rationale in Science Question Scoring

Jiazheng Li, Hainiu Xu, Zhaoyue Sun, Yuxiang Zhou, David West, Cesare Aloisi, Yulan He

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted at EMNLP 2024

2406.15951 2024-10-14 cs.CL 85%

Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration

Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, Yulia Tsvetkov

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2024

2409.08845 2024-09-16 cs.CL 85%

AIPO: Improving Training Objective for Iterative Preference Optimization

Yaojie Shen, Xinyao Wang, Yulei Niu, Ying Zhou, Lexin Tang, Libo Zhang, Fan Chen, Longyin Wen

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2407.05013 2024-07-09 cs.CL 85%

Progress or Regress? Self-Improvement Reversal in Post-training

Ting Wu, Xuefeng Li, Pengfei Liu

专题命中后训练与偏好优化：post-training(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2402.16029 2024-07-04 cs.CL 85%

GraphWiz: An Instruction-Following Language Model for Graph Problems

Nuo Chen, Yuhan Li, Jianheng Tang, Jia Li

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；preference optimization(abstract)；分类 cs.CL

Comments 27pages, 15 tables

2406.17092 2024-06-26 cs.CR cs.AI 85%

BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models

Yi Zeng, Weiyu Sun, Tran Ngoc Huynh, Dawn Song, Bo Li, Ruoxi Jia

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；分类 cs.AI

1. 后训练与偏好优化 4526 篇

Generative RLHF-V: Learning Principles from Multi-modal Human Preference

Diverse Preference Optimization

Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment

Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback

VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models

Aligning Visual Contrastive learning models via Preference Optimization

InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment

Towards Harmless Multimodal Assistants with Blind Preference Optimization

On a Connection Between Imitation Learning and RLHF

Taming Overconfidence in LLMs: Reward Calibration in RLHF

LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging

Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization

AURORA:Automated Training Framework of Universal Process Reward Models via Ensemble Prompting and Reverse Verification

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions

CALM: Unleashing the Cross-Lingual Self-Aligning Ability of Language Model Question Answering

CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning

Plug-and-Play Training Framework for Preference Optimization

LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robots

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training

As Simple as Fine-tuning: LLM Alignment via Bidirectional Negative Feedback Loss

Generative Retrieval with Preference Optimization for E-commerce Search

Understanding and Alleviating Memory Consumption in RLHF for LLMs

GDPO: Learning to Directly Align Language Models with Diversity Using GFlowNets

Calibrating LLMs with Preference Optimization on Thought Trees for Generating Rationale in Science Question Scoring

Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration

AIPO: Improving Training Objective for Iterative Preference Optimization

Progress or Regress? Self-Improvement Reversal in Post-training

GraphWiz: An Instruction-Following Language Model for Graph Problems

BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models