语言大模型 / LLM - arXivDaily 专题

2502.00657 2025-10-22 cs.LG cs.AI cs.CY stat.ML 84%

LLM Safety Alignment is Divergence Estimation in Disguise

Rajdeep Haldar, Ziyi Wang, Qifan Song, Guang Lin, Yue Xing

机构 * Department of Statistics, Purdue University（普渡大学统计系）； Department of Statistics, Michigan State University（密歇根州立大学统计系）

专题命中后训练与偏好优化：LLM(title,abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments Accepted to NeurIPS 2025

2408.12112 2025-10-22 cs.LG cs.AI cs.MA 84%

Balancing Act: Prioritization Strategies for LLM-Designed Restless Bandit Rewards

Shresth Verma, Niclas Boehmer, Lingkai Kong, Milind Tambe

机构 * Harvard University, USA（哈佛大学）； Hasso Plattner Institute, Germany（哈索普拉特纳研究所）

专题命中后训练与偏好优化：LLM(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

2508.12081 2025-10-21 cs.CV cs.AI cs.CL 84%

VimoRAG: Video-based Retrieval-augmented 3D Motion Generation for Motion Language Models

Haidong Xu, Guangwei Xu, Zhedong Zheng, Xiatian Zhu, Wei Ji, Xiangtai Li, Ruijie Guo, Meishan Zhang, Min zhang, Hao Fei

机构 * Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））； University of Macau（澳门大学）； University of Surrey（Surrey大学）； Nanjing University（南京大学）； Nanyang Technological University（南洋理工大学）； National University of Singapore（新加坡国立大学）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NeurIPS 2025; Project Page: https://walkermitty.github.io/VimoRAG

2510.05410 2025-10-08 cs.CL cs.LG 84%

Aligning Language Models with Clinical Expertise: DPO for Heart Failure Nursing Documentation in Critical Care

Junyi Fan, Li Sun, Negin Ashrafi, Kamiar Alaei, Maryam Pishgar

机构 * University of Southern California（南加州大学）； California State University（加州州立大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.LG

2509.24713 2025-09-30 cs.LG cs.AI 84%

Circuit-Aware Reward Training: A Mechanistic Framework for Longtail Robustness in RLHF

Jing Liu

机构 * ENS, Université PSL, EHESS, CNRS（高等科学研究所、巴黎国立大学、EHESS、国家科学研究中心）

专题命中后训练与偏好优化：RLHF(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

2503.12123 2025-09-22 cs.CL cs.AI 84%

MT-RewardTree: A Comprehensive Framework for Advancing LLM-Based Machine Translation via Reward Modeling

Zhaopeng Feng, Jiahan Ren, Jiayuan Su, Jiamei Zheng, Hongwei Wang, Zuozhu Liu

机构 * Zhejiang University（浙江大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2025 Findings. Project page:https://sabijun.github.io/MT_RewardTreePage

2504.13134 2025-08-06 cs.CL cs.LG stat.ML 84%

Energy-Based Reward Models for Robust Language Model Alignment

Anamika Lochab, Ruqi Zhang

机构 * Department of Computer Science（计算机科学系）； Purdue University（普渡大学）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted by COLM 2025

2501.03575 2025-07-11 cs.CV cs.AI cs.LG cs.RO 84%

Cosmos World Foundation Model Platform for Physical AI

NVIDIA, :, Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, Daniel Dworakowski, Jiaojiao Fan, Michele Fenzi, Francesco Ferroni, Sanja Fidler, Dieter Fox, Songwei Ge, Yunhao Ge, Jinwei Gu, Siddharth Gururani, Ethan He, Jiahui Huang, Jacob Huffman, Pooya Jannaty, Jingyi Jin, Seung Wook Kim, Gergely Klár, Grace Lam, Shiyi Lan, Laura Leal-Taixe, Anqi Li, Zhaoshuo Li, Chen-Hsuan Lin, Tsung-Yi Lin, Huan Ling, Ming-Yu Liu, Xian Liu, Alice Luo, Qianli Ma, Hanzi Mao, Kaichun Mo, Arsalan Mousavian, Seungjun Nah, Sriharsha Niverty, David Page, Despoina Paschalidou, Zeeshan Patel, Lindsey Pavao, Morteza Ramezanali, Fitsum Reda, Xiaowei Ren, Vasanth Rao Naik Sabavat, Ed Schmerling, Stella Shi, Bartosz Stefaniak, Shitao Tang, Lyne Tchapmi, Przemek Tredak, Wei-Cheng Tseng, Jibin Varghese, Hao Wang, Haoxiang Wang, Heng Wang, Ting-Chun Wang, Fangyin Wei, Xinyue Wei, Jay Zhangjie Wu, Jiashu Xu, Wei Yang, Lin Yen-Chen, Xiaohui Zeng, Yu Zeng, Jing Zhang, Qinsheng Zhang, Yuxuan Zhang, Qingqing Zhao, Artur Zolkowski

机构 * NVIDIA

专题命中后训练与偏好优化：foundation model(title,abstract)；post-training(abstract)；分类 cs.AI、cs.LG

2506.05748 2025-06-09 cs.LG cs.AI 84%

Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performance

Rudransh Agnihotri, Ananya Pandey

专题命中后训练与偏好优化：LLM(title,abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

2410.20898 2025-06-06 cs.CV cs.AI cs.LG cs.MM 84%

David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training

Weijian Luo, Colin Zhang, Debing Zhang, Zhengyang Geng

专题命中后训练与偏好优化：post-training(title,abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments Revision: paper accepted by the ICML2025 main conference

2505.24369 2025-06-02 cs.LG cs.AI 84%

Adversarial Preference Learning for Robust LLM Alignment

Yuanfu Wang, Pengyu Wang, Chenyang Xi, Bo Tang, Junyi Zhu, Wenqiang Wei, Chen Chen, Chao Yang, Jingfeng Zhang, Chaochao Lu, Yijun Niu, Keming Mao, Zhiyu Li, Feiyu Xiong, Jie Hu, Mingchuan Yang

专题命中后训练与偏好优化：LLM(title)；language model(abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ACL2025 Findings

2401.16332 2025-05-28 cs.CL cs.AI 84%

Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methods

Yotam Wolf, Noam Wies, Dorin Shteyman, Binyamin Rothberg, Yoav Levine, Amnon Shashua

机构 * The Hebrew University（希伯来大学）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(abstract)；分类 cs.CL、cs.AI

2505.17016 2025-05-23 cs.LG cs.AI cs.CV cs.RO 84%

Interactive Post-Training for Vision-Language-Action Models

Shuhan Tan, Kairan Dou, Yue Zhao, Philipp Krähenbühl

专题命中后训练与偏好优化：post-training(title,abstract)；SFT(abstract)；分类 cs.AI、cs.LG

Comments Project page: https://ariostgx.github.io/ript_vla/

2505.15694 2025-05-22 cs.LG cs.AI 84%

A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO

Xingyu Zhou, Yulian Wu, Francesco Orabona

专题命中后训练与偏好优化：RLHF(title,abstract)；preference optimization(abstract)；分类 cs.AI、cs.LG

2505.12504 2025-05-20 cs.LG cs.AI 84%

CPGD: Toward Stable Rule-based Reinforcement Learning for Language Models

Zongkai Liu, Fanqing Meng, Lingxiao Du, Zhixiang Zhou, Chao Yu, Wenqi Shao, Qiaosheng Zhang

机构 * Sun Yat-Sen University（中山大学）； Shanghai Innovation Institute（上海创新研究院）； Shanghai AI Laboratory（上海人工智能实验室）； Shanghai Jiao Tong University（上海交通大学）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(abstract)；分类 cs.AI、cs.LG

2505.06273 2025-05-14 cs.LG cs.AI 84%

Policy-labeled Preference Learning: Is Preference Enough for RLHF?

Taehyun Cho, Seokhun Ju, Seungyub Han, Dohyeong Kim, Kyungjae Lee, Jungwoo Lee

专题命中后训练与偏好优化：RLHF(title,abstract)；preference optimization(abstract)；分类 cs.AI、cs.LG

2405.13637 2025-05-12 cs.CV cs.AI cs.LG 84%

Curriculum Direct Preference Optimization for Diffusion and Consistency Models

Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, Nicu Sebe, Mubarak Shah

机构 * University of Bucharest（布加勒斯特大学）； University of Trento（特伦托大学）； University of Central Florida（中央佛罗里达大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments Accepted at CVPR 2025

2406.01660 2025-04-15 cs.LG cs.AI stat.ML 84%

Self-Improving Robust Preference Optimization

Eugene Choi, Arash Ahmadian, Matthieu Geist, Oilvier Pietquin, Mohammad Gheshlaghi Azar

专题命中后训练与偏好优化：preference optimization(title,abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

2504.01324 2025-04-03 cs.CV cs.AI cs.CL 84%

On Data Synthesis and Post-training for Visual Abstract Reasoning

Ke Zhu, Yu Wang, Jiangjiang Liu, Qunyi Xie, Shanshan Liu, Gang Zhang

专题命中后训练与偏好优化：post-training(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI

2503.15880 2025-03-21 cs.LG cs.CL 84%

InCo-DPO: Balancing Distribution Shift and Data Quality for Enhanced Preference Optimization

Yunan Wang, Jijie Li, Bo-Wen Zhang, Liangdong Wang, Guang Liu

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

2410.04734 2025-02-26 cs.LG cs.CL cs.CV 84%

TLDR: Token-Level Detective Reward Model for Large Vision Language Models

Deqing Fu, Tong Xiao, Rui Wang, Wang Zhu, Pengchuan Zhang, Guan Pang, Robin Jia, Lawrence Chen

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Published as a conference paper at ICLR 2025

2502.15145 2025-02-25 cs.LG cs.AI 84%

Projection Optimization: A General Framework for Multi-Objective and Multi-Group RLHF

Nuoya Xiong, Aarti Singh

专题命中后训练与偏好优化：RLHF(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

2502.10325 2025-02-17 cs.LG cs.AI 84%

Process Reward Models for LLM Agents: Practical Framework and Directions

Sanjiban Choudhury

专题命中后训练与偏好优化：LLM(title,abstract)；RLHF(abstract)；分类 cs.AI、cs.LG

Comments 17 pages, 7 figures

2412.07174 2024-12-11 cs.LG cs.AI 84%

Post-Training Statistical Calibration for Higher Activation Sparsity

Vui Seng Chua, Yujie Pan, Nilesh Jain

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

Comments ENLSP-IV NeurIPS Workshop 2024

2405.14734 2024-11-05 cs.CL cs.LG 84%

SimPO: Simple Preference Optimization with a Reference-Free Reward

Yu Meng, Mengzhou Xia, Danqi Chen

专题命中后训练与偏好优化：preference optimization(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2024. Code & models: https://github.com/princeton-nlp/SimPO. v3 updates: Gemma 2 results (Appendix J); more discussions about length normalization (Section 2.2) and KL regularization (Section 2.3)

2402.09345 2024-11-04 cs.LG cs.AI 84%

InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling

Yuchun Miao, Sen Zhang, Liang Ding, Rong Bao, Lefei Zhang, Dacheng Tao

专题命中后训练与偏好优化：RLHF(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments The paper has been accepted by NeurIPS 2024

2402.05369 2024-10-31 cs.LG cs.CL 84%

Noise Contrastive Alignment of Language Models with Explicit Rewards

Huayu Chen, Guande He, Lifan Yuan, Ganqu Cui, Hang Su, Jun Zhu

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2024

2409.12403 2024-09-20 cs.CL cs.AI 84%

Preference Alignment Improves Language Model-Based TTS

Jinchuan Tian, Chunlei Zhang, Jiatong Shi, Hao Zhang, Jianwei Yu, Shinji Watanabe, Dong Yu

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2407.15184 2024-07-23 cs.CY cs.AI cs.CL 84%

Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment

Karina Vida, Fabian Damken, Anne Lauscher

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments to be published in AIES 2024 Proceedings

2406.06496 2024-06-18 cs.LG cs.CL cs.CV 84%

Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generation

Oishi Banerjee, Hong-Yu Zhou, Subathra Adithan, Stephen Kwak, Kay Wu, Pranav Rajpurkar

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Added acknowledgemnts

1. 后训练与偏好优化 4526 篇

LLM Safety Alignment is Divergence Estimation in Disguise

Balancing Act: Prioritization Strategies for LLM-Designed Restless Bandit Rewards

VimoRAG: Video-based Retrieval-augmented 3D Motion Generation for Motion Language Models

Aligning Language Models with Clinical Expertise: DPO for Heart Failure Nursing Documentation in Critical Care

Circuit-Aware Reward Training: A Mechanistic Framework for Longtail Robustness in RLHF

MT-RewardTree: A Comprehensive Framework for Advancing LLM-Based Machine Translation via Reward Modeling

Energy-Based Reward Models for Robust Language Model Alignment

Cosmos World Foundation Model Platform for Physical AI

Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performance

David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training

Adversarial Preference Learning for Robust LLM Alignment

Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methods

Interactive Post-Training for Vision-Language-Action Models

A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO

CPGD: Toward Stable Rule-based Reinforcement Learning for Language Models

Policy-labeled Preference Learning: Is Preference Enough for RLHF?

Curriculum Direct Preference Optimization for Diffusion and Consistency Models

Self-Improving Robust Preference Optimization

On Data Synthesis and Post-training for Visual Abstract Reasoning

InCo-DPO: Balancing Distribution Shift and Data Quality for Enhanced Preference Optimization

TLDR: Token-Level Detective Reward Model for Large Vision Language Models

Projection Optimization: A General Framework for Multi-Objective and Multi-Group RLHF

Process Reward Models for LLM Agents: Practical Framework and Directions

Post-Training Statistical Calibration for Higher Activation Sparsity

SimPO: Simple Preference Optimization with a Reference-Free Reward

InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling

Noise Contrastive Alignment of Language Models with Explicit Rewards

Preference Alignment Improves Language Model-Based TTS

Decoding Multilingual Moral Preferences: Unveiling LLM's Biases Through the Moral Machine Experiment

Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generation