大模型对齐与安全 - arXivDaily 专题

2502.20757 2025-03-03 cs.CL 79%

The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agents

Yihong Tang, Kehai Chen, Xuefeng Bai, Zhengyu Niu, Bo Wang, Jie Liu, Min Zhang

专题命中偏好对齐：safety(title,abstract)；分类 cs.CL

2502.14356 2025-02-21 cs.CL 79%

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning

Huimin Xu, Xin Mao, Feng-Lin Li, Xiaobao Wu, Wang Chen, Wei Zhang, Anh Tuan Luu

专题命中偏好对齐：DPO(title,abstract)；分类 cs.CL

2502.06812 2025-02-19 cs.LG cs.GR 79%

Harness Local Rewards for Global Benefits: Effective Text-to-Video Generation Alignment with Patch-level Reward Models

Shuting Wang, Haihong Tang, Zhicheng Dou, Chenyan Xiong

专题命中偏好对齐：alignment(title)；DPO(abstract)；分类 cs.LG

2410.02743 2025-02-18 cs.CL 79%

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions

Yekun Chai, Haoran Sun, Huang Fang, Shuohuan Wang, Yu Sun, Hua Wu

专题命中偏好对齐：RLHF(title,abstract)；分类 cs.CL

2406.11107 2025-02-12 cs.CL 79%

Exploring Safety-Utility Trade-Offs in Personalized Language Models

Anvesh Rao Vijjini, Somnath Basu Roy Chowdhury, Snigdha Chaturvedi

专题命中偏好对齐：safety(title,abstract)；分类 cs.CL

Comments NAACL 2025

2403.10771 2025-02-04 cs.LG stat.ML 79%

A Probabilistic Approach for Model Alignment with Human Comparisons

Junyu Cao, Mohsen Bayati

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

2407.04185 2025-01-09 cs.CL 79%

HAF-RM: A Hybrid Alignment Framework for Reward Model Training

Shujun Liu, Xiaoyu Shen, Yuhang Lai, Siyuan Wang, Shengbin Yue, Zengfeng Huang, Xuanjing Huang, Zhongyu Wei

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

2412.10616 2024-12-17 cs.LG 79%

Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration

Avinandan Bose, Zhihan Xiong, Aadirupa Saha, Simon Shaolei Du, Maryam Fazel

专题命中偏好对齐：alignment(title)；RLHF(abstract)；分类 cs.LG

2409.12822 2024-12-10 cs.CL 79%

Language Models Learn to Mislead Humans via RLHF

Jiaxin Wen, Ruiqi Zhong, Akbir Khan, Ethan Perez, Jacob Steinhardt, Minlie Huang, Samuel R. Bowman, He He, Shi Feng

专题命中偏好对齐：RLHF(title,abstract)；分类 cs.CL

2404.16019 2024-12-04 cs.CL 79%

The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models

Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, Bertie Vidgen, Scott A. Hale

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Journal ref The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track (2024)

2406.15193 2024-11-27 cs.CL 79%

Inference Time Alignment with Reward-Guided Tree Search

Chia-Yu Hung, Navonil Majumder, Ambuj Mehrish, Soujanya Poria

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

2410.07571 2024-11-18 cs.CL cs.CV 79%

How Does Vision-Language Adaptation Impact the Safety of Vision Language Models?

Seongyun Lee, Geewook Kim, Jiyeon Kim, Hyunji Lee, Hoyeon Chang, Sue Hyun Park, Minjoon Seo

专题命中偏好对齐：safety(title,abstract)；分类 cs.CL

Comments Work in Progress

2411.09341 2024-11-15 cs.LG 79%

Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment

Yuang Cai, Yuyu Yuan, Jinsheng Shi, Qinhong Lin

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

2411.02712 2024-11-06 cs.CV cs.AI 79%

V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization

Yuxi Xie, Guanzhen Li, Xiao Xu, Min-Yen Kan

专题命中偏好对齐：DPO(title,abstract)；分类 cs.AI

Comments EMNLP 2024 Findings; 9 pages, 6 figures, 5 tables (16 pages, 8 figures, 8 tables including references and appendices)

2411.01111 2024-11-05 cs.AI 79%

Rule Based Rewards for Language Model Safety

Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian Kivlichan, Molly Lin, Alex Beutel, John Schulman, Lilian Weng

专题命中偏好对齐：safety(title,abstract)；分类 cs.AI

Comments Accepted at Neurips 2024

2410.22203 2024-10-30 cs.AI cs.HC 79%

Democratizing Reward Design for Personal and Representative Value-Alignment

Carter Blair, Kate Larson, Edith Law

专题命中偏好对齐：alignment(title,abstract)；分类 cs.AI

Comments 19 pages, 16 figures

2405.19806 2024-10-29 cs.LG 79%

Preference Alignment with Flow Matching

Minu Kim, Yongsik Lee, Sehyeok Kang, Jihwan Oh, Song Chong, Se-Young Yun

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

Comments 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

2410.17055 2024-10-24 cs.LG stat.ML 79%

Optimal Design for Reward Modeling in RLHF

Antoine Scheid, Etienne Boursier, Alain Durmus, Michael I. Jordan, Pierre Ménard, Eric Moulines, Michal Valko

专题命中偏好对齐：RLHF(title,abstract)；分类 cs.LG

2410.15651 2024-10-22 cs.LG 79%

Understanding and Alleviating Memory Consumption in RLHF for LLMs

Jin Zhou, Hanmei Yang, Steven, Tang, Mingcan Xiang, Hui Guan, Tongping Liu

专题命中偏好对齐：RLHF(title,abstract)；分类 cs.LG

2410.12194 2024-10-17 cs.CL 79%

Negative-Prompt-driven Alignment for Generative Language Model

Shiqi Qiao, Ning Xv, Biao Liu, Xin Geng

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

2404.12318 2024-10-15 cs.CL 79%

Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment

Zhaofeng Wu, Ananth Balashankar, Yoon Kim, Jacob Eisenstein, Ahmad Beirami

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments EMNLP 2024

2406.15951 2024-10-14 cs.CL 79%

Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration

Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, Yulia Tsvetkov

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments EMNLP 2024

2406.09279 2024-10-10 cs.CL 79%

Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback

Hamish Ivison, Yizhong Wang, Jiacheng Liu, Zeqiu Wu, Valentina Pyatkin, Nathan Lambert, Noah A. Smith, Yejin Choi, Hannaneh Hajishirzi

专题命中偏好对齐：DPO(title,abstract)；分类 cs.CL

Comments Neurips 2024 camera-ready

2402.05000 2024-10-08 cs.CL 79%

Pedagogical Alignment of Large Language Models

Shashank Sonkar, Kangqi Ni, Sapana Chaudhary, Richard G. Baraniuk

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments Accepted at EMNLP 2024 Findings Track

2312.07401 2024-10-08 cs.AI 79%

On Diversified Preferences of Large Language Model Alignment

Dun Zeng, Yong Dai, Pengyu Cheng, Longyue Wang, Tianhao Hu, Wanshun Chen, Nan Du, Zenglin Xu

专题命中偏好对齐：alignment(title,abstract)；分类 cs.AI

Comments EMNLP 2024

2406.12606 2024-10-04 cs.CL 79%

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment

Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Jingyuan Wang, Ji-Rong Wen

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments 14 pages, working in progress

2408.15116 2024-08-28 cs.AI 79%

Evaluating Stability of Unreflective Alignment

James Lucassen, Mark Henry, Philippa Wright, Owen Yeung

专题命中偏好对齐：alignment(title,abstract)；分类 cs.AI

2407.17387 2024-07-25 cs.CL 79%

PERSONA: A Reproducible Testbed for Pluralistic Alignment

Louis Castricato, Nathan Lile, Rafael Rafailov, Jan-Philipp Fränken, Chelsea Finn

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

2311.06025 2024-07-17 cs.CL 79%

ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences

Yuanhe Tian, Ruyi Gan, Yan Song, Jiaxing Zhang, Yongdong Zhang

专题命中偏好对齐：alignment(title)；RLHF(abstract)；分类 cs.CL

Comments 18 pages, 3 figures; Accepted by ACL-2024

2402.13210 2024-07-04 cs.LG 79%

Bayesian Reward Models for LLM Alignment

Adam X. Yang, Maxime Robeyns, Thomas Coste, Zhengyan Shi, Jun Wang, Haitham Bou-Ammar, Laurence Aitchison

专题命中偏好对齐：alignment(title)；RLHF(abstract)；分类 cs.LG

1. 偏好对齐 3235 篇

The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agents

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning

Harness Local Rewards for Global Benefits: Effective Text-to-Video Generation Alignment with Patch-level Reward Models

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions

Exploring Safety-Utility Trade-Offs in Personalized Language Models

A Probabilistic Approach for Model Alignment with Human Comparisons

HAF-RM: A Hybrid Alignment Framework for Reward Model Training

Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration

Language Models Learn to Mislead Humans via RLHF

The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models

Inference Time Alignment with Reward-Guided Tree Search

How Does Vision-Language Adaptation Impact the Safety of Vision Language Models?

Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment

V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization

Rule Based Rewards for Language Model Safety

Democratizing Reward Design for Personal and Representative Value-Alignment

Preference Alignment with Flow Matching

Optimal Design for Reward Modeling in RLHF

Understanding and Alleviating Memory Consumption in RLHF for LLMs

Negative-Prompt-driven Alignment for Generative Language Model

Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment

Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration

Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback

Pedagogical Alignment of Large Language Models

On Diversified Preferences of Large Language Model Alignment

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment

Evaluating Stability of Unreflective Alignment

PERSONA: A Reproducible Testbed for Pluralistic Alignment

ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences

Bayesian Reward Models for LLM Alignment