大模型对齐与安全 - arXivDaily 专题

2510.19474 2025-11-27 cs.LG 79%

g-DPO: Scalable Preference Optimization for Protein Language Models

g-DPO：可扩展的偏好优化用于蛋白质语言模型

Constance Ferragu, Jonathan D. Ziegler, Nicolas Deutschmann, Arthur Lindoulsi, Eli Bixby, Cradle ML Team

机构 * Cradle ML Team（Cradle团队）； Cradle Zürich, Switzerland（Cradle瑞士苏黎世）

专题命中偏好对齐：DPO(title,abstract)；分类 cs.LG

AI总结 g-DPO通过序列聚类和组近似方法提升蛋白质语言模型的偏好优化效率，实现更快的收敛速度和与标准DPO相当的性能。

Comments Accepted at two workshops: FM4LS NeurIPS 2025 (accepted-paper.html" target="_blank" rel="noopener">https://nips2025fm4ls.github.io/pages/accepted-paper.html) and MLSB in Copenhagen EurIPS 2025

2409.20059 2025-11-21 cs.CL 79%

Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis

基于偏好对齐是否总是提升基于大语言模型的翻译质量的最佳选项？一项实证分析

Hippolyte Gisserot-Boukhlef, Ricardo Rei, Emmanuel Malherbe, Céline Hudelot, Pierre Colombo, Nuno M. Guerreiro

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

AI总结本研究通过实证分析探讨了基于偏好的对齐在提升大语言模型翻译质量中的效果，发现CPO在高质量数据上表现优异，但可能在下游度量上存在不稳定性，同时基础模型生成翻译的表现与多个外部系统相当且更一致。

2508.17184 2025-11-20 cs.CL 79%

Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models

Xudong Han, Junjie Yang, Tianyang Wang, Ziqian Bi, Xinyuan Song, Junfeng Hao, Junhao Song

机构 * Department of Informatics, University of Sussex（信息学院，苏塞克斯大学）； Pingtan Research Institute, Xiamen University（平潭研究院，厦门大学）； Department of Computer Science, University of Liverpool（计算机科学系，利物浦大学）； Department of Computer Science, Purdue University（计算机科学系，普渡大学）； Department of Computer Science, Emory University（计算机科学系，埃默里大学）； AI Agent Lab, Vokram Group（AI代理实验室，Vokram集团）； Department of Computing, Imperial College London（计算系，帝国理工学院伦敦分校）

专题命中偏好对齐：alignment(title)；safety(abstract)；分类 cs.CL

Comments 24 pages, 7 figures, 5 tables

2508.09724 2025-11-18 cs.AI 79%

UDA: Unsupervised Debiasing Alignment for Pair-wise LLM-as-a-Judge

Yang Zhang, Cunxiang Wang, Lindong Wu, Wenbo Yu, Yidong Wang, Guangsheng Bao, Jie Tang

专题命中偏好对齐：alignment(title,abstract)；分类 cs.AI

2501.13264 2025-11-13 cs.CL 79%

OpenGenAlign: A Preference Dataset and Benchmark for Trustworthy Reward Modeling in Open-Ended, Long-Context Generation

Hanning Zhang, Juntong Song, Juno Zhu, Yuanhao Wu, Tong Zhang, Cheng Niu

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； NewsBreak

专题命中偏好对齐：trustworthy(title)；safety(abstract)；分类 cs.CL

Comments Preprint update

2503.23777 2025-11-12 cs.CL 79%

CONGRAD:Conflicting Gradient Filtering for Multilingual Preference Alignment

Jiangnan Li, Thuy-Trang Vu, Christian Herold, Amirhossein Tebbifakhr, Shahram Khadivi, Gholamreza Haffari

机构 * Department of Data Science and AI, Monash University, Australia（墨尔本大学数据科学与人工智能系）； eBay Inc.（eBay公司）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

2511.04880 2025-11-10 cs.AI 79%

DMA: Online RAG Alignment with Human Feedback

Yu Bai, Yukai Miao, Dawei Wang, Li Chen, Fei Long, Rundi Zhai, Dan Li, Yanyu Ren, Tianfeng Liu, Hongtao Xie, Ce Yang, Xuhui Cai

机构 * Zhongguancun Laboratory（中关村实验室）； Tsinghua University（清华大学）； China Mobile Communications Group Co., Ltd.（中国移动通信集团有限公司）； Beijing University of Posts and Telecommunications（北京邮电大学）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.AI

2506.04721 2025-11-04 cs.CL 79%

SPARTA ALIGNMENT: Collectively Aligning Multiple Language Models through Combat

Yuru Jiang, Wenxuan Ding, Shangbin Feng, Greg Durrett, Yulia Tsvetkov

机构 * Zhejiang University（浙江大学）； New York University（纽约大学）； University of Washington（华盛顿大学）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments NeurIPS 2025

2502.07193 2025-10-28 cs.LG stat.ML 79%

Provably Efficient Online RLHF with One-Pass Reward Modeling

Long-Fei Li, Yu-Yang Qian, Peng Zhao, Zhi-Hua Zhou

机构 * National Key Laboratory for Novel Software Technology, Nanjing University, China（国家新型软件技术实验室，南京大学）； School of Artificial Intelligence, Nanjing University, China（人工智能学院，南京大学）

专题命中偏好对齐：RLHF(title,abstract)；分类 cs.LG

Comments NeurIPS 2025; The first two authors contributed equally

2412.20984 2025-10-24 cs.LG 79%

Pareto-Optimal Energy Alignment for Designing Nature-Like Antibodies

Yibo Wen, Chenwei Xu, Jerry Yao-Chieh Hu, Kaize Ding, Han Liu

机构 * Northwestern University（西北大学）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

Comments 21 pages

2504.05831 2025-10-21 cs.CL 79%

Leveraging Robust Optimization for LLM Alignment under Distribution Shifts

Mingye Zhu, Yi Liu, Zheren Fu, Yongdong Zhang, Zhendong Mao

机构 * University of Science and Technology of China（中国科学技术大学）； State Key Laboratory of Communication Content Cognition（通信内容认知国家重点实验室）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments NeurIPS 2025

2510.06871 2025-10-10 cs.LG cs.CV 79%

SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Models

Huahui Yi, Kun Wang, Qiankun Li, Miao Yu, Liang Lin, Gongli Xi, Hao Wu, Xuming Hu, Kang Li, Yang Liu

机构 * West China Biomedical Big Data Center, West China Hospital, SCU（西昌生物医学大数据中心、西昌医院、SCU）； NTU ； USTC ； TeleAI, China Telecom（TeleAI、中国电信）； BUPT ； Tsinghua University（清华大学）； HKUST(Guangzhou)（HKUST（广州））

专题命中偏好对齐：safety(title,abstract)；分类 cs.LG

2411.04712 2025-10-07 cs.CV cs.LG 79%

SEE-DPO: Self Entropy Enhanced Direct Preference Optimization

Shivanshu Shekhar, Shreyas Singh, Tong Zhang

机构 * Siebel School of Computing and Data Science（计算与数据科学学院）； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； Fractal AI Research（Fractal AI研究院）

专题命中偏好对齐：DPO(title,abstract)；分类 cs.LG

2507.07510 2025-10-07 cs.CV cs.LG 79%

Divergence Minimization Preference Optimization for Diffusion Model Alignment

Binxu Li, Minkai Xu, Jiaqi Han, Meihua Dang, Stefano Ermon

机构 * Stanford University（斯坦福大学）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

2510.00647 2025-10-02 cs.CL 79%

MCM-DPO: Multifaceted Cross-Modal Direct Preference Optimization for Alt-text Generation

Jinlan Fu, Shenzhen Huangfu, Hao Fei, Yichong Huang, Xiaoyu Shen, Xipeng Qiu, See-Kiong Ng

机构 * National University of Singapore（新加坡国立大学）； Fudan University（复旦大学）； Harbin Institute of Technology（哈尔滨工业大学）； Eastern Institute of Technology（东方技术研究所）

专题命中偏好对齐：DPO(title,abstract)；分类 cs.CL

Comments Accepted by ACM MM 2025

2509.23667 2025-09-30 cs.LG 79%

Why Alignment Must Precede Distillation: A Minimal Working Explanation

Sungmin Cha, Kyunghyun Cho

机构 * New York University（纽约大学）； Genentech（基因泰克）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

Comments Preprint

2501.04561 2025-09-24 cs.CL cs.CV 79%

OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis

Run Luo, Ting-En Lin, Haonan Zhang, Yuchuan Wu, Xiong Liu, Min Yang, Yongbin Li, Longze Chen, Jiaming Li, Lei Zhang, Xiaobo Xia, Hamid Alinejad-Rokny, Fei Huang

机构 * Shenzhen Key Laboratory for High Performance Data Mining（深圳高性能数据挖掘重点实验室）； Shenzhen Institute of Advanced Technology（深圳先进技术研究院）； Chinese Academy of Sciences（中国科学院）； University of Chinese Academy of Sciences（中国科学院大学）； Tongyi Laboratory（通义实验室）； University of New South Wales（新南威尔士大学）； National University of Singapore（新加坡国立大学）； University of Science and Technology of China（中国科学技术大学）； MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition（脑启发智能感知与认知重点实验室）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

2509.04713 2025-09-08 cs.LG 79%

Natural Spectral Fusion: p-Exponent Cyclic Scheduling and Early Decision-Boundary Alignment in First-Order Optimization

Gongyue Zhang, Honghai Liu

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

2505.02666 2025-09-03 cs.CL 79%

A Survey on Progress in LLM Alignment from the Perspective of Reward Design

Miaomiao Ji, Yanqiu Wu, Zhibin Wu, Shoujin Wang, Jian Yang, Mark Dras, Usman Naseem

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments Preprint

2508.19532 2025-08-28 cs.CL 79%

Alignment with Fill-In-the-Middle for Enhancing Code Generation

Houxing Ren, Zimu Lu, Weikang Shi, Haotian Hou, Yunqiao Yang, Ke Wang, Aojun Zhou, Junting Pan, Mingjie Zhan, Hongsheng Li

机构 * CUHK MMLab（CUHK语音实验室）； SenseTime Research（商汤科技研究院）； CPII under InnoHK（创新香港下的CPII）； Shanghai AI Laboratory（上海人工智能实验室）； Beihang University（北航）

专题命中偏好对齐：alignment(title)；DPO(abstract)；分类 cs.CL

Comments Accepted to EMNLP 2025 (main conference)

2508.17718 2025-08-26 cs.CV cs.AI 79%

Instant Preference Alignment for Text-to-Image Diffusion Models

Yang Li, Songlin Yang, Xiaoxuan Han, Wei Wang, Jing Dong, Yueming Lyu, Ziyu Xue

机构 * New Laboratory of Pattern Recognition, CASIA（模式识别新实验室，中国科学院自动化研究所）； The Hong Kong University of Science and Technology（香港科技大学）； Nanjing university（南京大学）； Academy of Broadcasting Science, NRTA（广播科学研究院，国家广播电视总局）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.AI

Comments 17 figures

2508.08550 2025-08-13 cs.SD cs.CL 79%

Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization

Chaoqun Cui, Liangbin Huang, Shijing Wang, Zhe Tong, Zhaolong Huang, Xiao Zeng, Xiaofeng Liu

机构 * Alibaba Digital Media and Entertainment Group（阿里巴巴数字媒体与娱乐集团）； School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件学院）； Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University（北京交通大学交通数据挖掘与具身智能重点实验室）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments This paper is accepted by ACL2025 (Main)

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025: 4524-4546

2508.04963 2025-08-08 cs.IR cs.LG 79%

A Metric for MLLM Alignment in Large-scale Recommendation

Yubin Zhang, Yanhua Huang, Haiming Xu, Mingliang Qi, Chang Wang, Jiarui Jin, Xiangyuan Ren, Xiaodan Wang, Ruiwen Xu

机构 * Xiaohongshu Inc.（小红书公司）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

Comments Pre-print.Under Review

2508.04698 2025-08-07 cs.CL 79%

FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data

Thibaut Thonet, Germán Kruszewski, Jos Rozen, Pierre Erbacher, Marc Dymetman

机构 * NAVER Labs Europe（NAVER欧洲实验室）； Independent Researcher（独立研究者）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

2503.12854 2025-07-29 cs.CL 79%

Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation

Songjun Tu, Jiahao Lin, Xiangyu Tian, Qichao Zhang, Linjing Li, Yuqian Fu, Nan Xu, Wei He, Xiangyuan Lan, Dongmei Jiang, Dongbin Zhao

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）； Pengcheng Laboratory（鹏城实验室）； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）； Wenge Technology（文生科技）； Fudan University（复旦大学）

专题命中偏好对齐：DPO(title,abstract)；分类 cs.CL

Comments 23pages

Journal ref COLM2025

2507.07939 2025-07-23 cs.CL 79%

SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment

Guoxin Zang, Xue Li, Donglin Di, Lanshun Nie, Dechen Zhan, Yang Song, Lei Fan

机构 * Harbin Institute of Technology（哈尔滨工业大学）； University of New South Wales（新南威尔士大学）

专题命中偏好对齐：alignment(title)；DPO(abstract)；分类 cs.CL

Comments Accepted by ACMMM2025

2507.09016 2025-07-17 cs.LG 79%

Enhancing RLHF with Human Gaze Modeling

Karim Galliamov, Ivan Titov, Ilya Pershin

专题命中偏好对齐：RLHF(title,abstract)；分类 cs.LG

2410.08193 2025-07-16 cs.CL 79%

GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment

Yuancheng Xu, Udari Madhushani Sehwag, Alec Koppel, Sicheng Zhu, Bang An, Furong Huang, Sumitra Ganesh

专题命中偏好对齐：alignment(title,abstract)；分类 cs.CL

Comments Published at the Thirteenth International Conference on Learning Representations (ICLR 2025)

2504.10208 2025-07-08 cs.IR cs.LG 79%

From Prompting to Alignment: A Generative Framework for Query Recommendation

Erxue Min, Hsiu-Yuan Huang, Xihong Yang, Min Yang, Xin Jia, Yunfang Wu, Hengyi Cai, Junfeng Wang, Shuaiqiang Wang, Dawei Yin

机构 * Baidu Inc.（百度公司）； National Key Laboratory for Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）； Chinese Academy of Sciences（中国科学院）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.LG

2507.02173 2025-07-04 cs.AI 79%

Data Diversification Methods In Alignment Enhance Math Performance In LLMs

Berkan Dokmeci, Qingyang Wu, Ben Athiwaratkun, Ce Zhang, Shuaiwen Leon Song, James Zou

机构 * Washington University in St. Louis（华盛顿大学圣路易斯分校）； Together AI ； University of Chicago（芝加哥大学）； Stanford University（斯坦福大学）

专题命中偏好对齐：alignment(title,abstract)；分类 cs.AI

1. 偏好对齐 3235 篇

g-DPO: Scalable Preference Optimization for Protein Language Models

Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis

Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models

UDA: Unsupervised Debiasing Alignment for Pair-wise LLM-as-a-Judge

OpenGenAlign: A Preference Dataset and Benchmark for Trustworthy Reward Modeling in Open-Ended, Long-Context Generation

CONGRAD:Conflicting Gradient Filtering for Multilingual Preference Alignment

DMA: Online RAG Alignment with Human Feedback

SPARTA ALIGNMENT: Collectively Aligning Multiple Language Models through Combat

Provably Efficient Online RLHF with One-Pass Reward Modeling

Pareto-Optimal Energy Alignment for Designing Nature-Like Antibodies

Leveraging Robust Optimization for LLM Alignment under Distribution Shifts

SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Models

SEE-DPO: Self Entropy Enhanced Direct Preference Optimization

Divergence Minimization Preference Optimization for Diffusion Model Alignment

MCM-DPO: Multifaceted Cross-Modal Direct Preference Optimization for Alt-text Generation

Why Alignment Must Precede Distillation: A Minimal Working Explanation

OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis

Natural Spectral Fusion: p-Exponent Cyclic Scheduling and Early Decision-Boundary Alignment in First-Order Optimization

A Survey on Progress in LLM Alignment from the Perspective of Reward Design

Alignment with Fill-In-the-Middle for Enhancing Code Generation

Instant Preference Alignment for Text-to-Image Diffusion Models

Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization

A Metric for MLLM Alignment in Large-scale Recommendation

FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data

Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation

SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment

Enhancing RLHF with Human Gaze Modeling

GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment

From Prompting to Alignment: A Generative Framework for Query Recommendation

Data Diversification Methods In Alignment Enhance Math Performance In LLMs