大模型对齐与安全

Adam-mini: Use Fewer Learning Rates To Gain More

Yushun Zhang, Congliang Chen, Ziniu Li, Tian Ding, Chenwei Wu, Diederik P. Kingma, Yinyu Ye, Zhi-Quan Luo, Ruoyu Sun

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.AI、cs.LG

Direct Multi-Turn Preference Optimization for Language Agents

Wentao Shi, Mengqi Yuan, Junkang Wu, Qifan Wang, Fuli Feng

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.LG

Comments Accepted by EMNLP 2024 Main

On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback

Marcus Williams, Micah Carroll, Adhyyan Narang, Constantin Weisser, Brendan Murphy, Anca Dragan

专题命中 偏好对齐 ：safety(abstract)；分类 cs.AI、cs.LG

Comments Accepted to ICLR 2025

LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models

Shangqing Tu, Yucheng Wang, Daniel Zhang-Li, Yushi Bai, Jifan Yu, Yuhao Wu, Lei Hou, Huiqin Liu, Zhiyuan Liu, Bin Xu, Juanzi Li

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models

Yanggan Gu, Junzhuo Li, Sirui Huang, Xin Zou, Zhenghua Li, Xuming Hu

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments Under review

Federated Fine-Tuning of Large Language Models: Kahneman-Tversky vs. Direct Preference Optimization

Fernando Spadea, Oshani Seneviratne

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.LG

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System

Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, Maosong Sun

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Comments Under review

Process Reward Models for LLM Agents: Practical Framework and Directions

Sanjiban Choudhury

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.AI、cs.LG

Comments 17 pages, 7 figures

Syntriever: How to Train Your Retriever with Synthetic Data from LLMs

Minsang Kim, Seungjun Baek

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), Findings, Accepted

LLMs Can Teach Themselves to Better Predict the Future

Benjamin Turtel, Danny Franklin, Philipp Schoenegger

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

SimulPL: Aligning Human Preferences in Simultaneous Machine Translation

Donglei Yu, Yang Zhao, Jie Zhu, Yangyifan Xu, Yu Zhou, Chengqing Zong

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments Accepted to ICLR 2025. 23 pages,13 figures,11 tables

PORT: Preference Optimization on Reasoning Traces

Salem Lahlou, Abdalgader Abubaker, Hakim Hacid

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.LG

Dr. SoW: Density Ratio of Strong-over-weak LLMs for Reducing the Cost of Human Annotation in Preference Tuning

Guangxuan Xu, Kai Xu, Shivchander Sudalairaj, Hao Wang, Akash Srivastava

专题命中 偏好对齐 ：safety(abstract)；分类 cs.CL、cs.AI

Atla Selene Mini: A General Purpose Evaluation Model

Andrei Alexandru, Antonia Calvi, Henry Broomfield, Jackson Golden, Kyle Dai, Mathias Leys, Maurice Burger, Max Bartolo, Roman Engeler, Sashank Pisupati, Toby Drane, Young Sun Park

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning

Nirav Diwan, Tolga Ergen, Dongsub Shim, Honglak Lee

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.AI、cs.LG

Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback

Yen-Ting Lin, Di Jin, Tengyu Xu, Tianhao Wu, Sainbayar Sukhbaatar, Chen Zhu, Yun He, Yun-Nung Chen, Jason Weston, Yuandong Tian, Arash Rahnama, Sinong Wang, Hao Ma, Han Fang

专题命中 偏好对齐 ：trustworthy(abstract)；分类 cs.AI、cs.LG

Can LLMs Identify Gaps and Misconceptions in Students' Code Explanations?

Priti Oli, Rabin Banjade, Andrew M. Olney, Vasile Rus

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.AI、cs.CY

Two-Layer Retrieval-Augmented Generation Framework for Low-Resource Medical Question Answering Using Reddit Data: Proof-of-Concept Study

Sudeshna Das, Yao Ge, Yuting Guo, Swati Rajwal, JaMor Hairston, Jeanne Powell, Drew Walker, Snigdha Peddireddy, Sahithi Lakamana, Selen Bozkurt, Matthew Reyna, Reza Sameni, Yunyu Xiao, Sangmi Kim, Rasheeta Chandler, Natalie Hernandez, Danielle Mowery, Rachel Wightman, Jennifer Love, Anthony Spadaro, Jeanmarie Perrone, Abeed Sarker

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Comments Published in JMIR: https://www.jmir.org/2025/1/e66220

HuRef: HUman-REadable Fingerprint for Large Language Models

Boyi Zeng, Lizheng Wang, Yuncong Hu, Yi Xu, Chenghu Zhou, Xinbing Wang, Yu Yu, Zhouhan Lin

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024

Speech Recognition With LLMs Adapted to Disordered Speech Using Reinforcement Learning

Chirag Nagpal, Subhashini Venugopalan, Jimmy Tobin, Marilyn Ladewig, Katherine Heller, Katrin Tomanek

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.LG

Comments Accepted at ICASSP 2025

CareBot: A Pioneering Full-Process Open-Source Medical Language Model

Lulu Zhao, Weihao Zeng, Xiaofeng Shi, Hua Zhou

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.AI

Comments Accept by AAAI 2025

A Systematic Examination of Preference Learning through the Lens of Instruction-Following

Joongwon Kim, Anirudh Goyal, Aston Zhang, Bo Xiong, Rui Hou, Melanie Kambadur, Dhruv Mahajan, Hannaneh Hajishirzi, Liang Tan

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments 23 pages

Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimization

Thomas Savage, Stephen Ma, Abdessalem Boukil, Vishwesh Patel, Ekanath Rangan, Ivan Lopez, Jonathan H Chen

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Enhancing LLMs for Physics Problem-Solving using Reinforcement Learning with Human-AI Feedback

Avinash Anand, Kritarth Prasad, Chhavi Kirtani, Ashwin R Nair, Mohit Gupta, Saloni Garg, Anurag Gautam, Snehal Buldeo, Rajiv Ratn Shah

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.AI、cs.LG

REBEL: Reinforcement Learning via Regressing Relative Rewards

Zhaolin Gao, Jonathan D. Chang, Wenhao Zhan, Owen Oertell, Gokul Swamy, Kianté Brantley, Thorsten Joachims, J. Andrew Bagnell, Jason D. Lee, Wen Sun

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.LG

Comments New experimental results on general chat

Thai Financial Domain Adaptation of THaLLE -- Technical Report

KBTG Labs, Atthakorn Petchsod, Pornchanan Balee, Danupat Khamnuansin, Anuruth Lertpiya, Chanatip Saetia, Tawunrat Chalothorn, Thadpong Pongthawornkamol, Monchai Lertsutthiwong

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI

Video-Text Dataset Construction from Multi-AI Feedback: Promoting Weak-to-Strong Preference Learning for Video Large Language Models

Hao Yi, Qingyang Li, Yulan Hu, Fuzheng Zhang, Di Zhang, Yong Liu

专题命中 偏好对齐 ：alignment(abstract)；分类 cs.CL、cs.LG

LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning

Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, Wanli Ouyang, Dongzhan Zhou

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.CL、cs.AI

DLPO: Diffusion Model Loss-Guided Reinforcement Learning for Fine-Tuning Text-to-Speech Diffusion Models

Jingyi Chen, Ju-Seung Byun, Micha Elsner, Andrew Perrault

专题命中 偏好对齐 ：RLHF(abstract)；分类 cs.AI、cs.LG

Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization

Yuhan Fu, Ruobing Xie, Xingwu Sun, Zhanhui Kang, Xirong Li

专题命中 偏好对齐 ：DPO(abstract)；分类 cs.CL、cs.AI