语言大模型 / LLM - arXivDaily 专题

2511.06722 2025-11-11 cs.CV cs.AI cs.CL 88%

Revisiting the Data Sampling in Multimodal Post-training from a Difficulty-Distinguish View

Jianyu Qi, Ding Zou, Wenrui Yan, Rui Ma, Jiaxu Li, Zhijie Zheng, Zhiguo Yang, Rongchang Zhao

专题命中后训练与偏好优化：post-training(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Comments Accpeted by AAAI 2026

2503.03710 2025-10-31 cs.CL cs.CR cs.LG 88%

Improving LLM Safety Alignment with Dual-Objective Optimization

Xuandong Zhao, Will Cai, Tianneng Shi, David Huang, Licong Lin, Song Mei, Dawn Song

机构 * University of California, Berkeley（加州大学伯克利分校）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments ICML 2025

2510.24700 2025-10-29 cs.LG cs.AI cs.IT math.IT stat.ML 88%

Greedy Sampling Is Provably Efficient for RLHF

Di Wu, Chengshuai Shi, Jing Yang, Cong Shen

机构 * Electrical and Computer Engineering University of Virginia（电气与计算机工程大学弗吉尼亚大学）； Princeton Language and Intelligence Princeton University（普林斯顿语言与智能普林斯顿大学）

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Comments NeurIPS 2025

2506.22578 2025-10-14 cs.LG cs.AI stat.ML 88%

The Hidden Link Between RLHF and Contrastive Learning

Xufei Lv, Kehai Chen, Haoyuan Sun, Xuefeng Bai, Min Zhang, Houde Liu, Kehai Chen

机构 * Tsinghua University（清华大学）； Harbin Institute of Technology（哈尔滨工业大学）； Soochow University（苏州大学）

专题命中后训练与偏好优化：RLHF(title,abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

2505.16567 2025-10-10 cs.LG cs.AI cs.CR 88%

Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning

Thibaud Gloaguen, Mark Vero, Robin Staab, Martin Vechev

机构 * ETH Zurich（苏黎世联邦理工学院）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

2510.00194 2025-10-02 cs.LG cs.AI 88%

GRPO-$λ$: Credit Assignment improves LLM Reasoning

Prasanna Parthasarathi, Mathieu Reymond, Boxing Chen, Yufei Cui, Sarath Chandar

机构 * Noah’s Ark Lab, Huawei Technologies Ltd.（华为技术有限公司诺亚实验室）； Chandar Research Lab（昌达研究实验室）； Mila – Quebec AI Institute（魁北克AI研究院）； Université de Montréal（蒙特利尔大学）； Polytechnique Montréal Canada（蒙特利尔理工学院加拿大）； CIFAR AI Chair（CIFAR人工智能主席）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；post-training(abstract)

2509.21528 2025-09-29 cs.LG cs.AI 88%

Preemptive Detection and Steering of LLM Misalignment via Latent Reachability

Sathwik Karnik, Somil Bansal

机构 * Safe and Intelligent Autonomy Lab, Stanford University（斯坦福大学安全与智能自主实验室）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

2508.07137 2025-09-26 cs.LG cs.AI 88%

A Principled Loss Function for Direct Language Model Alignment

Yuandong Tan

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；preference optimization(abstract)

2509.16475 2025-09-23 cs.LG cs.CL 88%

Towards Universal Debiasing for Language Models-based Tabular Data Generation

Tianchun Li, Tianci Liu, Xingchen Wang, Rongzhe Wei, Pan Li, Lu Su, Jing Gao

机构 * Purdue University（普渡大学）； Georgia Institute of Technology（佐治亚理工学院）

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；large language model(abstract)；preference optimization(abstract)

Comments EMNLP 2025 Findings

2505.14442 2025-09-22 cs.CL cs.AI 88%

Creative Preference Optimization

Mete Ismayilzada, Antonio Laverghetta, Simone A. Luchini, Reet Patel, Antoine Bosselut, Lonneke van der Plas, Roger Beaty

机构 * EPFL（苏黎世联邦理工学院）； Università della Svizzera Italiana（瑞士联邦理工学院）； Wesleyan University（韦斯利安大学）； Pennsylvania State University（宾夕法尼亚州立大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted to EMNLP 2025 Findings

2507.20181 2025-07-29 cs.CL cs.AI 88%

SGPO: Self-Generated Preference Optimization based on Self-Improver

Hyeonji Lee, Daejin Jo, Seohwan Yun, Sungwoong Kim

机构 * Korea University（韩国大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

2502.03699 2025-07-25 cs.CL cs.AI cs.IR 88%

LLM Alignment as Retriever Optimization: An Information Retrieval Perspective

Bowen Jin, Jinsung Yoon, Zhen Qin, Ziqi Wang, Wei Xiong, Yu Meng, Jiawei Han, Sercan O. Arik

机构 * University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； Google Cloud AI Research（谷歌云人工智能研究）； Google DeepMind（谷歌DeepMind）； University of Virginia（弗吉尼亚大学）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments 26 pages

2411.06208 2025-07-18 cs.CL cs.AI 88%

IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization

Xinghua Zhang, Haiyang Yu, Cheng Fu, Fei Huang, Yongbin Li

机构 * Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团）

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；SFT(abstract)

Comments ACL 2025

2504.05352 2025-07-01 cs.LG cs.AI 88%

Achieving binary weight and activation for LLMs using Post-Training Quantization

Siqing Song, Chuang Wang, Ruiqi Wang, Yi Yang, Xu-Yao Zhang

机构 * MAIS, Institute of Automation, Chinese Academy of Sciences（自动化研究所，中国科学院）； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）； Central Media Technology Institute, Huawei Technologies Ltd.（华为技术有限公司媒体技术研究所）

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2506.04344 2025-06-06 cs.CL cs.LG 88%

GEM: Empowering LLM for both Embedding Generation and Language Understanding

Caojin Zhang, Qiang Zhang, Ke Li, Sai Vidyaranya Nuthalapati, Benyu Zhang, Jason Liu, Serena Li, Lizhu Zhang, Xiangjun Fan

机构 * Meta Inc（Meta公司）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

2505.24523 2025-06-02 cs.CL cs.AI 88%

Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors

Andrea Pedrotti, Michele Papucci, Cristiano Ciaccio, Alessio Miaschi, Giovanni Puccetti, Felice Dell'Orletta, Andrea Esuli

机构 * Department of Computer Science, University of Pisa（比萨大学计算机科学系）

专题命中后训练与偏好优化：language model(title,abstract)；LLM(abstract)；large language model(abstract)；preference optimization(abstract)

Comments Accepted at Findings of ACL 2025

2505.23788 2025-06-02 cs.CL cs.AI 88%

Nine Ways to Break Copyright Law and Why Our LLM Won't: A Fair Use Aligned Generation Framework

Aakash Sen Sharma, Debdeep Sanyal, Priyansh Srivastava, Sundar Atreya H., Shirish Karande, Mohan Kankanhalli, Murari Mandal

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments 30 Pages

2501.18511 2025-05-26 cs.LG cs.CL 88%

WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training

Benjamin Feuer, Chinmay Hegde

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；language model(abstract)；SFT(abstract)

Comments ICML 2025

2505.08849 2025-05-15 cs.CR cs.AI cs.LG 88%

Improved Algorithms for Differentially Private Language Model Alignment

Keyu Chen, Hao Tang, Qinglin Liu, Yizhao Xu

机构 * Peking University（北京大学）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；RLHF(abstract)；preference optimization(abstract)

2410.01639 2025-05-13 cs.LG cs.AI cs.CY 88%

Moral Alignment for LLM Agents

Elizaveta Tennant, Stephen Hailes, Mirco Musolesi

机构 * University College London（伦敦大学学院）； University of Bologna（博洛尼亚大学）

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments Published at the 13th International Conference on Learning Representations (ICLR'25), Singapore, Apr 2025. https://openreview.net/forum?id=MeGDmZjUXy

2411.00062 2025-04-11 cs.CL cs.AI physics.data-an stat.ML 88%

Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Play

Ziyu Ye, Rishabh Agarwal, Tianqi Liu, Rishabh Joshi, Sarmishta Velury, Quoc V. Le, Qijun Tan, Yuan Liu

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments spotlight @ neurips language gamification workshop. updated the problem description and added new online RL experiments in this version

2407.03038 2025-04-10 cs.CL cs.DC cs.LG 88%

Towards Federated RLHF with Aggregated Client Preference for LLMs

Feijie Wu, Xiaoze Liu, Haoyu Wang, Xingchen Wang, Lu Su, Jing Gao

专题命中后训练与偏好优化：RLHF(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments ICLR'25

2411.02442 2025-04-01 cs.CL cs.AI cs.IR 88%

TODO: Enhancing LLM Alignment with Ternary Preferences

Yuxiang Guo, Lu Yin, Bo Jiang, Jiaqi Zhang

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments Accepted to ICLR 2025

2503.21720 2025-03-28 cs.CL cs.AI 88%

Collab: Controlled Decoding using Mixture of Agents for LLM Alignment

Souradip Chakraborty, Sujay Bhatt, Udari Madhushani Sehwag, Soumya Suvra Ghosal, Jiahao Qiu, Mengdi Wang, Dinesh Manocha, Furong Huang, Alec Koppel, Sumitra Ganesh

专题命中后训练与偏好优化：LLM(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments Accepted to ICLR 2025

2410.07672 2025-03-04 cs.CL cs.AI 88%

MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization

Yougang Lyu, Lingyong Yan, Zihan Wang, Dawei Yin, Pengjie Ren, Maarten de Rijke, Zhaochun Ren

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments ICLR 2025

2503.00018 2025-03-04 cs.CL cs.AI 88%

Eeyore: Realistic Depression Simulation via Supervised and Preference Optimization

Siyang Liu, Bianca Brie, Wenda Li, Laura Biester, Andrew Lee, James Pennebaker, Rada Mihalcea

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

2501.01821 2025-02-28 cs.AI cs.CL 88%

SDPO: Segment-Level Direct Preference Optimization for Social Agents

Aobo Kong, Wentao Ma, Shiwan Zhao, Yongbin Li, Yuchuan Wu, Ke Wang, Xiaoqian Liu, Qicheng Li, Yong Qin, Fei Huang

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2502.18532 2025-02-27 cs.AI cs.LG 88%

CuDIP: Enhancing Theorem Proving in LLMs via Curriculum Learning-based Direct Preference Optimization

Shuming Shi, Ruobing Zuo, Gaolei He, Jianlin Wang, Chenyang Xu, Zhengfeng Yang

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

2502.10993 2025-02-18 cs.CL cs.LG 88%

RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization

Tianci Liu, Haoxiang Jiang, Tianze Wang, Ran Xu, Yue Yu, Linjun Zhang, Tuo Zhao, Haoyu Wang

专题命中后训练与偏好优化：preference optimization(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

2502.00203 2025-02-11 cs.LG cs.CL 88%

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment

Shengyang Sun, Yian Zhang, Alexander Bukharin, David Mosallanezhad, Jiaqi Zeng, Soumye Singhal, Gerald Shen, Adithya Renduchintala, Tugrul Konuk, Yi Dong, Zhilin Wang, Dmitry Chichkov, Olivier Delalleau, Oleksii Kuchaiev

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 8 pages, 4 figures; update author names

1. 后训练与偏好优化 4516 篇

Revisiting the Data Sampling in Multimodal Post-training from a Difficulty-Distinguish View

Improving LLM Safety Alignment with Dual-Objective Optimization

Greedy Sampling Is Provably Efficient for RLHF

The Hidden Link Between RLHF and Contrastive Learning

Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning

GRPO-$λ$: Credit Assignment improves LLM Reasoning

Preemptive Detection and Steering of LLM Misalignment via Latent Reachability

A Principled Loss Function for Direct Language Model Alignment

Towards Universal Debiasing for Language Models-based Tabular Data Generation

Creative Preference Optimization

SGPO: Self-Generated Preference Optimization based on Self-Improver

LLM Alignment as Retriever Optimization: An Information Retrieval Perspective

IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization

Achieving binary weight and activation for LLMs using Post-Training Quantization

GEM: Empowering LLM for both Embedding Generation and Language Understanding

Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors

Nine Ways to Break Copyright Law and Why Our LLM Won't: A Fair Use Aligned Generation Framework

WILDCHAT-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training

Improved Algorithms for Differentially Private Language Model Alignment

Moral Alignment for LLM Agents

Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Play

Towards Federated RLHF with Aggregated Client Preference for LLMs

TODO: Enhancing LLM Alignment with Ternary Preferences

Collab: Controlled Decoding using Mixture of Agents for LLM Alignment

MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization

Eeyore: Realistic Depression Simulation via Supervised and Preference Optimization

SDPO: Segment-Level Direct Preference Optimization for Social Agents

CuDIP: Enhancing Theorem Proving in LLMs via Curriculum Learning-based Direct Preference Optimization

RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment