语言大模型 / LLM - arXivDaily 专题

2509.16399 2025-09-23 cs.AI 83%

VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shaping

Guojun Xiong, Milind Tambe

机构 * Department of Computer Science, Harvard University（计算机科学系，哈佛大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 28pages, 19figures

2509.09629 2025-09-12 cs.CL 83%

Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems

Minghang Zhu, Zhengliang Shi, Zhiwei Xu, Shiguang Wu, Lingjie Wang, Pengjie Ren, Zhaochun Ren, Zhumin Chen

机构 * Shandong University（山东大学）； Leiden University（莱顿大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2025 Findings

2508.08550 2025-08-13 cs.SD cs.CL 83%

Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization

Chaoqun Cui, Liangbin Huang, Shijing Wang, Zhe Tong, Zhaolong Huang, Xiao Zeng, Xiaofeng Liu

机构 * Alibaba Digital Media and Entertainment Group（阿里巴巴数字媒体与娱乐集团）； School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件学院）； Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University（北京交通大学交通数据挖掘与具身智能重点实验室）

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；分类 cs.CL

Comments This paper is accepted by ACL2025 (Main)

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025: 4524-4546

2507.11554 2025-08-05 cs.CV cs.AI 83%

Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models

Zejian Li, Yize Li, Chenye Meng, Zhongni Liu, Yang Ling, Shengyuan Zhang, Guang Yang, Changyuan Yang, Zhiyuan Yang, Lingyun Sun

机构 * Zhejiang University（浙江大学）； University of Electronic Science and Technology of China（电子科技大学）； Peking University（北京大学）； Alibaba Group（阿里巴巴集团）

专题命中后训练与偏好优化：post-training(title,abstract)；preference optimization(abstract)；分类 cs.AI

Comments Accepted by ACM MM25

2507.07939 2025-07-23 cs.CL 83%

SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment

Guoxin Zang, Xue Li, Donglin Di, Lanshun Nie, Dechen Zhan, Yang Song, Lei Fan

机构 * Harbin Institute of Technology（哈尔滨工业大学）； University of New South Wales（新南威尔士大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted by ACMMM2025

2507.09016 2025-07-17 cs.LG 83%

Enhancing RLHF with Human Gaze Modeling

Karim Galliamov, Ivan Titov, Ilya Pershin

专题命中后训练与偏好优化：RLHF(title,abstract)；language model(abstract)；分类 cs.LG

2507.10923 2025-07-16 cs.AI 83%

Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization

Yuhao Wang, Keyan Ding, Kehua Feng, Zeyuan Wang, Ming Qin, Xiaotong Li, Qiang Zhang, Huajun Chen

机构 * Zhejiang University（浙江大学）； ZJU-Hangzhou Global Scientific and Technological Innovation Center（浙大杭州国际科创中心）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.AI

Comments Accepted at ACL 2025 (Main Conference)

2507.05911 2025-07-09 cs.SD cs.AI eess.AS 83%

Differentiable Reward Optimization for LLM based TTS system

Changfeng Gao, Zhihao Du, Shiliang Zhang

机构 * Speech Team, Tongyi Lab, Alibaba Group（通义实验室，阿里巴巴集团）

专题命中后训练与偏好优化：LLM(title)；language model(abstract)；RLHF(abstract)；分类 cs.AI

2506.21497 2025-06-27 cs.CL 83%

Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments

Jiashuo Wang, Kaitao Song, Chunpu Xu, Changhe Song, Yang Xiao, Dongsheng Li, Lili Qiu, Wenjie Li

机构 * Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）； Microsoft Research Asia（微软亚洲研究院）

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(abstract)；分类 cs.CL

2411.04291 2025-06-23 cs.CL cs.CV 83%

Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Models

Saketh Bachu, Erfan Shayegani, Rohit Lal, Trishna Chakraborty, Arindam Dutta, Chengyu Song, Yue Dong, Nael Abu-Ghazaleh, Amit K. Roy-Chowdhury

专题命中后训练与偏好优化：language model(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments Accepted by ICML 2025 as a spotlight poster

2502.07599 2025-06-09 cs.CL 83%

DPO-Shift: Shifting the Distribution of Direct Preference Optimization

Xiliang Yang, Feng Jiang, Qianen Zhang, Lei Zhao, Xiao Li

机构 * David S. Hippocampus Department of Computer Science（计算机科学系）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2506.02708 2025-06-04 cs.CV cs.CL 83%

Iterative Self-Improvement of Vision Language Models for Image Scoring and Self-Explanation

Naoto Tanji, Toshihiko Yamasaki

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted to ICIP2025

2506.02211 2025-06-04 cs.AI 83%

Improving LLM-Generated Code Quality with GRPO

Maxime Robeyns, Laurence Aitchison

机构 * University of Bristol（布里斯托大学）

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

2412.17018 2025-06-04 cs.AI 83%

GAS: Generative Auto-bidding with Post-training Search

Yewen Li, Shuai Mao, Jingtong Gao, Nan Jiang, Yunjian Xu, Qingpeng Cai, Fei Pan, Peng Jiang, Bo An

机构 * Nanyang Technological University（南洋理工大学）； The Chinese University of Hong Kong（香港中文大学）； City University of Hong Kong（香港城市大学）； Kuaishou Technology（快手科技）

专题命中后训练与偏好优化：post-training(title,abstract)；foundation model(abstract)；分类 cs.AI

2505.19095 2025-05-27 cs.AI 83%

ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World

Runliang Niu, Jinglong Ji, Yi Chang, Qi Wang

机构 * School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.AI

2505.15948 2025-05-23 cs.CL cs.DL cs.SI 83%

Citation Parsing and Analysis with Language Models

Parth Sarin, Juan Pablo Alperin

专题命中后训练与偏好优化：language model(title,abstract)；post-training(abstract)；分类 cs.CL

Comments Presented at the Workshop on Open Citations & Open Scholarly Metadata 2025

2412.20299 2025-05-14 cs.CL 83%

No Preference Left Behind: Group Distributional Preference Optimization

Binwei Yao, Zefan Cai, Yun-Shiuan Chuang, Shanglin Yang, Ming Jiang, Diyi Yang, Junjie Hu

机构 * University of Wisconsin-Madison（威斯康星大学麦迪逊分校）； Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校）； Stanford University（斯坦福大学）

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2505.06548 2025-05-13 cs.CL 83%

REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback

Aniruddha Roy, Pretam Ray, Abhilash Nandy, Somak Aditya, Pawan Goyal

机构 * Indian Institute of Technology, Kharagpur（印度理工学院，克哈格普尔）

专题命中后训练与偏好优化：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 11 pages

2411.03700 2025-05-13 cs.CL 83%

The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models

Anaelia Ovalle, Krunoslav Lehman Pavasovic, Louis Martin, Luke Zettlemoyer, Eric Michael Smith, Kai-Wei Chang, Adina Williams, Levent Sagun

机构 * FAIR, Meta（FAIR与Meta）； Meta United States（Meta美国分公司）； Meta France（Meta法国分公司）； Gen AI, Meta（Meta生成式人工智能部门）； UCLA United States（加州大学洛杉矶分校）

专题命中后训练与偏好优化：language model(title)；SFT(abstract)；preference optimization(abstract)；分类 cs.CL

Comments Accepted to 2025 ACM FAccT

2505.02363 2025-05-06 cs.CL 83%

SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning

Tianjian Li, Daniel Khashabi

机构 * Johns Hopkins University（约翰霍普金斯大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments To appear in ICML 2025

2504.06562 2025-04-18 cs.CL 83%

FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion

Longguang Zhong, Fanqi Wan, Ziyi Yang, Guosheng Liang, Tianyuan Shi, Xiaojun Quan

专题命中后训练与偏好优化：preference optimization(title,abstract)；SFT(abstract)；分类 cs.CL

2410.09893 2025-04-07 cs.CL 83%

RMB: Comprehensively Benchmarking Reward Models in LLM Alignment

Enyu Zhou, Guodong Zheng, Binghai Wang, Zhiheng Xi, Shihan Dou, Rong Bao, Wei Shen, Limao Xiong, Jessica Fan, Yurong Mou, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

专题命中后训练与偏好优化：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by ICLR2025

2503.20105 2025-03-27 cs.AI cs.RO 83%

Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrations

Ran Tian, Kratarth Goel

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(abstract)；分类 cs.AI

Comments ICLR 2025 Spotlight

2503.10480 2025-03-14 cs.CL cs.CV cs.RO 83%

World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning

Siyin Wang, Zhaoye Fei, Qinyuan Cheng, Shiduo Zhang, Panpan Cai, Jinlan Fu, Xipeng Qiu

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2410.18640 2025-03-07 cs.CL 83%

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model

Wenhong Zhu, Zhiwei He, Xiaofeng Wang, Pengfei Liu, Rui Wang

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

Comments ICLR 2025(Spotlight)

2502.20847 2025-03-03 cs.LG 83%

Gradient Imbalance in Direct Preference Optimization

Qinwei Ma, Jingzhe Shi, Can Jin, Jenq-Neng Hwang, Serge Belongie, Lei Li

专题命中后训练与偏好优化：preference optimization(title,abstract)；RLHF(abstract)；分类 cs.LG

Comments 15 pages, 2 figures

2412.03187 2025-02-27 cs.CL 83%

Weighted-Reward Preference Optimization for Implicit Model Fusion

Ziyi Yang, Fanqi Wan, Longguang Zhong, Tianyuan Shi, Xiaojun Quan

专题命中后训练与偏好优化：preference optimization(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Accepted to ICLR 2025

2502.14356 2025-02-21 cs.CL 83%

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning

Huimin Xu, Xin Mao, Feng-Lin Li, Xiaobao Wu, Wang Chen, Wei Zhang, Anh Tuan Luu

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

2411.14421 2024-11-27 cs.LG 83%

From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption

Shourya Bose, Yijiang Li, Amy Van Sant, Yu Zhang, Kibaek Kim

专题命中后训练与偏好优化：foundation model(title,abstract)；post-training(abstract)；分类 cs.LG

Comments NeurIPS 2024 Workshop on Time Series in the Age of Large Models

2411.09661 2024-11-15 cs.CL 83%

Adaptive Decoding via Latent Preference Optimization

Shehzaad Dhuliawala, Ilia Kulikov, Ping Yu, Asli Celikyilmaz, Jason Weston, Sainbayar Sukhbaatar, Jack Lanchantin

专题命中后训练与偏好优化：preference optimization(title,abstract)；language model(abstract)；分类 cs.CL

1. 后训练与偏好优化 4535 篇

VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shaping

Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems

Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization

Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models

SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment

Enhancing RLHF with Human Gaze Modeling

Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization

Differentiable Reward Optimization for LLM based TTS system

Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments

Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Models

DPO-Shift: Shifting the Distribution of Direct Preference Optimization

Iterative Self-Improvement of Vision Language Models for Image Scoring and Self-Explanation

Improving LLM-Generated Code Quality with GRPO

GAS: Generative Auto-bidding with Post-training Search

ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World

Citation Parsing and Analysis with Language Models

No Preference Left Behind: Group Distributional Preference Optimization

REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback

The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models

SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning

FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion

RMB: Comprehensively Benchmarking Reward Models in LLM Alignment

Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrations

World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model

Gradient Imbalance in Direct Preference Optimization

Weighted-Reward Preference Optimization for Implicit Model Fusion

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning

From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption

Adaptive Decoding via Latent Preference Optimization