语言大模型 / LLM - arXivDaily 专题

2510.23658 2025-11-13 cs.LG cs.AI 90%

Aligning Diffusion Language Models via Unpaired Preference Optimization

Vaibhav Jindal, Hejian Sang, Chun-Mao Lai, Yanning Chen, Zhipeng Wang

机构 * LinkedIn Corporation（LinkedIn公司）； University of California San Diego（加州大学圣地亚哥分校）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

2410.12854 2025-10-27 cs.CL cs.AI 90%

TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees

Weibin Liao, Xu Chu, Yasha Wang

机构 * School of Computer Science, Peking University（北京大学计算机科学系）； Center on Frontiers of Computing Studies, Peking University（北京大学前沿计算研究中心）； National Research and Engineering Center of Software Engineering, Peking University（北京大学软件工程研究中心）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICLR 2025

2506.23601 2025-09-30 cs.CL cs.AI 90%

Semantic-guided Diverse Decoding for Large Language Model

Weijie Shi, Yue Cui, Yaguang Wu, Jingzhi Fang, Shibo Zhang, Mengze Li, Sirui Han, Jia Zhu, Jiajie Xu, Xiaofang Zhou

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

2506.04039 2025-09-23 cs.CV cs.AI cs.CL 90%

Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization

Jiulong Wu, Zhengliang Shi, Shuaiqiang Wang, Jizhou Huang, Dawei Yin, Lingyong Yan, Min Cao, Min Zhang

机构 * Soochow University（苏州大学）； Baidu Inc.（百度公司）； Shandong University（山东大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments This paper is accepted by EMNLP2025

2509.08329 2025-09-11 cs.LG cs.AI 90%

Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing

Lukas Toral, Teddy Lazebnik

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

2509.06980 2025-09-10 cs.LG cs.AI 90%

RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use

Jiajun Chai, Guojun Yin, Zekun Xu, Chuhuai Yue, Yi Jia, Siyu Xia, Xiaohan Wang, Jiwen Jiang, Xiaoguang Li, Chengqi Dong, Hang He, Wei Lin

机构 * RLFactory Team（RLFactory团队）

专题命中后训练与偏好优化：post-training(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

2508.15790 2025-08-25 cs.CL cs.AI 90%

KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration

Nan Wang, Yongqi Fan, yansha zhu, ZongYu Wang, Xuezhi Cao, Xinyan He, Haiyun Jiang, Tong Ruan, Jingping Liu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2505.19743 2025-08-19 cs.CL cs.LG 90%

Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models

Yang Zhang, Yu Yu, Bo Tang, Yu Zhu, Chuxiong Sun, Wenqiang Wei, Jie Hu, Zipeng Xie, Zhiyu Li, Feiyu Xiong, Edward Chung

机构 * Hong Kong Polytechnic University（香港理工大学）； MemTensor (Shanghai) Technology Co., Ltd（MemTensor（上海）科技有限公司）； University of Science and Technology of China（中国科学技术大学）； China Telecom Corporation Limited Beijing Research Institute（中国电信北京研究院）； Nanjing University of Information Science and Technology（南京信息工程大学）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.LG

Comments Accepted to 34th International Joint Conference on Artificial Intelligence (IJCAI 2025)

2508.01174 2025-08-05 cs.LG cs.AI 90%

RSPO: Risk-Seeking Policy Optimization for Pass@k and Max@k Metrics in Large Language Models

Kaichen Zhang, Shenghao Gao, Yuzhong Hong, Haipeng Sun, Junwei Bao, Hongfei Jiang, Yang Song, Hong Dingqian, Hui Xiong

机构 * Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））； Zuoyebang Education Technology（左就业教育科技）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；post-training(abstract)；分类 cs.AI、cs.LG

2507.17107 2025-07-30 cs.LG cs.AI 90%

Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Models

Andrii Balashov

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments The manuscript has been withdrawn due to significant overlap in methodology and results with a prior work (arXiv:2505.11711) that we were not aware of at the time of submission. To maintain academic integrity and avoid redundancy in the literature, we have chosen to withdraw this version

2507.04099 2025-07-16 cs.CL cs.AI 90%

Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branching

Thomas Savage

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2507.07725 2025-07-11 cs.CL cs.AI 90%

Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization

Zhijin Dong

机构 * Peking University（北京大学）

专题命中后训练与偏好优化：post-training(title,abstract)；preference optimization(title)；large language model(abstract)；language model(abstract)

2503.13377 2025-07-01 cs.CV cs.AI cs.CL 90%

Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding

Ye Wang, Ziheng Wang, Boshen Xu, Yang Du, Kejun Lin, Zihan Xiao, Zihao Yue, Jianzhong Ju, Liang Zhang, Dingyi Yang, Xiangnan Fang, Zewen He, Zhenbo Luo, Wenxuan Wang, Junqi Lin, Jian Luan, Qin Jin

机构 * AIM3 Lab, Renmin University of China（中国人民大学AIM3实验室）； MiLM Plus, Xiaomi Inc.（小米公司MiLM Plus）

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；SFT(abstract)；分类 cs.CL、cs.AI

Comments Project Page: https://xuboshen.github.io/Time-R1/

2506.03541 2025-06-05 cs.CL cs.AI 90%

Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancement

Xiaofeng Zhou, Heyan Huang, Lizi Liao

机构 * Beijing Institute of Technology（北京理工大学）； Singapore Management University（新加坡国立大学）

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 10 figures. The camera-ready paper for Findings of ACL 2025

2501.04945 2025-06-03 cs.CL cs.AI 90%

Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models

Qingyu Ren, Jie Zeng, Qianyu He, Jiaqing Liang, Yanghua Xiao, Weikang Zhou, Zeye Sun, Fei Yu

机构 * Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University（上海数据科学 key laboratory，计算机科学与人工智能学院，复旦大学）； School of Data Science, Fudan University（数据科学学院，复旦大学）； Ant Group（蚂蚁集团）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

2502.11051 2025-05-28 cs.CL cs.AI 90%

MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models

Jiahao Huo, Yibo Yan, Xu Zheng, Yuanhuiyi Lyu, Xin Zou, Zhihua Wei, Xuming Hu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI

Comments Accepted as ACL 2025 Findings

2505.12435 2025-05-20 cs.LG cs.AI cs.DC 90%

SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment

Wenqiao Zhu, Ji Liu, Lulu Wang, Jun Wu, Yulun Zhang

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, to appear in ACL'25

2410.12851 2025-04-22 cs.CL cs.AI 90%

VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models

Lisa Dunlap, Krishna Mandal, Trevor Darrell, Jacob Steinhardt, Joseph E Gonzalez

机构 * UC Berkeley（加州大学伯克利分校）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments unironic use of the word 'vibe', added more analysis and cooler graphs. added website link

2410.01532 2025-04-01 cs.CL cs.AI cs.CV cs.HC 90%

Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models

Angela Lopez-Cardona, Carlos Segura, Alexandros Karatzoglou, Sergi Abadal, Ioannis Arapakis

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；分类 cs.CL、cs.AI

Comments This paper has been accepted to ICLR 2025

2503.03039 2025-03-06 cs.LG cs.AI 90%

LLM Misalignment via Adversarial RLHF Platforms

Erfan Entezami, Ali Naseh

专题命中后训练与偏好优化：LLM(title,abstract)；RLHF(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

2410.02205 2025-02-11 cs.CL cs.AI cs.LO 90%

Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models

Yinhong Liu, Zhijiang Guo, Tianya Liang, Ehsan Shareghi, Ivan Vulić, Nigel Collier

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

2402.13846 2025-02-04 cs.AI cs.CL cs.CR 90%

Large Language Models are Advanced Anonymizers

Robin Staab, Mark Vero, Mislav Balunović, Martin Vechev

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments International Conference on Learning Representations (ICLR 2024)

2412.14516 2024-12-20 cs.LG cs.CL 90%

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment

Teng Xiao, Yige Yuan, Huaisheng Zhu, Mingxiao Li, Vasant G Honavar

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted by NeurIPS 2024 Main

2412.13702 2024-12-20 cs.CL cs.AI 90%

Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models

Kunat Pipatanakul, Potsawee Manakul, Natapong Nitarach, Warit Sirichotedumrong, Surapon Nonesung, Teetouch Jaknamon, Parinthapat Pengpun, Pittawat Taveekitworachai, Adisai Na-Thalang, Sittipong Sripaisarnmongkol, Krisanapong Jirayoot, Kasima Tharnpipitchai

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；post-training(abstract)；分类 cs.CL、cs.AI

Comments technical report, 55 pages

2406.10023 2024-10-30 cs.LG cs.CL stat.ML 90%

Deep Bayesian Active Learning for Preference Modeling in Large Language Models

Luckeciano C. Melo, Panagiotis Tigas, Alessandro Abate, Yarin Gal

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

Comments 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

2410.06961 2024-10-10 cs.CL cs.AI 90%

Self-Boosting Large Language Models with Synthetic Preference Data

Qingxiu Dong, Li Dong, Xingxing Zhang, Zhifang Sui, Furu Wei

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

2405.15194 2024-10-10 cs.LG cs.AI 90%

Extracting Heuristics from Large Language Models for Reward Shaping in Reinforcement Learning

Siddhant Bhambri, Amrita Bhattacharjee, Durgesh Kalwar, Lin Guan, Huan Liu, Subbarao Kambhampati

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI、cs.LG

2405.00747 2024-10-07 cs.LG cs.AI 90%

Soft Preference Optimization: Aligning Language Models to Expert Distributions

Arsalan Sharifnassab, Saber Salehkaleybar, Sina Ghiassian, Surya Kanoria, Dale Schuurmans

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

2408.03297 2024-08-20 cs.CL cs.AI 90%

KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models

Ruizhe Zhang, Yongxin Xu, Yuzhen Xiao, Runchuan Zhu, Xinke Jiang, Xu Chu, Junfeng Zhao, Yasha Wang

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

2405.20830 2024-06-03 cs.CL cs.LG 90%

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment

Yueqin Yin, Zhendong Wang, Yujia Xie, Weizhu Chen, Mingyuan Zhou

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；分类 cs.CL、cs.LG

1. 后训练与偏好优化 4507 篇

Aligning Diffusion Language Models via Unpaired Preference Optimization

TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees

Semantic-guided Diverse Decoding for Large Language Model

Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization

Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing

RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use

KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration

Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models

RSPO: Risk-Seeking Policy Optimization for Pass@k and Max@k Metrics in Large Language Models

Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Models

Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branching

Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization

Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding

Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancement

Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models

MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models

SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment

VibeCheck: Discover and Quantify Qualitative Differences in Large Language Models

Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models

LLM Misalignment via Adversarial RLHF Platforms

Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models

Large Language Models are Advanced Anonymizers

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment

Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models

Deep Bayesian Active Learning for Preference Modeling in Large Language Models

Self-Boosting Large Language Models with Synthetic Preference Data

Extracting Heuristics from Large Language Models for Reward Shaping in Reinforcement Learning

Soft Preference Optimization: Aligning Language Models to Expert Distributions

KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment