语言大模型 / LLM - arXivDaily 专题

2506.23998 2025-08-12 cs.CL 90%

Auto-TA: Towards Scalable Automated Thematic Analysis (TA) via Multi-Agent Large Language Models with Reinforcement Learning

Seungjun Yi, Joakim Nguyen, Huimin Xu, Terence Lim, Andrew Well, Mia Markey, Ying Ding

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments Presented at ACL 2025 SRW

2502.18679 2025-07-25 cs.CL 90%

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data

Siqi Guo, Ilgee Hong, Vicente Balmaseda, Changlong Yu, Liang Qiu, Xin Liu, Haoming Jiang, Tuo Zhao, Tianbao Yang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；SFT(abstract)；preference optimization(abstract)

Comments 18 pages, 7 figures

2411.01834 2025-05-28 cs.CL eess.AS 90%

Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback

Guan-Ting Lin, Prashanth Gurunath Shivakumar, Aditya Gourav, Yile Gu, Ankur Gandhe, Hung-yi Lee, Ivan Bulyko

机构 * Amazon AGI（亚马逊人工智能实验室）； Graduate Institute of Communication Engineering, National Taiwan University（国立台湾大学通信工程研究所）

专题命中后训练与偏好优化：language model(title,abstract)；SLM(title,abstract)；large language model(abstract)；preference optimization(abstract)

Comments Accepted by ACL 2025

2505.08507 2025-05-14 cs.LG 90%

InfoPO: On Mutual Information Maximization for Large Language Model Alignment

Teng Xiao, Zhen Ge, Sujay Sanghavi, Tian Wang, Julian Katz-Samuels, Marc Versage, Qingjun Cui, Trishul Chilimbi

机构 * Amazon（亚马逊）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；post-training(abstract)；preference optimization(abstract)

Comments NAACL 2025

2505.01073 2025-05-05 cs.AI 90%

Retrieval Augmented Learning: A Retrial-based Large Language Model Self-Supervised Learning and Autonomous Knowledge Generation

Zongyuan Li, Pengfei Li, Runnan Qi, Yanan Ni, Lumin Jiang, Hui Wu, Xuebo Zhang, Kuihua Huang, Xian Guo

机构 * College of Artificial Intelligence, Nankai University（南开大学人工智能学院）； Laboratory for Big Data and Decision, National University of Defense Technology（国防科技大学大数据与决策实验室）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

2504.14838 2025-04-22 cs.AI 90%

Establishing Reliability Metrics for Reward Models in Large Language Models

Yizhou Chen, Yawen Liu, Xuesi Wang, Qingtao Yu, Guangda Huzhang, Anxiang Zeng, Han Yu, Zhiming Zhou

机构 * Nanyang Technological University（新加坡国立大学）； Shanghai University of Finance and Economics（上海金融学院）

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2411.15124 2025-04-16 cs.CL 90%

Tulu 3: Pushing Frontiers in Open Language Model Post-Training

Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, Hannaneh Hajishirzi

专题命中后训练与偏好优化：language model(title,abstract)；post-training(title,abstract)；SFT(abstract)；preference optimization(abstract)

Comments Added Tulu 3 405B results and additional analyses

2410.17509 2025-04-15 cs.LG 90%

WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models

Jinghan Jia, Jiancheng Liu, Yihua Zhang, Parikshit Ram, Nathalie Baracaldo, Sijia Liu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

Comments NeurIPS'24

2503.15129 2025-03-20 cs.AI 90%

Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models

Man Fai Wong, Chee Wei Tan

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2503.09407 2025-03-13 cs.CL 90%

Got Compute, but No Data: Lessons From Post-training a Finnish LLM

Elaine Zosa, Ville Komulainen, Sampo Pyysalo

专题命中后训练与偏好优化：LLM(title,abstract)；post-training(title,abstract)；instruction tuning(abstract)；preference optimization(abstract)

Comments 7 pages

Journal ref Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)

2501.15007 2025-01-28 cs.AI cs.CE q-bio.QM 90%

Controllable Protein Sequence Generation with LLM Preference Optimization

Xiangyu Liu, Yi Liu, Silei Chen, Wei Hu

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted in the 39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025)

2501.01336 2025-01-03 cs.CL 90%

Aligning Large Language Models for Faithful Integrity Against Opposing Argument

Yong Zhao, Yang Deng, See-Kiong Ng, Tat-Seng Chua

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；preference optimization(abstract)

Comments 17 pages, 5 figures

2412.10742 2024-12-17 cs.CL 90%

WEPO: Web Element Preference Optimization for LLM-based Web Navigation

Jiarun Liu, Jia Hao, Chunhong Zhang, Zheng Hu

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

Comments Published at AAAI 2025

2411.02712 2024-11-06 cs.CV cs.AI 90%

V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization

Yuxi Xie, Guanzhen Li, Xiao Xu, Min-Yen Kan

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)

Comments EMNLP 2024 Findings; 9 pages, 6 figures, 5 tables (16 pages, 8 figures, 8 tables including references and appendices)

2410.14184 2024-10-21 cs.CL 90%

MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time

Mozhi Zhang, Pengyu Wang, Chenkun Tan, Mianqiu Huang, Dong Zhang, Yaqian Zhou, Xipeng Qiu

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；preference optimization(abstract)

Comments 19 pages, 6 figures

2410.06682 2024-10-14 cs.CV cs.CL eess.IV 90%

Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization

Changli Tang, Yixuan Li, Yudong Yang, Jimin Zhuang, Guangzhi Sun, Wei Li, Zujun Ma, Chao Zhang

专题命中后训练与偏好优化：LLM(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

2408.14874 2024-08-30 cs.CL 90%

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data

Han Xia, Songyang Gao, Qiming Ge, Zhiheng Xi, Qi Zhang, Xuanjing Huang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；preference optimization(abstract)

2407.13928 2024-07-22 cs.CL 90%

BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization

Ahmed Allam

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)

2312.15997 2024-07-04 cs.CL 90%

Aligning Large Language Models with Human Preferences through Representation Engineering

Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Jianhao Zhu, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2405.20253 2024-05-31 cs.CL 90%

Evaluating Large Language Model Biases in Persona-Steered Generation

Andy Liu, Mona Diab, Daniel Fried

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments Accepted to Findings of ACL 2024. Code and data available at https://github.com/andyjliu/persona-steered-generation-bias

2404.01258 2024-04-03 cs.CV cs.AI 90%

Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward

Ruohong Zhang, Liangke Gui, Zhiqing Sun, Yihao Feng, Keyang Xu, Yuanhan Zhang, Di Fu, Chunyuan Li, Alexander Hauptmann, Yonatan Bisk, Yiming Yang

专题命中后训练与偏好优化：language model(title,abstract)；preference optimization(title,abstract)；LLM(abstract)；large language model(abstract)

2403.18341 2024-03-28 cs.CL 90%

IterAlign: Iterative Constitutional Alignment of Large Language Models

Xiusi Chen, Hongzhi Wen, Sreyashi Nag, Chen Luo, Qingyu Yin, Ruirui Li, Zheng Li, Wei Wang

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments NAACL 2024

2310.14422 2024-02-02 cs.CL 90%

Large Language Models are biased to overestimate profoundness

Eugenio Herrera-Berg, Tomás Vergara Browne, Pablo León-Villagrá, Marc-Lluís Vives, Cristian Buc Calderon

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；RLHF(abstract)；prompting(abstract)

Comments 5 pages, 3 figures

Journal ref https://aclanthology.org/2023.emnlp-main.599

2312.07876 2023-12-14 cs.AI 90%

Causality Analysis for Evaluating the Security of Large Language Models

Wei Zhao, Zhe Li, Jun Sun

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

2304.01852 2023-08-25 cs.CL 90%

Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models

Yiheng Liu, Tianle Han, Siyuan Ma, Jiayue Zhang, Yuanyuan Yang, Jiaming Tian, Hao He, Antong Li, Mengshen He, Zhengliang Liu, Zihao Wu, Lin Zhao, Dajiang Zhu, Xiang Li, Ning Qiang, Dingang Shen, Tianming Liu, Bao Ge

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments 21 pages, 4 figures, accepted by Meta-Radiology

Journal ref Meta-Radiology (2023)100017

2303.05453 2023-03-10 cs.CL cs.CY 90%

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, Scott A. Hale

专题命中后训练与偏好优化：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；RLHF(abstract)

Comments 19 pages, 1 table

2608.11381 2026-08-13 cs.AI cs.LG 新提交 90%

From Numbers to Judgment: Specialist LLM Agents and Reinforcement Learning for European Listed Real Estate

从数字到判断：专业大语言模型智能体与欧洲上市房地产的强化学习研究

Pardis Taghavi, Santosh Bhavani

专题命中后训练与偏好优化：LLM(title,summary_cn)；post-training(abstract)；prompting(abstract)；分类 cs.AI、cs.LG

AI总结本研究以欧洲上市房地产分析为对象，提出Larix框架将分析维度映射为专业LLM智能体，结合GRPO微调Qwen3.5-9B，实现数值任务与判断任务的性能提升且可迁移至新企业与监管体系。

2605.12991 2026-08-11 cs.LG cs.AI 版本更新 90%

Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy

不只是RLHF：为何仅对齐不足以解决多智能体趋同

Adarsh Kumarappan, Ananya Mujoo

机构 * California Institute of Technology（加州理工学院）； Evergreen Valley College（艾弗绿谷学院）

专题命中后训练与偏好优化：RLHF(title,title_cn)；LLM(abstract,abstract_cn)；分类 cs.AI、cs.LG

AI总结本文研究了多智能体系统在模拟同伴分歧下的错误率问题，发现预训练基础模型与指令模型存在相似的替换模式，且错误率较高。通过激活修补发现错误集中在中间层，修复后可恢复大部分正确率差距。研究还指出压力抑制了清洁推理特征，而非激活新的趋同回路。

2604.10727 2026-08-04 stat.ML cs.AI cs.LG math.PR math.ST stat.TH 版本更新 90%

Tail-Aware Information-Theoretic Bounds for LLM Alignment under Heavy-Tailed Rewards

尾感知信息论泛化用于RLHF和SGLD

Huiming Zhang, Binghan Li, Wan Tian, Qiang Sun

机构 * Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）； Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（北京未来区块链与隐私计算高精尖创新中心）； Advanced Institute of Information Technology, Peking University（北京大学信息技术高等研究院）； Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机技术研究所）； Computer and Mathematical Sciences, Computer Science, and Statistics, University of Toronto（多伦多大学计算机与数学科学、计算机科学和统计学系）； MBZUAI（穆罕默德·本·扎耶德人工智能大学）

专题命中后训练与偏好优化：RLHF(title_cn,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

AI总结本文提出一种尾依赖信息论框架，用于处理亚韦尔bull数据，通过移位对数f_θ-分歧来界定制换测度期望，从而在重尾数据下获得更精确的泛化界限。

Comments 47 pages, 6 figures

2606.27580 2026-06-29 cs.LG cs.AI 新提交 90%

Retroactive Advantage Correction: Closed-Form V-Trace Bias Correction for Delay-Aware RLHF

回溯优势修正：面向延迟感知RLHF的闭式V-Trace偏差修正

Arnav Raj

专题命中后训练与偏好优化：RLHF(title,title_cn)；分类 cs.AI、cs.LG

AI总结针对RLHF中奖励信号延迟的问题，提出回溯优势修正（RAC），通过非负核函数队列化延迟反馈并注入优势估计，理论证明其无偏性，在表格MDP实验中偏差降低达47.9倍。

Comments Accepted at the ICML 2026 Workshop on Reinforcement Learning from World Feedback (RLxF). Code: https://github.com/deadsmash07/rac-rlxf-code

1. 后训练与偏好优化 4489 篇

Auto-TA: Towards Scalable Automated Thematic Analysis (TA) via Multi-Agent Large Language Models with Reinforcement Learning

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data

Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback

InfoPO: On Mutual Information Maximization for Large Language Model Alignment

Retrieval Augmented Learning: A Retrial-based Large Language Model Self-Supervised Learning and Autonomous Knowledge Generation

Establishing Reliability Metrics for Reward Models in Large Language Models

Tulu 3: Pushing Frontiers in Open Language Model Post-Training

WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models

Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models

Got Compute, but No Data: Lessons From Post-training a Finnish LLM

Controllable Protein Sequence Generation with LLM Preference Optimization

Aligning Large Language Models for Faithful Integrity Against Opposing Argument

WEPO: Web Element Preference Optimization for LLM-based Web Navigation

V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization

MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time

Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data

BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization

Aligning Large Language Models with Human Preferences through Representation Engineering

Evaluating Large Language Model Biases in Persona-Steered Generation

Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward

IterAlign: Iterative Constitutional Alignment of Large Language Models

Large Language Models are biased to overestimate profoundness

Causality Analysis for Evaluating the Security of Large Language Models

Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

From Numbers to Judgment: Specialist LLM Agents and Reinforcement Learning for European Listed Real Estate

Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy

Tail-Aware Information-Theoretic Bounds for LLM Alignment under Heavy-Tailed Rewards

Retroactive Advantage Correction: Closed-Form V-Trace Bias Correction for Delay-Aware RLHF