语言大模型 / LLM

Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models

Yuhua Jiang, Jiawei Huang, Yufeng Yuan, Xin Mao, Yu Yue, Qianchuan Zhao, Lin Yan

机构 * Tsinghua University（清华大学） ； ETH Zurich（苏黎世联邦理工学院） 纠错

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization

Shuaijie She, Yu Bao, Yu Lu, Lu Xu, Tao Li, Wenhao Zhu, Shujian Huang, Shanbo Cheng, Lu Lu, Yuxuan Wang

机构 * Nanjing University（南京大学） 纠错

专题命中 后训练与偏好优化 ：LLM(title,abstract)；preference optimization(title,abstract)；分类 cs.CL、cs.LG

Comments 18 pages, 4 figures,

Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models

Haitao Hong, Yuchen Yan, Xingyu Wu, Guiyang Hou, Wenqi Zhang, Weiming Lu, Yongliang Shen, Jun Xiao

机构 * Zhejiang University（浙江大学） 纠错

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Project Page: https://zju-real.github.io/cooper Code: https://github.com/zju-real/cooper

A Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval-Augmented Generation in Large Language Models

Qikai Wei, Huansheng Ning, Chunlong Han, Jianguo Ding

机构 * School of Computer and Communication Engineering, University of Science and Technology Beijing（计算机与通信工程学院，北京科技大学） ； Blekinge Institute of Technology（布莱金厄学院） 纠错

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Derailing Non-Answers via Logit Suppression at Output Subspace Boundaries in RLHF-Aligned Language Models

Harvey Dam, Jonas Knochelmann, Vinu Joseph, Ganesh Gopalakrishnan

专题命中 后训练与偏好优化 ：language model(title,abstract)；RLHF(title)；large language model(abstract)；分类 cs.CL、cs.LG

R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO

Huanjin Yao, Qixiang Yin, Jingyi Zhang, Min Yang, Yibo Wang, Wenhao Wu, Fei Su, Li Shen, Minghui Qiu, Dacheng Tao, Jiaxing Huang

机构 * Nanyang Technological University（南洋理工大学） ； ByteDance（字节跳动） ； Tsinghua University（清华大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） ； The University of Sydney（悉尼大学） 纠错

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Technical report

Direct Density Ratio Optimization: A Statistically Consistent Approach to Aligning Large Language Models

Rei Higuchi, Taiji Suzuki

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.LG

Enhancing Large Language Models with Reward-guided Tree Search for Knowledge Graph Question and Answering

Xiao Long, Liansheng Zhuang, Chen Shen, Shaotian Yan, Yifei Li, Shafei Wang

机构 * School of Cyberspace Science and Technology, University of Science and Technology of China (USTC)（信息科学与技术学院，中国科学技术大学） ； Peng Cheng Laboratory（鹏城实验室） 纠错

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Token-Level Uncertainty-Aware Objective for Language Model Post-Training

Tingkai Liu, Ari S. Benjamin, Anthony M. Zador

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(title,abstract)；分类 cs.CL、cs.AI

Detecting Mode Collapse in Language Models via Narration

Sil Hamilton

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(abstract,comments)；instruction tuning(abstract)；RLHF(abstract)

Comments To appear in the proceedings of the first Workshop on the Scaling Behavior of Large Language Models (EACL 2024)

Journal ref https://aclanthology.org/2024.scalellm-1.5/

CONSTRUCTA: Automating Commercial Construction Schedules in Fabrication Facilities with Large Language Models

Yifan Zhang, Xue Yang

专题命中 后训练与偏好优化 ：large language model(title)；language model(title)；LLM(abstract)；RLHF(abstract)

Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning

Hao Sun, Mihaela van der Schaar

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model

Yueqin Yin, Shentao Yang, Yujia Xie, Ziyi Yang, Yuting Sun, Hany Awadalla, Weizhu Chen, Mingyuan Zhou

专题命中 后训练与偏好优化 ：language model(title,abstract)；RLHF(title,abstract)；分类 cs.CL、cs.AI

Generating bilingual example sentences with large language models as lexicography assistants

Raphael Merx, Ekaterina Vylomova, Kemal Kurniawan

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(title)；LLM(abstract)；分类 cs.CL、cs.AI

Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering

Xinyan Guan, Yanjiang Liu, Xinyu Lu, Boxi Cao, Ben He, Xianpei Han, Le Sun, Jie Lou, Bowen Yu, Yaojie Lu, Hongyu Lin

专题命中 后训练与偏好优化 ：foundation model(title,abstract)；post-training(title,abstract)；分类 cs.CL、cs.AI

Enhancing the Traditional Chinese Medicine Capabilities of Large Language Model through Reinforcement Learning from AI Feedback

Song Yu, Xiaofei Xu, Fangfei Xu, Li Li

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 2 figures

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models

Yanjun Chen, Dawei Zhu, Yirong Sun, Xinghao Chen, Wei Zhang, Xiaoyu Shen

专题命中 后训练与偏好优化 ：language model(title,abstract)；RLHF(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 27 figures (including 18 in the appendix), submitted to EMNLP 2024

SeRA: Self-Reviewing and Alignment of Large Language Models using Implicit Reward Margins

Jongwoo Ko, Saket Dingliwal, Bhavana Ganesh, Sailik Sengupta, Sravan Bodapati, Aram Galstyan

专题命中 后训练与偏好优化 ：large language model(title)；language model(title)；RLHF(abstract)；preference optimization(abstract)

Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cycle

Emman Haider, Daniel Perez-Becker, Thomas Portet, Piyush Madan, Amit Garg, Atabak Ashfaq, David Majercak, Wen Wen, Dongwoo Kim, Ziyi Yang, Jianwen Zhang, Hiteshi Sharma, Blake Bullwinkel, Martin Pouliot, Amanda Minnich, Shiven Chawla, Solianna Herrera, Shahed Warreth, Maggie Engler, Gary Lopez, Nina Chikanov, Raja Sekhar Rao Dheekonda, Bolor-Erdene Jagdagdorj, Roman Lutz, Richard Lundeen, Tori Westerhoff, Pete Bryan, Christian Seifert, Ram Shankar Siva Kumar, Andrew Berkley, Alex Kessler

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(title,abstract)；分类 cs.CL、cs.AI

Strong and weak alignment of large language models with human values

Mehdi Khamassi, Marceau Nahon, Raja Chatila

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted for publication in Scientific Reports, special issue on AI aligment

Transforming and Combining Rewards for Aligning Large Language Models

Zihao Wang, Chirag Nagpal, Jonathan Berant, Jacob Eisenstein, Alex D'Amour, Sanmi Koyejo, Victor Veitch

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(title)；RLHF(abstract)；分类 cs.CL、cs.AI

Aligning Large Language Models from Self-Reference AI Feedback with one General Principle

Rong Bao, Rui Zheng, Shihan Dou, Xiao Wang, Enyu Zhou, Bo Wang, Qi Zhang, Liang Ding, Dacheng Tao

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 3 figures

Human Alignment of Large Language Models through Online Preference Optimisation

Daniele Calandriello, Daniel Guo, Remi Munos, Mark Rowland, Yunhao Tang, Bernardo Avila Pires, Pierre Harvey Richemond, Charline Le Lan, Michal Valko, Tianqi Liu, Rishabh Joshi, Zeyu Zheng, Bilal Piot

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(title)；RLHF(abstract)；分类 cs.AI、cs.LG

Language Models are Bounded Pragmatic Speakers: Understanding RLHF from a Bayesian Cognitive Modeling Perspective

Khanh Nguyen

专题命中 后训练与偏好优化 ：language model(title,abstract)；RLHF(title)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Proceedings of the First Workshop on Theory of Mind in Communicating Agents at (TOM @ ICML 2023)

Preference Optimization for Molecular Language Models

Ryan Park, Ryan Theisen, Navriti Sahni, Marcel Patek, Anna Cichońska, Rayees Rahman

专题命中 后训练与偏好优化 ：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.AI、cs.LG

Domain-Specific Language Model Post-Training for Indonesian Financial NLP

Ni Putu Intan Maharani, Yoga Yustiawan, Fauzy Caesar Rochim, Ayu Purwarianti

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in ICEEI 2023 (International Conference on Electrical Engineering and Informatics 2023)

Pretraining Language Models with Human Preferences

Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Bhalerao, Christopher L. Buckley, Jason Phang, Samuel R. Bowman, Ethan Perez

专题命中 后训练与偏好优化 ：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.LG

Comments ICML 2023

Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models

Renat Aksitov, Chung-Ching Chang, David Reitter, Siamak Shakeri, Yunhsuan Sung

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Federated Foundation Language Model Post-Training Should Focus on Open-Source Models

联邦基础语言模型后训练应聚焦开源模型

Nikita Agrawal, Ruben Mayer

机构 * University of Bayreuth（拜罗伊特大学） 纠错

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(title,abstract)；分类 cs.LG；foundation model(comments)

AI总结 本文批判性分析联邦后训练中使用黑盒模型的问题，主张采用开源模型以符合联邦学习的数据隐私和自治原则。

Comments Accepted at International Workshop on Federated Learning in the Age of Foundation Models In Conjunction with IJCAI 2026

TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Text

TextCloak：基于强化学习的不可学习文本防御未经授权的大语言模型利用

Chengshuai Zhao, Pingchuan Ma, Dawei Li, Bohan Jiang, Zhiyuan Yu, Zhen Tan, Huan Liu

专题命中 后训练与偏好优化 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 本研究提出TextCloak框架，采用强化学习与GRPO-UE技术生成不可学习文本，可削弱大语言模型的未经授权微调，兼具实用性与鲁棒性。