语言大模型 / LLM

g-DPO: Scalable Preference Optimization for Protein Language Models

g-DPO：可扩展的偏好优化用于蛋白质语言模型

Constance Ferragu, Jonathan D. Ziegler, Nicolas Deutschmann, Arthur Lindoulsi, Eli Bixby, Cradle ML Team

机构 * Cradle ML Team（Cradle团队） ； Cradle Zürich, Switzerland（Cradle瑞士苏黎世） 纠错

专题命中 后训练与偏好优化 ：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.LG

AI总结 g-DPO通过序列聚类和组近似方法提升蛋白质语言模型的偏好优化效率，实现更快的收敛速度和与标准DPO相当的性能。

Comments Accepted at two workshops: FM4LS NeurIPS 2025 (accepted-paper.html" target="_blank" rel="noopener">https://nips2025fm4ls.github.io/pages/accepted-paper.html) and MLSB in Copenhagen EurIPS 2025

Reg-DPO: SFT-Regularized Direct Preference Optimization with GT-Pair for Improving Video Generation

Jie Du, Xinyu Gong, Qingshan Tan, Wen Li, Yangming Cheng, Weitao Wang, Chenlu Zhan, Suhui Wu, Hao Zhang, Jun Zhang

专题命中 后训练与偏好优化 ：SFT(title,abstract)；preference optimization(title,abstract)；分类 cs.AI

Comments The paper is withdrawn due to the need for further revision and verification of experimental results. A revised version will be resubmitted once the updates are completed

JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse

Muyao Li, Zihao Wang, Kaichen He, Xiaojian Ma, Yitao Liang

机构 * Peking University（北京大学） ； BIGAI（大疆创新） 纠错

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(title,abstract)；分类 cs.AI

Comments Accepted by ACL 2025

Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Models

Sihao Wu, Gaojie Jin, Wei Huang, Jianhong Wang, Xiaowei Huang

机构 * University of Liverpool（利物浦大学） ； University of Exeter（埃克塞特大学） ； Purple Mountain Laboratories（紫金山实验室） ； University of Bristol（布里斯托大学） 纠错

专题命中 后训练与偏好优化 ：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.AI

Post-Training Language Models for Continual Relation Extraction

Sefika Efeoglu, Adrian Paschke, Sonja Schimmler

机构 * Freie Universität Berlin（柏林自由大学） ； Technische Universität Berlin（柏林技术大学） ； Fraunhofer Institute for Open Communication Systems（弗劳恩霍夫开放通信系统研究所） 纠错

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(title)；large language model(abstract)；分类 cs.CL

Comments 17 pages, Initial Results and Reporting of the work. This work has been submitted to the IEEE for possible publication

Multi-Layer GRPO: Enhancing Reasoning and Self-Correction in Large Language Models

Fei Ding, Baiqiao Wang, Zijian Zeng, Youwei Wang

机构 * Shenzhen Jinzheng Youzhi Technology（深圳金政智科技） 纠错

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

Aligning Large Language Models with Implicit Preferences from User-Generated Content

Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang

机构 * University of Notre Dame（诺特大学） ； Amazon.com Inc（亚马逊公司） 纠错

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2025 Main Conference

LPOI: Listwise Preference Optimization for Vision Language Models

Fatemeh Pesaran Zadeh, Yoojin Oh, Gunhee Kim

专题命中 后训练与偏好优化 ：preference optimization(title,abstract)；language model(title)；RLHF(abstract)；分类 cs.AI

Comments ACL 2025 Main. Code is released at https://github.com/fatemehpesaran310/lpoi

Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes

Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, Dongyan Zhao

机构 * Antiquus S. Hippocampus, Natalia Cerebro & Amelie P. Amygdale Department of Computer Science Cranberry-Lemon University Pittsburgh, PA 15213, USA（计算机科学系， Cranberry-Lemon 大学） ； Ji Q. Ren & Yevgeny LeNet Department of Computational Neuroscience University of the Witwatersrand Joburg, South Africa（计算神经科学系， 威特沃特斯兰大学） 纠错

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leaps

Han Wang, Yilin Zhao, Dian Li, Xiaohan Wang, Gang Liu, Xuguang Lan, Hui Wang

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI

TablePilot: Recommending Human-Preferred Tabular Data Analysis with Large Language Models

Deyin Yi, Yihao Liu, Lang Cao, Mengyu Zhou, Haoyu Dong, Shi Han, Dongmei Zhang

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Generative AI Agents with Large Language Model for Satellite Networks via a Mixture of Experts Transmission

Ruichen Zhang, Hongyang Du, Yinqiu Liu, Dusit Niyato, Jiawen Kang, Zehui Xiong, Abbas Jamalipour, Dong In Kim

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.LG

Comments 15 pages, 10 figures

Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models

Wei Jie Yeo, Ranjan Satapathy, Erik Cambria

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Under review

MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models

Ziyu Liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Conghui He, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中 后训练与偏好优化 ：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.AI

Comments Project URL: https://github.com/Liuziyu77/MIA-DPO

Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning

Zongmeng Zhang, Yufeng Shi, Jinhua Zhu, Wengang Zhou, Xiang Qi, Peng Zhang, Houqiang Li

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments ICML 2024

Journal ref Proceedings of the 41st International Conference on Machine Learning, PMLR 235:59827-59850, 2024

Just Say What You Want: Only-prompting Self-rewarding Online Preference Optimization

Ruijie Xu, Zhihan Liu, Yongfei Liu, Shipeng Yan, Zhaoran Wang, Zhi Zhang, Xuming He

专题命中 后训练与偏好优化 ：prompting(title,abstract)；preference optimization(title)；RLHF(abstract)；分类 cs.AI

EPO: Hierarchical LLM Agents with Environment Preference Optimization

Qi Zhao, Haotian Fu, Chen Sun, George Konidaris

专题命中 后训练与偏好优化 ：LLM(title,abstract)；preference optimization(title,abstract)；分类 cs.LG

Comments EMNLP 2024

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment

Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Jingyuan Wang, Ji-Rong Wen

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 14 pages, working in progress

TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models

Chen Zhang, Chengguang Tang, Dading Chong, Ke Shi, Guohua Tang, Feng Jiang, Haizhou Li

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments EMNLP-2024 Findings

Alignment with Preference Optimization Is All You Need for LLM Safety

Reda Alami, Ali Khalifa Almansoori, Ahmed Alzubaidi, Mohamed El Amine Seddik, Mugariya Farooq, Hakim Hacid

专题命中 后训练与偏好优化 ：LLM(title,abstract)；preference optimization(title,abstract)；分类 cs.LG

SeaLLMs -- Large Language Models for Southeast Asia

Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Zhiqiang Hu, Chenhui Shen, Yew Ken Chia, Xingxuan Li, Jianyu Wang, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, Lidong Bing

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments Technical report, ACL 2024 DEMO TRACK

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint

Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Junchen Wan, Fuzheng Zhang, Di Zhang, Ji-Rong Wen

专题命中 后训练与偏好优化 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL

Comments 18 pages, Findings of ACL2024

Preference optimization of protein language models as a multi-objective binder design paradigm

Pouria Mistani, Venkatesh Mysore

专题命中 后训练与偏好优化 ：language model(title,abstract)；preference optimization(title,abstract)；分类 cs.AI

Comments Published at the GEM workshop, ICLR 2024. Generative and Experimental Perspectives for Biomolecular Design (https://www.gembio.ai/)

The Capacity for Moral Self-Correction in Large Language Models

Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, Karina Nguyen, Liane Lovitt, Michael Sellitto, Nelson Elhage, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robert Lasenby, Robin Larson, Sam Ringer, Sandipan Kundu, Saurav Kadavath, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Christopher Olah, Jack Clark, Samuel R. Bowman, Jared Kaplan

专题命中 后训练与偏好优化 ：language model(title,abstract)；large language model(title)；RLHF(abstract)；分类 cs.CL

Language Chameleon: Transformation analysis between languages using Cross-lingual Post-training based on Pre-trained language models

Suhyune Son, Chanjun Park, Jungseob Lee, Midan Shim, Chanhee Lee, Yoonna Jang, Jaehyung Seo, Heuiseok Lim

专题命中 后训练与偏好优化 ：language model(title,abstract)；post-training(title,abstract)；分类 cs.CL

RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction

RRC：基于排序的奖励构造解锁LLM强化学习中的生成式奖励模型

Chenglong Wang, Ziming Zhu, Yifu Huo, Bei Li, Qiaozhi He, Yan Ding, Xiaoyang Hao, Yuxin Gao, Tianhua Zhou, Xiaojia Chang, Tongran Liu, Jingbo Zhu

机构 * School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院） ； NiuTrans Research（NiuTrans研究院） ； Institute of Psychology, CAS（中国科学院心理研究所） ； Kunming University of Science and Technology（昆明理工大学） 纠错

专题命中 后训练与偏好优化 ：LLM(title,title_cn)；分类 cs.CL、cs.LG

AI总结 本研究针对生成式奖励模型在LLM强化学习中潜力未充分发挥的问题，提出RRC方法，通过两种互补策略提升RL训练效果，在多基准上取得一致增益。

EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents

EvoHarness-RL：为长视野大语言模型智能体学习自进化运行时管控器

Xuying Ning, Dongqi Fu, Tianxin Wei, Hanqing Zeng, Yuanchen Bei, Bingxuan Li, Zihao Li, Qifan Wang, Xiang Shen, Yifan Wu, Jiayi Liu, Hong Li, Yinglong Xia, Xiangjun Fan, Hanghang Tong, Jingrui He

机构 * University of Illinois Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Meta AI 纠错

专题命中 后训练与偏好优化 ：LLM(title,summary_cn)；分类 cs.CL、cs.LG

AI总结 本研究提出 EvoHarness-RL 方法，通过学习管控器策略解决长视野 LLM 智能体的外部状态管理问题，在 ALFWorld 上达到 96.9% 的任务成功率，验证了可训练管控器策略的有效性。

Comments Accepted to LLA@COLM 2026

LAB-Tab: LLM-Augmented Bayesian Network Adaptation for Few-Shot Tabular Generation

LAB-Tab：面向小样本表格生成的大语言模型增强型贝叶斯网络适配方法

Zijian Shen, Taijie Chen, Bin Zhou, Ziyang Jiang, Jintao Ke

专题命中 后训练与偏好优化 ：LLM(title,summary_cn)；分类 cs.AI、cs.LG

AI总结 针对小样本表格生成的分布偏移问题，提出LLM增强的贝叶斯网络适配框架LAB-Tab，在六个ACS分布偏移场景中表现优于基准方法，性能提升显著。

Comments 21 pages, 6 figures. Preprint

Branching Policy Optimization: Sandbox-Native Language Agent Reinforcement Learning

分支策略优化：沙盒原生语言智能体强化学习

Bowei He, Yankai Chen, Xiaokun Zhang, Xue Liu

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） ； McGill Univeristy（麦吉尔大学） ； City University of Hong Kong（香港城市大学） 纠错

专题命中 后训练与偏好优化 ：language agent(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

AI总结 研究基于可执行沙盒的大语言模型智能体强化学习，提出分支策略优化（BPO）算法，利用沙盒特性构建新展开拓扑，通过自适应快照、分叉动作等方式计算优势，实验验证该算法能提升成功率、降低方差并减少策略更新。

Comments Accepted by WAIC Academic 2026

Helpfulness Hurts: Domain-Dependent Degradation of Mid-Trained Compassion Values Under Post-Training

帮助有害：后训练中领域依赖的中期训练同情价值观退化

Jasmine Brazilek, Juliana Seawell

机构 * Compassion Aligned Machine Learning (CaML)（同情心对齐机器学习实验室） 纠错

专题命中 后训练与偏好优化 ：post-training(title,abstract)；SFT(abstract,abstract_cn)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 研究后训练数据领域对中期训练同情价值观的影响，发现帮助性训练比编程训练更显著降低动物同情和道德推理能力，但跨语言迁移存在差异。