大模型对齐与安全

MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Intervention

Yuxin Chen, Chen Tang, Jianglan Wei, Chenran Li, Ran Tian, Xiang Zhang, Wei Zhan, Peter Stone, Masayoshi Tomizuka

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI、cs.LG

An AI-Based Behavioral Health Safety Filter and Dataset for Identifying Mental Health Crises in Text-Based Conversations

Benjamin W. Nelson, Celeste Wong, Matthew T. Silvestrini, Sooyoon Shin, Alanna Robinson, Jessica Lee, Eric Yang, John Torous, Andrew Trister

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI

Comments Main Text: 2943; Abstract: 256; Tables and Figures: 5

A Granular Study of Safety Pretraining under Model Abliteration

Shashank Agnihotri, Jonas Jakubassa, Priyam Dey, Sachin Goyal, Bernt Schiele, Venkatesh Babu Radhakrishnan, Margret Keuper

机构 * Data and Web Science Group, University of Mannheim（曼海姆大学数据与网络科学组） ； Vision and AI Lab, Indian Institute of Science（印度科学院视觉与人工智能实验室） ； Carnegie Mellon University（卡内基梅隆大学） ； Max-Planck-Institute for Informatics, Saarland Informatics Campus（马克斯·普朗克信息研究所，萨尔兰信息校园） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at NeurIPS 2025 bWorkshop Lock-LLM. *Equal Contribution

Predictive Modeling and Explainable AI for Veterinary Safety Profiles, Residue Assessment, and Health Outcomes Using Real-World Data and Physicochemical Properties

Hossein Sholehrasa, Xuan Xu, Doina Caragea, Jim E. Riviere, Majid Jaberi-Douraki

机构 * DATA Consortium and FARAD Program, Kansas State University, Olathe, KS, USA（1DATA Consortium和FARAD计划，堪萨斯州立大学） ； Department of Computer Science, Kansas State University, Manhattan, KS, USA（计算机科学系，堪萨斯州立大学） ； Department of Statistics, Kansas State University, Manhattan, KS, USA（统计学系，堪萨斯州立大学） ； Department of Mathematics, Kansas State University, Olathe, KS, USA（数学系，堪萨斯州立大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Dynamic Policy Fusion for User Alignment Without Re-Interaction

Ajsal Shereef Palattuparambil, Thommen George Karimpanal, Santu Rana

机构 * A2I2 ； Deakin University（德肯大学） ； School of IT（信息学院） 纠错

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI、cs.LG

Improving Aviation Safety Analysis: Automated HFACS Classification Using Reinforcement Learning with Group Relative Policy Optimization

Arash Ahmadi, Sarah Sharif, Yaser Banad

机构 * School of Electrical, and Computer Engineering, University of Oklahoma（电气与计算机工程学院，俄克拉荷马大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI

Rethinking Safety in LLM Fine-tuning: An Optimization Perspective

Minseon Kim, Jin Myung Kwak, Lama Alssum, Bernard Ghanem, Philip Torr, David Krueger, Fazl Barez, Adel Bibi

机构 * Microsoft Research（微软研究院） ； KAIST（韩国科学技术院） ； KAUST（韩国科学技术大学） ； Université de Montréal（蒙特利尔大学） ； Mila（蒙特利尔人工智能研究院） ； WhiteBox（WhiteBox公司） ； University of Oxford（牛津大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems

H. M. Sabbir Ahmad, Ehsan Sabouni, Alexander Wasilkoff, Param Budhraja, Zijian Guo, Songyuan Zhang, Chuchu Fan, Christos Cassandras, Wenchao Li

机构 * Boston University（波士顿大学） ； Massachusetts Institute of Technology（麻省理工学院） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Building Effective Safety Guardrails in AI Education Tools

Hannah-Beth Clark, Laura Benton, Emma Searle, Margaux Dowland, Matthew Gregory, Will Gayne, John Roberts

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.CY

Comments 9 pages, published in proceedings of International Conference on Artificial Intelligence in Education (AIED) 2025: Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium, Blue Sky, and WideAIED

Journal ref Communications in Computer and Information Science (2025) vol 2590, pp. 129-136

HarmonyGuard: Toward Safety and Utility in Web Agents via Adaptive Policy Enhancement and Dual-Objective Optimization

Yurun Chen, Xavier Hu, Yuhan Liu, Keting Yin, Juncheng Li, Zhuosheng Zhang, Shengyu Zhang

机构 * Zhejiang University（浙江大学） ； Xiamen University（厦门大学） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI

Conformal Safety Shielding for Imperfect-Perception Agents

William Scarbro, Calum Imrie, Sinem Getir Yaman, Kavan Fatehi, Corina S. Pasareanu, Radu Calinescu, Ravi Mangal

机构 * Colorado State University, USA（科罗拉多州立大学） ； University of York, UK（约克大学） ； Carnegie Mellon University, USA（卡内基梅隆大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 32 pages; Equal contribution by W. Scarbro and C. Imrie; Accepted at 25th International Conference on Runtime Verification, 2025 (RV25)

The Safety Gap Toolkit: Evaluating Hidden Dangers of Open-Source Models

Ann-Kathrin Dombrowski, Dillon Bowen, Adam Gleave, Chris Cundy

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CY、cs.LG

Comments 9 pages plus appendix

Assuring the Safety of Reinforcement Learning Components: AMLAS-RL

Calum Corrie Imrie, Ioannis Stefanakos, Sepeedeh Shahbeigi, Richard Hawkins, Simon Burton

机构 * Department of Computer Science, University of York（约克大学计算机科学系） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

ROCKET-2: Steering Visuomotor Policy via Cross-View Goal Alignment

Shaofei Cai, Zhancun Mu, Anji Liu, Yitao Liang

机构 * Peking University（北京大学） ； University of California, Los Angeles（加州大学洛杉矶分校） ； Team CraftJarvis（CraftJarvis团队） 纠错

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI、cs.LG

SafeCoT: Improving VLM Safety with Minimal Reasoning

Jiachen Ma, Zhanhui Zhou, Chao Yang, Chaochao Lu

机构 * Shanghai AI Laboratory（上海人工智能实验室） ； Fudan University（复旦大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset

Khaoula Chehbouni, Jonathan Colaço Carr, Yash More, Jackie CK Cheung, Golnoosh Farnadi

机构 * McGill University（麦吉尔大学） ； Mila - Quebec AI Institute（魁北克人工智能研究所） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.CY

Comments NAACL Main Conference 2025 - Accepted as an Oral

Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs

Wenxuan Wang, Xiaoyuan Liu, Kuiyi Gao, Jen-tse Huang, Youliang Yuan, Pinjia He, Shuai Wang, Zhaopeng Tu

机构 * Renmin University of China（中国人民大学） ； Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Chinese University of Hong Kong（香港中文大学） ； Johns Hopkins University（约翰霍普金斯大学） ； Hong Kong University of Science and Technology（香港科技大学） ； Tencent（腾讯） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2025

Learning Safety Constraints for Large Language Models

Xin Chen, Yarden As, Andreas Krause

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Comments ICML 2025 (Spotlight)

Criticality and Safety Margins for Reinforcement Learning

Alexander Grushin, Walt Woods, Alvaro Velasquez, Simon Khan

机构 * Galois, Inc.（Galois公司） ； University of Colorado Boulder（科罗拉多大学波德摩尔分校） ； Air Force Research Laboratory（空军研究实验室） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Comments 17 pages, 10 figures

Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models

Makesh Narsimhan Sreedhar, Traian Rebedea, Christopher Parisien

机构 * NVIDIA Santa Clara, CA（NVIDIA圣克拉拉分校） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI

Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training

Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Jiahao Xu, Tian Liang, Pinjia He, Zhaopeng Tu

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2025 main

Safety Modulation: Enhancing Safety in Reinforcement Learning through Cost-Modulated Rewards

Hanping Zhang, Yuhong Guo

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Hyperbolic Safety-Aware Vision-Language Models

Tobia Poppi, Tejaswi Kasarla, Pascal Mettes, Lorenzo Baraldi, Rita Cucchiara

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI

Comments CVPR 2025

Urban Safety Perception Through the Lens of Large Multimodal Models: A Persona-based Approach

Ciro Beneduce, Bruno Lepri, Massimiliano Luca

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.CY

Learning diverse attacks on large language models for robust red-teaming and safety tuning

Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, Moksh Jain

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.LG

Comments ICLR 2025

Vision-Based Generic Potential Function for Policy Alignment in Multi-Agent Reinforcement Learning

Hao Ma, Shijie Wang, Zhiqiang Pu, Siyao Zhao, Xiaolin Ai

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI、cs.LG

SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities

Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, Radha Poovendran

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI

Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities

Minh Duc Chu, Zihao He, Rebecca Dorn, Kristina Lerman

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.CL、cs.CY

GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model

Zhehua Zhou, Xuan Xie, Jiayang Song, Zhan Shu, Lei Ma

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.LG

Intelligent Approaches to Predictive Analytics in Occupational Health and Safety in India

Ritwik Raj Saxena

专题命中 安全训练 ：safety(title,abstract)；分类 cs.AI、cs.CY