大模型对齐与安全

When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models

Wei Cai, Shujuan Liu, Jian Zhao, Ziyan Shi, Yusheng Zhao, Yuchen Yuan, Tianle Zhang, Chi Zhang, Xuelong Li

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Safety Pretraining: Toward the Next Generation of Safe AI

Pratyush Maini, Sachin Goyal, Dylan Sam, Alex Robey, Yash Savani, Yiding Jiang, Andy Zou, Matt Fredrikson, Zacharcy C. Lipton, J. Zico Kolter

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.LG

Large Language Model's Multi-Capability Alignment in Biomedical Domain

Wentao Wu, Linqing Chen, Hanmeng Zhong, Weilei Wang

专题命中 安全训练 ：alignment(title,abstract)；safety(abstract)；分类 cs.AI

Measuring What Matters: A Framework for Evaluating Safety Risks in Real-World LLM Applications

Jia Yi Goh, Shaun Khoo, Nyx Iskandar, Gabriel Chua, Leanne Tan, Jessica Foo

专题命中 安全训练 ：safety(title,abstract)；AI safety(abstract)；分类 cs.CY

Multi-modal Representations for Fine-grained Multi-label Critical View of Safety Recognition

Britty Baby, Vinkle Srivastav, Pooja P. Jain, Kun Yuan, Pietro Mascagni, Nicolas Padoy

机构 * University of Strasbourg, CNRS, INSERM, ICube, UMR7357, Strasbourg, France（斯特拉斯堡大学，法国国家科学研究中心，法国国家卫生研究院，ICube，UMR7357，斯特拉斯堡，法国） ； Fondazione Policlinico Universitario A. Gemelli IRCCS, Università Cattolica del Sacro Cuore, Rome, Italy（A. Gemelli IRCCS大学医院，罗马，意大利） ； CAMP, Technische Universität München, Munich, Germany（慕尼黑技术大学，德国） ； Institute of Image-Guided Surgery, IHU Strasbourg, Strasbourg, France（影像引导手术研究所，斯特拉斯堡，法国） 纠错

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

The Structural Safety Generalization Problem

Julius Broomfield, Tom Gibbs, Ethan Kosak-Hine, George Ingebretsen, Tia Nasir, Jason Zhang, Reihaneh Iranmanesh, Sara Pieri, Reihaneh Rabbany, Kellin Pelrine

机构 * Georgia Tech（佐治亚理工学院） ； Mila ； UC Berkeley（加州大学伯克利分校） ； Stanford（斯坦福大学） ； MBZUAI（穆桑大学人工智能研究所） ； McGill（麦吉尔大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agents

Yuting Huang, Leilei Ding, Zhipeng Tang, Tianfu Wang, Xinrui Lin, Wuyang Zhang, Mingxiao Ma, Yanyong Zhang

机构 * University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Comments 16 pages, 10 figures

CURATe: Benchmarking Personalised Alignment of Conversational AI Assistants

Lize Alberts, Benjamin Ellis, Andrei Lupu, Jakob Foerster

专题命中 安全训练 ：alignment(title,abstract)；safety(abstract)；分类 cs.AI

Dynamic safety cases for frontier AI

Carmen Cârlan, Francesca Gomez, Yohan Mathew, Ketana Krishna, René King, Peter Gebauer, Ben R. Smith

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.CY

Comments 75 pages, 41 tables/figures

SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering

Zouying Cao, Yifei Yang, Hai Zhao

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Comments Extended version of paper accepted to AAAI 2025. 14 pages, 6 figures

WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs

Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, Nouha Dziri

专题命中 安全训练 ：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL

Comments NeurIPS 2024 Camera Ready. First two authors contributed equally. Third and fourth authors contributed equally

Self and Cross-Model Distillation for LLMs: Effective Methods for Refusal Pattern Alignment

Jie Li, Yi Liu, Chongyang Liu, Xiaoning Ren, Ling Shi, Weisong Sun, Yinxing Xue

专题命中 安全训练 ：alignment(title,abstract)；RLHF(abstract)；分类 cs.CL

Comments The method used in the paper has obvious problems and ambiguities. The security enhancement method we used cannot be considered distillation, but it is described as distillation in the paper, and the experiment lacks comparison and baseline, which has been criticized by many peers. In order to avoid further dissemination, we have decided to withdraw the paper

Safe to Serve: Aligning Instruction-Tuned Models for Safety and Helpfulness

Avinash Amballa, Durga Sandeep Saluru, Gayathri Akkinapalli, Abhishek Sureddy, Akshay Kumar Sureddy

专题命中 安全训练 ：safety(title,abstract)；DPO(abstract)；分类 cs.CL

Comments 18 pages

Locking Down the Finetuned LLMs Safety

Minjun Zhu, Linyi Yang, Yifan Wei, Ningyu Zhang, Yue Zhang

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Mitigating Exaggerated Safety in Large Language Models

Ruchira Ray, Ruchi Bhalani

专题命中 安全训练 ：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL

Comments 17 pages, 8 figures, 2 tables

Is it safe to cross? Interpretable Risk Assessment with GPT-4V for Safety-Aware Street Crossing

Hochul Hwang, Sunjae Kwon, Yekyung Kim, Donghyun Kim

专题命中 安全训练 ：safety(title,abstract)；trustworthy(abstract)；分类 cs.AI

Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models

Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, Timothy Hospedales

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.LG

Comments ICML 2024

MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance

Renjie Pi, Tianyang Han, Jianshu Zhang, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, Tong Zhang

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions

Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, James Zou

专题命中 安全训练 ：safety(title,abstract)；harmlessness(abstract)；分类 cs.CL

SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models

Bertie Vidgen, Nino Scherrer, Hannah Rose Kirk, Rebecca Qian, Anand Kannappan, Scott A. Hale, Paul Röttger

专题命中 安全训练 ：safety(title,abstract)；AI safety(abstract)；分类 cs.CL

MART: Improving LLM Safety with Multi-round Automatic Red-Teaming

Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, Yuning Mao

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Towards a framework for evaluating the safety, acceptability and efficacy of AI systems for health: an initial synthesis

Jessica Morley, Caroline Morton, Kassandra Karpathakis, Mariarosaria Taddeo, Luciano Floridi

专题命中 安全训练 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Frictive Policy Optimization for LLMs: Epistemic Intervention, Risk-Sensitive Control, and Reflective Alignment

摩擦性政策优化用于大语言模型：认知干预、风险敏感控制与反思对齐

James Pustejovsky, Nikhil Krishnaswamy

机构 * Brandeis University（布拉德雷大学） ； Colorado State University（科罗拉多州立大学） 纠错

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 本文提出摩擦性政策优化框架，通过干预管理认知与规范风险，引入摩擦干预分类和统一方法，提升模型的认知能力与对齐性。

Comments Frictive Policy Optimization; epistemic alignment; risk-sensitive control; LLM alignment; clarification and refusal; preference learning; trust regions; dialogue agents

Being Considerate as a Pathway Towards Pluralistic Alignment for Agentic AI

Parand A. Alamdari, Toryn Q. Klassen, Rodrigo Toro Icarte, Sheila A. McIlraith

专题命中 安全训练 ：alignment(title,abstract)；分类 cs.AI、cs.CY、cs.LG

Comments Pluralistic Alignment Workshop at NeurIPS 2024

From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoring

Yang Li, Qiang Sheng, Yehan Yang, Xueyao Zhang, Juan Cao

机构 * Yang Li Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences（中国科学院计算技术研究所、中国科学院自动化研究所、中国科学院大学） ； Qiang Sheng Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所、中国科学院自动化研究所） ； Yehan Yang Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所、中国科学院自动化研究所） ； Xueyao Zhang The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Juan Cao Media Synthesis and Forensics Lab, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences（中国科学院计算技术研究所、中国科学院自动化研究所、中国科学院大学） 纠错

专题命中 安全训练 ：alignment(abstract)；DPO(abstract)；safety(abstract)；harmlessness(abstract)

Comments NeurIPS 2025 Accepted Paper

Regime-Conditional Verification: Correctness Estimation for Adapting and Monitoring Safety Classifiers

条件 regime 验证：安全分类器适配与监控的正确性估计

Thiago Sandoval, Ufuk Topcu

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 本文提出 Regime-Conditional Verification（RCV），通过轻量级封装器适配安全分类器，在不重新训练的情况下提升策略遵守度，可检测分布偏移并仅在必要时微调，在多分类器、数据集及部署场景中表现优异。

Comments 16 pages including technical appendix, 6 figures

SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectives

SteeringSafety：针对大语言模型在多安全视角下的表征引导基准测试

Vincent Siu, Nicholas Crispino, David Park, Nathan W. Henry, Zhun Wang, Yang Liu, Dawn Song, Chenguang Wang

机构 * University of California, Santa Cruz（加州大学圣克鲁兹分校） ； Washington University in St. Louis（华盛顿大学圣路易斯分校） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 本研究推出SteeringSafety基准，测试DIM等引导方法在3款大模型的9种安全视角表现，发现方法与模型、视角的匹配影响性能，且存在多视角纠缠问题，需多安全角度评估引导方法。

Comments Accepted at ICML 2026

RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion

RedDiffuser：通过强化扩散审计多模态安全故障

Ruofan Wang, Xingjun Ma

机构 * Fudan University（复旦大学） 纠错

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)

AI总结 研究多模态系统在有害上下文暴露下的安全审计，提出RedDiffuser框架通过扩散模型生成视觉输入，揭示隐藏的安全漏洞，实验显示VLMs在部分有毒文本与视觉上下文结合时存在广泛安全问题。

Large Language Models for Cancer Communication: Evaluating Linguistic Quality, Safety, and Accessibility in Generative AI

Agnik Saha, Victoria Churchill, Anny D. Rodriguez, Ugur Kursuncu, Muhammed Y. Idris

机构 * Georgia State University（佐治亚州立大学） ； Morehouse School of Medicine（莫尔豪斯医学院） 纠错

专题命中 安全训练 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Journal ref JMIR Cancer 2026;12:e82971

Mask2Shield: Strengthening LLM Safety against Neuron-Pruning Attacks

Mask2Shield：增强大语言模型抵御神经元剪枝攻击的安全性

Ying JinCheng, Minghui Xu, Yinhao Xiao, Xiuzhen Cheng, Wencheng Yang

专题命中 安全训练 ：safety(title,abstract)；alignment(abstract)

AI总结 研究针对大语言模型神经元剪枝攻击问题，提出Mask2Shield方法，通过掩码前向对齐训练模型，减少对可移除安全神经元集的依赖，降低针对性剪枝攻击成功率，同时保持能力基准。