大模型对齐与安全

Involuntary Jailbreak: On Self-Prompting Attacks

强制性越狱：关于自我提示攻击

Yangyang Guo, Yangyan Li, Mohan Kankanhalli

机构 * National University of Singapore（新加坡国立大学） ； Alibaba Group（阿里巴巴集团） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.AI

AI总结 研究揭示了大型语言模型中一种新的漏洞，通过简单提示策略可强制越狱多数主流模型，促使重新评估防护机制的鲁棒性。

Cisco Integrated AI Security and Safety Framework Report

思科集成AI安全与安全框架报告

Amy Chang, Tiffany Saade, Sanket Mendapara, Adam Swanda, Ankit Garg

机构 * Cisco AI Threat and Security Research（思科人工智能威胁与安全研究） 纠错

专题命中 越狱攻击 ：safety(title,abstract)；prompt injection(abstract)；AI safety(abstract)；分类 cs.AI

AI总结 本文提出思科集成AI安全与安全框架，旨在统一分类和操作化AI风险，涵盖安全与安全，适用于威胁识别、风险优先级排序等，具有全面性和扩展性。

sudoLLM: On Multi-role Alignment of Language Models

sudoLLM: 关于语言模型的多角色对齐

Soumadeep Saha, Akshay Chaturvedi, Joy Mahapatra, Utpal Garain

机构 * ISI Kolkata（印度Kolkata ISI研究所） ； IRIT Toulouse（法国图卢兹 IRIT 研究所） 纠错

专题命中 越狱攻击 ：alignment(title,abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL

AI总结 sudoLLM通过注入用户偏见信号，实现多角色对齐的LLM，提升安全性和抗攻击能力。

Comments Accepted to EMNLP 2025 (findings)

Journal ref In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 366-384, Suzhou, China. Association for Computational Linguistics

Unified Defense for Large Language Models against Jailbreak and Fine-Tuning Attacks in Education

Xin Yi, Yue Li, Dongsheng Shi, Linlin Wang, Xiaoling Wang, Liang He

机构 * Shanghai Institute of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China（教育人工智能研究所，华东师范大学，上海200062，中国） ； School of Computer Science and Technology, East China Normal University, Shanghai 200062, China（计算机科学与技术学院，华东师范大学，上海200062，中国） ； School of Computer Science（计算机科学学院） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?

Berk Atil, Rebecca J. Passonneau, Fred Morstatter

机构 * Penn State University（宾夕法尼亚州立大学） ； Information Sciences Institute, USC（信息科学研究所） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

Bag of Tricks for Subverting Reasoning-based Safety Guardrails

Shuo Chen, Zhen Han, Haokun Chen, Bailan He, Shengyun Si, Jingpei Wu, Philip Torr, Volker Tresp, Jindong Gu

机构 * LMU Munich（慕尼黑大学） ； Siemens（西门子） ； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心） ； Technical University of Berlin（柏林技术大学） ； Konrad Zuse School of Excellence in Reliable AI (relAI)（Konrad Zuse可靠性人工智能卓越学院） ； DFKI ； AWS AI（AWS人工智能） ； University of Oxford（牛津大学） 纠错

专题命中 越狱攻击 ：safety(title,abstract)；alignment(abstract)；jailbreak(abstract)；分类 cs.CL

Comments OpenAI Red-teaming Challenge Winner and Oral Presentation

Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Models

Shuang Liang, Zhihao Xu, Jialing Tao, Hui Xue, Xiting Wang

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.AI

Comments Withdrawn due to an accidental duplicate submission. This paper (arXiv:2510.15430) was unintentionally submitted as a new entry instead of a new version of our previous work (arXiv:2508.09201)

GuardNet: Graph-Attention Filtering for Jailbreak Defense in Large Language Models

Javad Forough, Mohammad Maheri, Hamed Haddadi

机构 * Imperial College London（伦敦帝国理工学院） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.LG

Bidirectional Intention Inference Enhances LLMs' Defense Against Multi-Turn Jailbreak Attacks

Haibo Tong, Dongcheng Zhao, Guobin Shen, Xiang He, Dachuan Lin, Feifei Zhao, Yi Zeng

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.AI

AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender

Weixiang Zhao, Jiahe Guo, Yulin Hu, Yang Deng, An Zhang, Xingyu Sui, Xinyang Han, Yanyan Zhao, Bing Qin, Tat-Seng Chua, Ting Liu

机构 * Harbin Institute of Technology（哈尔滨工业大学） ； Singapore Management University（新加坡管理大学） ； National University of Singapore（国立新加坡大学） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

Comments 19 pages, 6 figures, 10 tables

Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs

Yu Yan, Sheng Sun, Zhe Wang, Yijun Lin, Zenghao Duan, zhifei zheng, Min Liu, Zhiyi yin, Jianping Zhang

机构 * State Key Lab of Processors, Institute of Computing Technology, CAS（中国科学院计算技术研究所状态关键实验室） ； University of Chinese Academy of Sciences（中国科学院大学） ； People’s Public Security University of China（中国人民公安大学） ； Chinese University of Hong Kong（香港大学） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.AI

Prefill-level Jailbreak: A Black-Box Risk Analysis of Large Language Models

Yakai Li, Jiekang Hu, Weiduan Sang, Luping Ma, Dongsheng Nie, Weijuan Zhang, Aimin Yu, Yi Su, Qingjia Huang, Qihang Zhou

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所） ； School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院） ； Sinochem Energy High-Tech Co., Ltd.（中石化能源高科技有限公司） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.AI

SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks

Xiangman Li, Xiaodong Wu, Qi Li, Jianbing Ni, Rongxing Lu

机构 * Department of Electrical and Computer Engineering, Queen’s University（电气与计算机工程系，皇后大学） ； School of Computing, Queen’s University（计算机学院，皇后大学） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.LG

CCFC: Core & Core-Full-Core Dual-Track Defense for LLM Jailbreak Protection

Jiaming Hu, Haoyu Wang, Debarghya Mukherjee, Ioannis Ch. Paschalidis

机构 * Department. of Math & Statistics, Boston University（数学与统计学系，波士顿大学） ； Department. of Computer Science, University at Albany（计算机科学系，阿尔巴尼大学） ； Department. of ECE & Systems Eng., Department. of Biomedical Eng., Faculty of Computing & Data Sciences, Boston University（电子工程与系统工程系，生物医学工程系，计算与数据科学学院，波士顿大学） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；prompt injection(abstract)；分类 cs.AI

Comments 11 pages, 1 figure

MetaCipher: A Time-Persistent and Universal Multi-Agent Framework for Cipher-Based Jailbreak Attacks for LLMs

Boyuan Chen, Minghao Shao, Abdul Basit, Siddharth Garg, Muhammad Shafique

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.LG

Trust & Safety of LLMs and LLMs in Trust & Safety

Doohee You, Dan Chon

机构 * Google（谷歌） 纠错

专题命中 越狱攻击 ：safety(title,abstract)；jailbreak(abstract)；prompt injection(abstract)；分类 cs.AI

Comments 11 pages

HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States

Yilei Jiang, Xinyan Gao, Tianshuo Peng, Yingshui Tan, Xiaoyong Zhu, Bo Zheng, Xiangyu Yue

机构 * MMLab, The Chinese University of Hong Kong（中文大学香港大学） ； Future Lab, Alibaba Group（阿里集团未来实验室） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

Comments Accepted by ACL 2025 (Main)

Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models

Lei Jiang, Zixun Zhang, Zizhou Wang, Xiaobing Sun, Zhen Li, Liangli Zhen, Xiaohua Xu

机构 * University of Science and Technology of China（中国科学技术大学） ； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Institute of High Performance Computing, A*STAR, Singapore（新加坡A*STAR高性能计算研究所） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

Comments 15 pages, 9 figures

SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression

Yucheng Li, Surin Ahn, Huiqiang Jiang, Amir H. Abdi, Yuqing Yang, Lili Qiu

机构 * Microsoft Corporation（微软公司） ； University of Surrey（塞夫尔大学） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models

Youze Wang, Wenbo Hu, Yinpeng Dong, Jing Liu, Hanwang Zhang, Richang Hong

机构 * School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院） ； Tsinghua University（清华大学） ； Institute of automation, Chinese academy of science（中国科学院自动化研究所） ； School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.AI

Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learning

Weiyang Guo, Zesheng Shi, Zhuo Li, Yequan Wang, Xuebo Liu, Wenya Wang, Fangming Liu, Min Zhang, Jing Li

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；red teaming(abstract)；分类 cs.AI

Comments 21 pages, 8 figures

Latent-space adversarial training with post-aware calibration for defending large language models against jailbreak attacks

Xin Yi, Yue Li, Dongsheng Shi, Linlin Wang, Xiaoling Wang, Liang He

机构 * Lab of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China（教育人工智能实验室，东华大学，上海200062，中国） ； Shanghai Institute of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China（教育人工智能研究所，东华大学，上海200062，中国） ； School of Computer Science and Technology, East China Normal University, Shanghai 200062, China（计算机科学与技术学院，东华大学，上海200062，中国） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks

Guobin Shen, Dongcheng Zhao, Linghao Feng, Xiang He, Jihang Wang, Sicheng Shen, Haibo Tong, Yiting Dong, Jindong Li, Xiang Zheng, Yi Zeng

机构 * Beijing Institute of AI Safety and Governance (Beijing-AISI)（北京人工智能安全与治理研究院） ； Beijing Key Laboratory of Safe AI and Superalignment（北京安全人工智能与超对齐重点实验室） ； BrainCog Lab, CASIA（CASIA脑认知实验室） 纠错

专题命中 越狱攻击 ：safety(title,abstract)；alignment(abstract)；jailbreak(abstract)；分类 cs.CL

Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers

Viet-Anh Nguyen, Shiqian Zhao, Gia Dao, Runyi Hu, Yi Xie, Luu Anh Tuan

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs

Linbao Li, Yannan Liu, Daojing He, Yu Li

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization

Yidan Wang, Yanan Cao, Yubing Ren, Fang Fang, Zheng Lin, Binxing Fang

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所） ； School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院） ； Cyberspace Institute of Advanced Technology, Guangzhou University（广州大学网络空间技术研究院） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL

Comments Accepted to ACL 2025 (main)

h4rm3l: A language for Composable Jailbreak Attack Synthesis

Moussa Koulako Bala Doumbouya, Ananjan Nandi, Gabriel Poesia, Davide Ghilardi, Anna Goldie, Federico Bianchi, Dan Jurafsky, Christopher D. Manning

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments Accepted to the Thirteenth International Conference on Learning Representations (ICLR 2025)

Exploiting Prefix-Tree in Structured Output Interfaces for Enhancing Jailbreak Attacking

Yanzeng Li, Yunfan Xiong, Jialun Zhong, Jinchao Zhang, Jie Zhou, Lei Zou

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.AI

"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreak

Lingrui Mei, Shenghua Liu, Yiwei Wang, Baolong Bi, Jiayi Mao, Xueqi Cheng

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；red teaming(abstract)；分类 cs.CL

Comments COLING 2025

Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models

Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Ji-Rong Wen

专题命中 越狱攻击 ：alignment(title,abstract)；jailbreak(abstract)；harmlessness(abstract)；分类 cs.CL

Comments ECCV 2024 Oral