大模型对齐与安全

PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flips

Zachary Coalson, Jeonghyun Woo, Chris S. Lin, Joyce Qu, Yu Sun, Shiyang Chen, Lishan Yang, Gururaj Saileshwar, Prashant Nair, Bo Fang, Sanghyun Hong

机构 * Oregon State University（俄勒冈州立大学） ； University of British Columbia（不列颠哥伦比亚大学） ； University of Toronto（多伦多大学） ； George Mason University（乔治·梅森大学） ； Rutgers University（罗格斯大学） ； University of Texas at Arlington（德克萨斯大学阿灵顿分校） 纠错

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.LG

Comments Pre-print

FC-Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowcharts

Ziyi Zhang, Zhen Sun, Zongmin Zhang, Jihui Guo, Xinlei He

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） 纠错

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

Comments Accepted to Findings of EMNLP 2025

Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

Huizhen Shu, Xuying Li, Qirui Wang, Yuji Kosuga, Mengqiu Tian, Zhuo Li

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Models

Aaron Dharna, Cong Lu, Jeff Clune

专题命中 越狱攻击 ：safety(abstract)；jailbreak(abstract)；AI safety(abstract)；分类 cs.AI、cs.LG

Comments 67 pages, accepted to RLC 2025

Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial States

Xin Wei Chia, Swee Liang Wong, Jonathan Pan

专题命中 越狱攻击 ：safety(abstract)；jailbreak(abstract)；prompt injection(abstract)；分类 cs.AI、cs.LG

Comments 4 figures

Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs

Haoming Yang, Ke Ma, Xiaojun Jia, Yingfei Sun, Qianqian Xu, Qingming Huang

机构 * School of Electronic, Electrical and communication Engineering, UCAS, Beijing.（电子电气与通信工程学院，中国科学院大学，北京） ； Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, CAS, Beijing.（智能信息处理重点实验室，计算技术研究所，中国科学院，北京） ； School of Computer Science and Technology, UCAS, Beijing.（计算机科学与技术学院，中国科学院大学，北京） ； Key Laboratory of Big Data Mining and Knowledge management, UCAS, Beijing（大数据挖掘与知识管理重点实验室，中国科学院大学，北京） ； Nanyang Technological University, Singapore.（南洋理工大学，新加坡） ； Shenzhen Campus of Sun Yat-sen University, Shenzhen.（孙中山大学深圳校区，深圳） 纠错

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Shouwei Ruan, Jialing Tao, YueFeng Chen, Hui Xue, Xingxing Wei

机构 * Institution1（机构1） ； Institution2（机构2） 纠错

专题命中 越狱攻击 ：safety(abstract)；jailbreak(abstract)；red teaming(abstract)；分类 cs.CL、cs.AI

Comments ICCV2025

RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking

Yifan Jiang, Kriti Aggarwal, Tanmay Laud, Kashif Munir, Jay Pujara, Subhabrata Mukherjee

机构 * Hippocratic AI（希波克拉底AI） ； Information Sciences Institute, University of Southern California（信息科学研究所，南加州大学） 纠错

专题命中 越狱攻击 ：safety(abstract)；jailbreak(abstract)；red teaming(abstract)；分类 cs.CL、cs.LG

Comments Accepted in ACL 2025 Findings

QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Language

Qingsong Zou, Jingyu Xiao, Qing Li, Zhi Yan, Yuhang Wang, Li Xu, Wenxuan Wang, Kuofeng Gao, Ruoyu Li, Yong Jiang

机构 * Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院） ； Pengcheng Laboratory（鹏城实验室） ； The Chinese University of Hong Kong（香港中文大学） ； Jilin University（吉林大学） ； Southwest University（西南大学） ； University of Electronic Science and Technology of China（电子科技大学） ； Shenzhen University（深圳大学） 纠错

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments To appear in ACL 2025

Adversarial Attack on Large Language Models using Exponentiated Gradient Descent

Sajib Biswas, Mao Nishino, Samuel Jacob Chacko, Xiuwen Liu

机构 * Department of Computer Science, Florida State University（计算机科学系，佛罗里达州立大学） ； Department of Mathematics, Florida State University（数学系，佛罗里达州立大学） 纠错

专题命中 越狱攻击 ：RLHF(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.LG

Comments Accepted to International Joint Conference on Neural Networks (IJCNN) 2025

Towards Universal and Black-Box Query-Response Only Attack on LLMs with QROA

Hussein Jawad, Yassine Chenik, Nicolas J. -B. Brunel

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.LG

Graph of Attacks: Improved Black-Box and Interpretable Jailbreaks for LLMs

Mohammad Akbar-Tajari, Mohammad Taher Pilehvar, Mohammad Mahmoody

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 1 figure, 6 tables

Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM

Shaoqing Zhang, Zhuosheng Zhang, Kehai Chen, Rongxiang Weng, Muyun Yang, Tiejun Zhao, Min Zhang

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments AAAI 2025

Geneshift: Impact of different scenario shift on Jailbreaking LLM

Tianyi Wu, Zhiwei Xue, Yue Liu, Jiaheng Zhang, Bryan Hooi, See-Kiong Ng

专题命中 越狱攻击 ：safety(abstract)；jailbreak(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI

Na'vi or Knave: Jailbreaking Language Models via Metaphorical Avatars

Yu Yan, Sheng Sun, Junqi Tong, Min Liu, Qi Li

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Our study requires further in-depth research to ensure the comprehensiveness and adequacy of the methodology

FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts

Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, Xiaoyun Wang

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments AAAI 2025 (Oral)

ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templates

Fengqing Jiang, Zhangchen Xu, Luyao Niu, Bill Yuchen Lin, Radha Poovendran

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

Comments This paper is accepted to AAAI 2025

Unleashing the Unseen: Harnessing Benign Datasets for Jailbreaking Large Language Models

Wei Zhao, Zhe Li, Yige Li, Jun Sun

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Playing Language Game with LLMs Leads to Jailbreaking

Yu Peng, Zewen Long, Fangming Dong, Congyi Li, Shu Wu, Kai Chen

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

DROJ: A Prompt-Driven Attack against Large Language Models

Leyang Hu, Boran Wang

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers

Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, Cho-Jui Hsieh

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Jailbreaking Black Box Large Language Models in Twenty Queries

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, Eric Wong

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition

Ziyang Zhang, Qizhen Zhang, Jakob Foerster

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Accepted at ICML 20224

Evaluating and Mitigating Linguistic Discrimination in Large Language Models

Guoliang Dong, Haoyu Wang, Jun Sun, Xinyu Wang

专题命中 越狱攻击 ：alignment(abstract)；safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Benign-to-Toxic Jailbreaking: Inducing Harmful Responses from Harmless Prompts

Hee-Seon Kim, Minbeom Kim, Wonjun Lee, Kihyun Kim, Changick Kim

机构 * Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院） 纠错

专题命中 越狱攻击 ：jailbreak(abstract,comments)；alignment(abstract)；safety(abstract)；分类 cs.AI

Comments LVLM, Jailbreak

LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper

Daoyuan Wu, Shuai Wang, Yang Liu, Ning Liu

专题命中 越狱攻击 ：jailbreak(abstract,comments)；alignment(abstract)；safety(abstract)；分类 cs.AI

Comments Fixed the bibliography reference issue in our LLM jailbreak defense vision paper submitted on 24 Feb 2024

Fine-grained CLIP fine-tuning with self-annotated region alignment

基于自标注区域对齐的细粒度CLIP微调

Chenyang Zhao, Wei Lin, Janet H. Hsiao, Antoni B. Chan

机构 * City University of Hong Kong（香港城市大学） ； Hong Kong University of Science & Technology（香港科技大学） 纠错

专题命中 越狱攻击 ：alignment(title,abstract)

AI总结 研究针对CLIP在细粒度表示上的局限，提出SFF-CLIP方法，仅用图像-文本对输入，通过运行时区域-短语对齐方案提升其细粒度表示能力，在相关任务中性能显著提升且维持原CLIP图像级任务表现。

SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills

SkillSentry：用于智能体技能动态安全测试的自适应蜜罐世界

Nizhang Li, Zonghao Ying, Xiangfan Wu, Zonglei Jing, Xixun Lin, Hao Zhang, Wenxin Zhang, Jiaye Lin, Quanchen Zou, Xiangzheng Zhang

专题命中 越狱攻击 ：safety(title,abstract)

AI总结 SkillSentry是基于自适应蜜罐世界的动态安全测试框架，可测试大语言模型智能体外部技能的条件性有害行为，在基准测试及规避场景下性能优于基线，代码公开。

A hardware-safety-gated system for LLM-written native ARTIQ control code on a trapped-ion platform

基于硬件安全门控的LLM编写原生ARTIQ控制代码系统在离子阱平台上的应用

Duanyang Wang, Lu Qi, Yuanheng Xie, Norbert M. Linke, Kenneth R. Brown

专题命中 越狱攻击 ：safety(title,abstract)

AI总结 提出一种硬件安全门控系统，通过令牌授权机制确保LLM代理在离子阱实验中安全自主地编写和执行控制代码，并在实验中验证了其有效性和可移植性。

Comments 15 pages, 5 figures

Towards Identification and Intervention of Safety-Critical Parameters in Large Language Models

向大规模语言模型中安全关键参数的识别与干预迈进

Weiwei Qi, Zefeng Wu, Tianhang Zheng, Zikang Zhang, Xiaojun Jia, Zhan Qin, Kui Ren

专题命中 越狱攻击 ：safety(title,abstract)

AI总结 本文提出ESI框架量化不同参数对LLM安全的影响，揭示不同架构中安全关键模式，并引入SET和SPA干预方法提升安全性和稳定性。

Comments 20 pages, 6 figures, 8 tables