大模型对齐与安全 - arXivDaily 专题

2403.17710 2025-08-26 cs.CR cs.AI 83%

Optimization-based Prompt Injection Attack to LLM-as-a-Judge

Jiawen Shi, Zenghui Yuan, Yinuo Liu, Yue Huang, Pan Zhou, Lichao Sun, Neil Zhenqiang Gong

机构 * Huazhong University of Science and Technology（华中科技大学）； University of Notre Dame（圣母大学）； Lehigh University（莱文森大学）； Duke University（杜克大学）

专题命中越狱攻击：prompt injection(title,abstract)；jailbreak(abstract)；分类 cs.AI

Comments To appear in the Proceedings of The ACM Conference on Computer and Communications Security (CCS), 2024

2505.17066 2025-08-12 cs.CR cs.AI 83%

Improving LLM Outputs Against Jailbreak Attacks with Expert Model Integration

Tatia Tsmindashvili, Ana Kolkhidashvili, Dachi Kurtskhalia, Nino Maghlakelidze, Elene Mekvabishvili, Guram Dentoshvili, Orkhan Shamilov, Zaal Gachechiladze, Steven Saporta, David Dachi Choladze

专题命中越狱攻击：jailbreak(title,abstract)；prompt injection(abstract)；分类 cs.AI

Journal ref IEEE Access, vol. 13, pp. 134976-134988, Jul. 2025

2507.06043 2025-08-07 cs.CR cs.AI 83%

CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representations

Xiaohu Li, Yunfeng Ning, Zepeng Bao, Mayi Xu, Jianhao Chen, Tieyun Qian

机构 * School of Computer Science, Wuhan University, China（武汉大学计算机学院）； Zhongguancun Academy, Beijing, China（中关村学院）

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.AI

Comments Accepted to ACL 2025 (Findings), camera-ready version

2507.19227 2025-07-28 cs.CL 83%

Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generation

Yuanhe Zhang, Fangzhou Xie, Zhenhong Zhou, Zherui Li, Hao Chen, Kun Wang, Yufei Guo

专题命中越狱攻击：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL

2507.13474 2025-07-21 cs.CL 83%

Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers

Liang Lin, Zhihao Xu, Xuehai Tang, Shi Liu, Biyu Zhou, Fuqing Zhu, Jizhong Han, Songlin Hu

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）； Renmin University of China（中国人民大学）

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；分类 cs.CL

2504.03770 2025-07-18 cs.CR cs.AI 83%

JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model

Yi Nian, Shenzhe Zhu, Yuehan Qin, Li Li, Ziyi Wang, Chaowei Xiao, Yue Zhao

机构 * University of Southern California（南加州大学）； University of Toronto（多伦多大学）； University of Maryland（马里兰大学）； University of Wisconsin-Madison（威斯康星大学麦迪逊分校）

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.AI

2507.07341 2025-07-11 cs.AI cs.CR 83%

On the Impossibility of Separating Intelligence from Judgment: The Computational Intractability of Filtering for AI Alignment

Sarah Ball, Greg Gluch, Shafi Goldwasser, Frauke Kreuter, Omer Reingold, Guy N. Rothblum

机构 * Ludwig-Maximilians-Universität in Munich（慕尼黑路易斯-马克西米利安大学）； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）； University of California at Berkeley（加州大学伯克利分校）； JPSM University of Maryland（马里兰大学JPSM分校）； Stanford University（斯坦福大学）； Apple（苹果公司）

专题命中越狱攻击：alignment(title,abstract)；safety(abstract)；分类 cs.AI

2502.11853 2025-07-04 cs.LG 83%

StructTransform: A Scalable Attack Surface for Safety-Aligned Large Language Models

Shehel Yoosuf, Temoor Ali, Ahmed Lekssays, Mashael AlSabah, Issa Khalil

机构 * Qatar Computing Research Institute（卡塔尔计算研究所）

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；分类 cs.LG

2404.08793 2025-06-10 cs.CR cs.CL cs.HC 83%

JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models

Yingchaojie Feng, Zhizhang Chen, Zhining Kang, Sijia Wang, Haoyu Tian, Wei Zhang, Minfeng Zhu, Wei Chen

机构 * State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD与计算机图形学国家重点实验室）； Laboratory of Art and Archaeology Image (Zhejiang University), Ministry of Education（浙江大学艺术与考古图像实验室（教育部））； Hangzhou City University（杭州市大学）； Zhejiang University（浙江大学）

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

2505.24519 2025-06-02 cs.CV cs.CL 83%

AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders

Yuqi Zhang, Yuchun Miao, Zuchao Li, Liang Ding

机构 * School of Computer Science, Wuhan University（武汉大学计算机学院）； School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）； The University of Sydney（悉尼大学）

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

Comments 11 pages, 7 figures

2502.11084 2025-06-02 cs.CL 83%

Rewrite to Jailbreak: Discover Learnable and Transferable Implicit Harmfulness Instruction

Yuting Huang, Chengyuan Liu, Yifeng Feng, Yiquan Wu, Chao Wu, Fei Wu, Kun Kuang

机构 * Zhejiang University（浙江大学）

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

Comments 22 pages, 10 figures, accepted to ACL 2025 findings

2505.18864 2025-05-27 cs.CL 83%

Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework

Binhao Ma, Hanqing Guo, Zhengping Jay Luo, Rui Duan

机构 * Department of Computer Science University of Missouri-Kansas City（计算机科学系密苏里大学-康科特分校）； Department of Computer Science and Physics Rider University（计算机科学与物理系 Rider大学）； Department of Electrical and Computer Engineering University of Hawai’i at Mānoa（电气与计算机工程系夏威夷大学马诺亚分校）

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL

2505.16446 2025-05-23 cs.LG 83%

Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models

Zhaoxin Wang, Handing Wang, Cong Tian, Yaochu Jin

机构 * Xidian University（西安电子科技大学）； Westlake University（西湖大学）

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.LG

2412.17034 2025-05-22 cs.CL 83%

Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models

Lang Gao, Jiahui Geng, Xiangliang Zhang, Preslav Nakov, Xiuying Chen

机构 * MBZUAI ； Huazhong University of Science and Technology（华中科技大学）； University of Notre Dame（诺丁汉大学）

专题命中越狱攻击：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL

Comments 17 pages, 9 figures

2505.15406 2025-05-22 cs.SD cs.AI eess.AS 83%

Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models

Zirui Song, Qian Jiang, Mingxuan Cui, Mingzhe Li, Lang Gao, Zeyu Zhang, Zixiang Xu, Yanbo Wang, Chenxi Wang, Guangxian Ouyang, Zhenhao Chen, Xiuying Chen

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎德·本·扎耶德人工智能大学）； ByteDance（字节跳动）； Australia National University（澳大利亚国立大学）

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.AI

Comments We release AJailBench, including both static and optimized adversarial data, to facilitate future research: https://github.com/mbzuai-nlp/AudioJailbreak

2505.12060 2025-05-20 cs.CL 83%

Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement

Peng Ding, Jun Kuang, Zongyu Wang, Xuezhi Cao, Xunliang Cai, Jiajun Chen, Shujian Huang

机构 * National Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）； Meituan Inc., China（美团公司）

专题命中越狱攻击：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL

Comments Acccepted by ACL 2025 Findings, 21 pages, 9 figures, 14 tables

2411.04223 2025-05-13 cs.CL 83%

Diversity Helps Jailbreak Large Language Models

Weiliang Zhao, Daniel Ben-Levi, Wei Hao, Junfeng Yang, Chengzhi Mao

机构 * Columbia University（哥伦比亚大学）； Rutgers University（罗格斯大学）

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

2504.13562 2025-04-21 cs.CL 83%

DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification

Yu Li, Han Jiang, Zhihua Wei

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

2504.04976 2025-04-08 cs.CL 83%

A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models

Carlos Peláez-González, Andrés Herrera-Poyatos, Cristina Zuheros, David Herrera-Poyatos, Virilo Tejedor, Francisco Herrera

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL

Comments 21 pages, 5 figures

2504.03735 2025-04-08 cs.CR cs.AI cs.CL cs.CY cs.LG 83%

Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots

Erfan Shayegani, G M Shahariar, Sara Abdali, Lei Yu, Nael Abu-Ghazaleh, Yue Dong

专题命中越狱攻击：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.CY

2503.19134 2025-03-26 cs.CL cs.CR 83%

MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks

Wenhao You, Bryan Hooi, Yiwei Wang, Youke Wang, Zong Ke, Ming-Hsuan Yang, Zi Huang, Yujun Cai

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

2503.10872 2025-03-25 cs.CV cs.AI 83%

TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models

Xiangyu Yin, Yi Qi, Jinwei Hu, Zhen Chen, Yi Dong, Xingyu Zhao, Xiaowei Huang, Wenjie Ruan

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.AI

2401.09002 2025-03-19 cs.CL 83%

AttackEval: How to Evaluate the Effectiveness of Jailbreak Attacking on Large Language Models

Dong Shu, Chong Zhang, Mingyu Jin, Zihao Zhou, Lingyao Li, Yongfeng Zhang

专题命中越狱攻击：jailbreak(title,abstract)；prompt injection(abstract)；分类 cs.CL

Comments Accepted by ACM SIGKDD Explorations 2025

2404.01833 2025-02-27 cs.CR cs.AI 83%

Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack

Mark Russinovich, Ahmed Salem, Ronen Eldan

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.AI

Comments Accepted at USENIX Security 2025

2408.11182 2025-02-10 cs.CR cs.AI 83%

Hide Your Malicious Goal Into Benign Narratives: Jailbreak Large Language Models through Carrier Articles

Zhilong Wang, Haizhou Wang, Nanqing Luo, Lan Zhang, Xiaoyan Sun, Yebo Cao, Peng Liu

专题命中越狱攻击：jailbreak(title,abstract)；prompt injection(abstract)；分类 cs.AI

2410.01438 2025-02-04 cs.LG 83%

The Great Contradiction Showdown: How Jailbreak and Stealth Wrestle in Vision-Language Models?

Ching-Chia Kao, Chia-Mu Yu, Chun-Shien Lu, Chu-Song Chen

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.LG

2412.17544 2024-12-24 cs.AI 83%

Retention Score: Quantifying Jailbreak Risks for Vision Language Models

Zaitang Li, Pin-Yu Chen, Tsung-Yi Ho

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.AI

Comments 14 pages, 8 figures, AAAI 2025

Journal ref AAAI 2025

2401.06561 2024-12-17 cs.CL 83%

Intention Analysis Makes LLMs A Good Jailbreak Defender

Yuqi Zhang, Liang Ding, Lefei Zhang, Dacheng Tao

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

Comments COLING 2025

2412.08608 2024-12-12 cs.SD cs.AI cs.CR eess.AS 83%

AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models

Mintong Kang, Chejian Xu, Bo Li

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.AI

2411.09259 2024-12-10 cs.CV cs.CL 83%

Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey

Xuannan Liu, Xing Cui, Peipei Li, Zekun Li, Huaibo Huang, Shuhan Xia, Miaoxuan Zhang, Yueying Zou, Ran He

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

Comments ongoing work

1. 越狱攻击 1715 篇

Optimization-based Prompt Injection Attack to LLM-as-a-Judge

Improving LLM Outputs Against Jailbreak Attacks with Expert Model Integration

CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representations

Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generation

Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers

JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model

On the Impossibility of Separating Intelligence from Judgment: The Computational Intractability of Filtering for AI Alignment

StructTransform: A Scalable Attack Surface for Safety-Aligned Large Language Models

JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models

AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders

Rewrite to Jailbreak: Discover Learnable and Transferable Implicit Harmfulness Instruction

Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework

Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models

Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models

Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models

Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement

Diversity Helps Jailbreak Large Language Models

DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification

A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models

Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots

MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks

TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models

AttackEval: How to Evaluate the Effectiveness of Jailbreak Attacking on Large Language Models

Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack

Hide Your Malicious Goal Into Benign Narratives: Jailbreak Large Language Models through Carrier Articles

The Great Contradiction Showdown: How Jailbreak and Stealth Wrestle in Vision-Language Models?

Retention Score: Quantifying Jailbreak Risks for Vision Language Models

Intention Analysis Makes LLMs A Good Jailbreak Defender

AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models

Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey