大模型对齐与安全 - arXivDaily 专题

2510.02609 2025-11-12 cs.SE 67%

RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents

Chengquan Guo, Chulin Xie, Yu Yang, Zhaorun Chen, Zinan Lin, Xander Davies, Yarin Gal, Dawn Song, Bo Li

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

2510.01342 2025-10-10 cs.CR 67%

Fine-Tuning Jailbreaks under Highly Constrained Black-Box Settings: A Three-Pronged Approach

Xiangfang Li, Yu Wang, Bo Li

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

2504.00218 2025-10-10 cs.MA cs.AI cs.CL cs.LG 67%

$\textit{Agents Under Siege}$: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks

Rana Muhammad Shahroz Khan, Zhen Tan, Sukwon Yun, Charles Fleming, Tianlong Chen

机构 * University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）； Arizona State University（亚利桑那州立大学）； Cisco（思科）

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2509.23281 2025-09-30 cs.RO 67%

Preventing Robotic Jailbreaking via Multimodal Domain Adaptation

Francesco Marchiori, Rohan Sinha, Christopher Agia, Alexander Robey, George J. Pappas, Mauro Conti, Marco Pavone

机构 * University of Padova（帕多瓦大学）； Stanford University（斯坦福大学）； Carnegie Mellon University（卡内基梅隆大学）； University of Pennsylvania（宾夕法尼亚大学）； Örebro University（奥雷布罗大学）； NVIDIA Research（NVIDIA研究）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

Comments Project page: https://j-dapt.github.io/. 9 pages, 6 figures

2508.13739 2025-09-25 cs.CV 67%

Enhancing Targeted Adversarial Attacks on Large Vision-Language Models via Intermediate Projector

Yiming Cao, Yanjie Li, Kaisheng Liang, Bin Xiao

专题命中越狱攻击：alignment(abstract)；safety(abstract)

2502.13329 2025-09-24 cs.CL cs.AI cs.LG 67%

Language Models Can Predict Their Own Behavior

Dhananjay Ashok, Jonathan May

机构 * Information Sciences Institute（信息科学研究所）； University of Southern California（南加州大学）

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Presented at the Thirty-Ninth Annual Conference on Neural Information Processing Systems (2025)

2509.00391 2025-09-03 cs.CL cs.AI cs.CR cs.LG 67%

The Resurgence of GCG Adversarial Attacks on Large Language Models

Yuting Tan, Xuying Li, Zhuo Li, Huizhen Shu, Peikang Hu

机构 * HydroX AI San Jose CA USA（HydroX AI）

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 pages, 5 figures

2504.01308 2025-08-05 cs.CV 67%

Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks

Jiawei Wang, Yushen Zuo, Yuanjun Chai, Zhendong Liu, Yicheng Fu, Yichun Feng, Kin-Man Lam

机构 * University of Science and Technology of China（中国科学技术大学）； The Hong Kong Polytechnic University（香港理工大学）； University of Washington（华盛顿大学）； Nanjing University（南京大学）； Stanford University（斯坦福大学）； University of Chinese Academy of Sciences（中国科学院大学）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

Comments ICCV 2025

2506.17052 2025-06-23 cs.LG cs.AI cs.CL 67%

From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers

Jingtong Su, Julia Kempe, Karen Ullrich

机构 * NYU（纽约大学）； Meta AI ； FAIR

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2502.10794 2025-06-18 cs.CV 67%

Distraction is All You Need for Multimodal Large Language Model Jailbreaking

Zuopeng Yang, Jiluan Fan, Anli Yan, Erdun Gao, Xin Lin, Tao Li, Kanghua Mo, Changyu Dong

机构 * Guangzhou University（广州大学）； Shanghai Jiao Tong University（上海交通大学）； Australian Institute for Machine Learning, The University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学）

专题命中越狱攻击：alignment(abstract)；safety(abstract)

Comments CVPR 2025 highlight

2505.19684 2025-05-29 cs.CV 67%

VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Models

Bingrui Sima, Linhua Cong, Wenxuan Wang, Kun He

机构 * Huazhong University of Science and Technology（华中科技大学）； Hong Kong University of Science and Technology（香港科技大学）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

2501.18280 2025-05-20 cs.CL cs.AI cs.LG cs.NE 67%

Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models

Haoyu Liang, Youran Sun, Yunfeng Cai, Jun Zhu, Bo Zhang

机构 * Dept. of Comp. Sci. and Tech., Inst. for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, 100084, China（计算机科学与技术系、人工智能研究所、清华-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学、北京）； Department of Mathematical Sciences, Tsinghua University（数学科学系、清华大学）； BIMSA, Beijing, China（BIMSA、北京、中国）

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.06643 2025-05-13 cs.CR 67%

Practical Reasoning Interruption Attacks on Reasoning Large Language Models

Yu Cui, Cong Zuo

专题命中越狱攻击：jailbreak(abstract)；prompt injection(abstract)

2504.21680 2025-05-01 cs.CR 67%

Hoist with His Own Petard: Inducing Guardrails to Facilitate Denial-of-Service Attacks on Retrieval-Augmented Generation of LLMs

Pan Suo, Yu-Ming Shang, San-Chuan Guo, Xi Zhang

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

Comments 11 pages, 6 figures. This work will be submitted to the IEEE for possible publication

2503.11750 2025-03-18 cs.CV cs.CR 67%

Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization

Shuyang Hao, Yiwei Wang, Bryan Hooi, Jun Liu, Muhao Chen, Zi Huang, Yujun Cai

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

2407.00075 2025-03-03 cs.AI cs.CL cs.CR cs.LG 67%

Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference

Anton Xue, Avishree Khare, Rajeev Alur, Surbhi Goel, Eric Wong

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

2502.08374 2025-02-13 cs.CV 67%

AdvSwap: Covert Adversarial Perturbation with High Frequency Info-swapping for Autonomous Driving Perception

Yuanhao Huang, Qinfan Zhang, Jiandong Xing, Mengyue Cheng, Haiyang Yu, Yilong Ren, Xiao Xiong

专题命中越狱攻击：safety(abstract)；AI safety(abstract)

Comments 27th IEEE International Conference on Intelligent Transportation Systems (ITSC)

2502.06390 2025-02-12 cs.CV 67%

When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs

Aobotao Dai, Xinyu Ma, Lei Chen, Songze Li, Lin Wang

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

2502.01154 2025-02-04 cs.CL cs.AI cs.CR cs.LG 67%

Jailbreaking with Universal Multi-Prompts

Yu-Ling Hsu, Hsuan Su, Shang-Tse Chen

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by NAACL Findings 2025

2502.00580 2025-02-04 cs.CR cs.AI cs.CL cs.CY 67%

Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation

Stuart Armstrong, Matija Franklin, Connor Stevens, Rebecca Gorman

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.CY

2501.16378 2025-01-29 cs.LG cs.AI cs.CL cs.CV 67%

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update

Qing Li, Jiahui Geng, Zongxiong Chen, Kun Song, Lei Ma, Fakhri Karray

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.16822 2024-12-12 cs.CL cs.AI cs.LG 67%

Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts

Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäschel, Roberta Raileanu

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.12329 2024-12-10 cs.CL cs.AI cs.CR cs.LG 67%

Query-Based Adversarial Prompt Generation

Jonathan Hayase, Ema Borevkovic, Nicholas Carlini, Florian Tramèr, Milad Nasr

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

2410.09804 2024-11-28 cs.CR cs.AI cs.CL cs.LG cs.NE 67%

BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Models

Xinyuan Wang, Victor Shea-Jay Huang, Renmiao Chen, Hao Wang, Chengwei Pan, Lei Sha, Minlie Huang

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

2401.17263 2024-11-11 cs.LG cs.AI cs.CL cs.CV 67%

Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks

Andy Zhou, Bo Li, Haohan Wang

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024 Spotlight; code available at https://github.com/lapisrocks/rpo

2312.02119 2024-11-01 cs.LG cs.AI cs.CL cs.CR stat.ML 67%

Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, Amin Karbasi

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted for presentation at NeurIPS 2024. Code: https://github.com/RICommunity/TAP

2406.01288 2024-10-31 cs.CL cs.AI cs.CR cs.LG 67%

Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses

Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Jing Jiang, Min Lin

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024

2402.17840 2024-10-08 cs.CL cs.AI cs.CR cs.LG 67%

Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, Himabindu Lakkaraju

专题命中越狱攻击：prompt injection(abstract)；分类 cs.CL、cs.AI、cs.LG

2406.20053 2024-07-01 cs.CR cs.AI cs.CL cs.LG 67%

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation

Danny Halawi, Alexander Wei, Eric Wallace, Tony T. Wang, Nika Haghtalab, Jacob Steinhardt

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 22 pages

2406.08725 2024-06-14 cs.CR 67%

RL-JACK: Reinforcement Learning-powered Black-box Jailbreaking Attack against LLMs

Xuan Chen, Yuzhou Nie, Lu Yan, Yunshu Mao, Wenbo Guo, Xiangyu Zhang

专题命中越狱攻击：alignment(abstract)；safety(abstract)

1. 越狱攻击 1717 篇

RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents

Fine-Tuning Jailbreaks under Highly Constrained Black-Box Settings: A Three-Pronged Approach

$\textit{Agents Under Siege}$: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks

Preventing Robotic Jailbreaking via Multimodal Domain Adaptation

Enhancing Targeted Adversarial Attacks on Large Vision-Language Models via Intermediate Projector

Language Models Can Predict Their Own Behavior

The Resurgence of GCG Adversarial Attacks on Large Language Models

Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks

From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers

Distraction is All You Need for Multimodal Large Language Model Jailbreaking

VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Models

Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models

Practical Reasoning Interruption Attacks on Reasoning Large Language Models

Hoist with His Own Petard: Inducing Guardrails to Facilitate Denial-of-Service Attacks on Retrieval-Augmented Generation of LLMs

Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization

Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference

AdvSwap: Covert Adversarial Perturbation with High Frequency Info-swapping for Autonomous Driving Perception

When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs

Jailbreaking with Universal Multi-Prompts

Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update

Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts

Query-Based Adversarial Prompt Generation

BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Models

Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks

Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses

Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation

RL-JACK: Reinforcement Learning-powered Black-box Jailbreaking Attack against LLMs