大模型对齐与安全 - arXivDaily 专题

2503.05264 2025-03-10 cs.CR cs.AI 70%

Jailbreaking is (Mostly) Simpler Than You Think

Mark Russinovich, Ahmed Salem

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.AI

2501.18632 2025-03-05 cs.CR cs.CL 70%

Towards Safe AI Clinicians: A Comprehensive Study on Large Language Model Jailbreaking in Healthcare

Hang Zhang, Qian Lou, Yanshan Wang

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL

2503.00224 2025-03-04 cs.CL 70%

À la recherche du sens perdu: your favourite LLM might have more to say than you can understand

K. O. T. Erziev

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

Comments 22 pages

2502.13175 2025-02-26 cs.CR cs.AI cs.RO 70%

Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks

Wenpeng Xing, Minghao Li, Mohan Li, Meng Han

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2502.17254 2025-02-25 cs.LG 70%

REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective

Simon Geisler, Tom Wollschläger, M. H. I. Abdalla, Vincent Cohen-Addad, Johannes Gasteiger, Stephan Günnemann

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.LG

Comments 30 pages, 6 figures, 15 tables

2501.14073 2025-02-19 cs.CL 70%

LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language

Yubin Ge, Neeraja Kirtane, Hao Peng, Dilek Hakkani-Tür

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

Comments 16 pages

2502.05772 2025-02-11 cs.CV cs.AI 70%

Effective Black-Box Multi-Faceted Attacks Breach Vision Large Language Model Guardrails

Yijun Yang, Lichao Wang, Xiao Yang, Lanqing Hong, Jun Zhu

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI

2410.17922 2025-02-11 cs.AI 70%

Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Models

Weidi Luo, He Cao, Zijing Liu, Yu Wang, Aidan Wong, Bing Feng, Yuan Yao, Yu Li

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2501.17715 2025-01-30 cs.CL 70%

RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts

Eujeong Choi, Younghun Jeong, Soomin Kim, Won Ik Cho

专题命中越狱攻击：jailbreak(abstract)；red teaming(abstract)；分类 cs.CL

Comments PACLIC 38

2411.19335 2024-12-20 cs.CR cs.AI 70%

PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning

Shenghui Li, Edith C. -H. Ngai, Fanghua Ye, Thiemo Voigt

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI

2412.06181 2024-12-10 cs.CR cs.AI 70%

Enhancing Adversarial Resistance in LLMs with Recursion

Bryan Li, Sounak Bagchi, Zizhan Wang

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.AI

2311.03191 2024-12-02 cs.LG cs.CR 70%

DeepInception: Hypnotize Large Language Model to Be Jailbreaker

Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, Bo Han

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.LG

2411.17693 2024-11-27 cs.CL 70%

Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats

Jiaxin Wen, Vivek Hebbar, Caleb Larson, Aryan Bhatt, Ansh Radhakrishnan, Mrinank Sharma, Henry Sleight, Shi Feng, He He, Ethan Perez, Buck Shlegeris, Akbir Khan

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL

2411.07494 2024-11-13 cs.CL 70%

Rapid Response: Mitigating LLM Jailbreaks with a Few Examples

Alwin Peng, Julian Michael, Henry Sleight, Ethan Perez, Mrinank Sharma

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

2410.13691 2024-11-12 cs.RO cs.AI 70%

Jailbreaking LLM-Controlled Robots

Alexander Robey, Zachary Ravichandran, Vijay Kumar, Hamed Hassani, George J. Pappas

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2410.16950 2024-10-23 cs.CR cs.AI 70%

Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In

Itay Nakash, George Kour, Guy Uziel, Ateret Anaby-Tavor

专题命中越狱攻击：safety(abstract)；prompt injection(abstract)；分类 cs.AI

2405.17894 2024-10-15 cs.CV cs.AI 70%

White-box Multimodal Jailbreaks Against Large Vision-Language Models

Ruofan Wang, Xingjun Ma, Hanxu Zhou, Chuanjun Ji, Guangnan Ye, Yu-Gang Jiang

专题命中越狱攻击：alignment(abstract)；jailbreak(abstract)；分类 cs.AI

2410.09040 2024-10-14 cs.CL 70%

AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation

Zijun Wang, Haoqin Tu, Jieru Mei, Bingchen Zhao, Yisen Wang, Cihang Xie

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.CL

2406.03589 2024-09-26 cs.CL 70%

Ranking Manipulation for Conversational Search Engines

Samuel Pfrommer, Yatong Bai, Tanmay Gautam, Somayeh Sojoudi

专题命中越狱攻击：safety(abstract)；prompt injection(abstract)；分类 cs.CL

Comments 2024 Conference on Empirical Methods in Natural Language Processing (Main)

2402.18104 2024-06-11 cs.CR cs.AI 70%

Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction

Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, Kai Chen

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

2405.16567 2024-05-29 cs.AI cs.CR 70%

Automatic Jailbreaking of the Text-to-Image Generative AI Systems

Minseon Kim, Hyomin Lee, Boqing Gong, Huishuai Zhang, Sung Ju Hwang

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments Under review

2310.15851 2024-03-25 cs.CL 70%

Self-Guard: Empower the LLM to Safeguard Itself

Zezhong Wang, Fangkai Yang, Lu Wang, Pu Zhao, Hongru Wang, Liang Chen, Qingwei Lin, Kam-Fai Wong

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

2403.09513 2024-03-15 cs.CR cs.AI 70%

AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting

Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, Chaowei Xiao

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.AI

Comments Multimodal Large Language Models Defense, 25 Pages

2402.18649 2024-03-01 cs.CR cs.AI 70%

A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems

Fangzhou Wu, Ning Zhang, Somesh Jha, Patrick McDaniel, Chaowei Xiao

专题命中越狱攻击：alignment(abstract)；safety(abstract)；分类 cs.AI

2310.14303 2023-11-14 cs.CL 70%

Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases

Rishabh Bhardwaj, Soujanya Poria

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；分类 cs.CL

Comments Under Review

2302.06455 2023-02-14 cs.AI cs.FL 70%

Incremental Satisfiability Modulo Theory for Verification of Deep Neural Networks

Pengfei Yang, Zhiming Chi, Zongxin Liu, Mengyu Zhao, Cheng-Chao Huang, Shaowei Cai, Lijun Zhang

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.AI

2009.00097 2020-09-02 cs.LG cs.CR cs.CV stat.ML 70%

Adversarial Eigen Attack on Black-Box Models

Linjun Zhou, Peng Cui, Yinan Jiang, Shiqiang Yang

专题命中越狱攻击：safety(abstract)；AI safety(abstract)；分类 cs.LG

2502.01576 2026-06-04 cs.CV 69%

Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models

Robust-LLaVA：大规模鲁棒图像编码器对多模态大语言模型的有效性

Hashmat Shadab Malik, Fahad Shamshad, Muzammal Naseer, Karthik Nandakumar, Fahad Khan, Salman Khan

机构 * Mohamed bin Zayed University of AI（Mohamed bin Zayed人工智能大学）； Khalifa University（卡利法大学）； Michigan State University（密歇根州立大学）； Australian National University（澳大利亚国立大学）

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)；trustworthy(comments)

AI总结本文提出利用大规模对抗预训练的图像分类模型替代CLIP编码器，以增强多模态大语言模型对视觉对抗扰动的鲁棒性，在无需额外对抗训练的情况下，在视觉问答、图像描述和越狱攻击任务中取得显著鲁棒性提升。

Comments Accepted at Trustworthy FMs Workshop Trust Before Use: Building Foundation Models that You Can Trust (ICCVW) 2025

2608.16032 2026-08-18 cs.CR 新提交 67%

Proof-of-Execution Memory: Defending LLM Agents Against Forged-Reasoning Attacks by Verifying What Actually Happened

执行证明内存：通过验证实际发生的内容防御大语言模型智能体的伪造推理攻击

Md Habibur Rahman, Jaeho Kim

专题命中越狱攻击：safety(abstract)；trustworthy(abstract)

AI总结该研究针对大语言模型智能体的伪造推理攻击，提出执行证明内存（PoEM）防御方案，通过维护防篡改的HMAC链式分类账验证实际执行步骤，使攻击成功率降至0%，且误报率极低、开销微小。

Comments 8 pages, 6 figures, 5 tables. Code: https://github.com/bithabib/ai_security

2605.29237 2026-08-18 cs.CR 版本更新 67%

Evolving Skill-Structured Attack Memory Enhances LLM Jailbreaking

演化技能结构攻击记忆增强大语言模型越狱

Junke Zhang, Jianwei Wang, Sishuo Chen, Yizhang He, Qingshuai Feng, Zhengyi Yang

专题命中越狱攻击：safety(abstract)；jailbreak(abstract)

AI总结提出MemAttack框架，通过技能结构化的攻击记忆建模、生命周期驱动的记忆演化和探索-利用平衡的记忆选择，实现高效的黑盒越狱攻击，在AdvBench上达到98.00%攻击成功率。

Comments Under review

1. 越狱攻击 1717 篇

Jailbreaking is (Mostly) Simpler Than You Think

Towards Safe AI Clinicians: A Comprehensive Study on Large Language Model Jailbreaking in Healthcare

À la recherche du sens perdu: your favourite LLM might have more to say than you can understand

Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks

REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective

LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language

Effective Black-Box Multi-Faceted Attacks Breach Vision Large Language Model Guardrails

Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Models

RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts

PEFT-as-an-Attack! Jailbreaking Language Models during Federated Parameter-Efficient Fine-Tuning

Enhancing Adversarial Resistance in LLMs with Recursion

DeepInception: Hypnotize Large Language Model to Be Jailbreaker

Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats

Rapid Response: Mitigating LLM Jailbreaks with a Few Examples

Jailbreaking LLM-Controlled Robots

Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In

White-box Multimodal Jailbreaks Against Large Vision-Language Models

AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation

Ranking Manipulation for Conversational Search Engines

Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction

Automatic Jailbreaking of the Text-to-Image Generative AI Systems

Self-Guard: Empower the LLM to Safeguard Itself

AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting

A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems

Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases

Incremental Satisfiability Modulo Theory for Verification of Deep Neural Networks

Adversarial Eigen Attack on Black-Box Models

Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models

Proof-of-Execution Memory: Defending LLM Agents Against Forged-Reasoning Attacks by Verifying What Actually Happened

Evolving Skill-Structured Attack Memory Enhances LLM Jailbreaking