大模型对齐与安全 - arXivDaily 专题

2402.10196 2024-02-16 cs.CL cs.AI 62%

A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents

Lingbo Mo, Zeyi Liao, Boyuan Zheng, Yu Su, Chaowei Xiao, Huan Sun

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI

2306.14043 2024-02-13 cs.LG cs.AI cs.CR 62%

Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks

Pranav Dahiya, Ilia Shumailov, Ross Anderson

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments USENIX Security 2024 (https://www.usenix.org/conference/usenixsecurity24/presentation/dahiya)

2402.05467 2024-02-09 cs.AI cs.CL cs.CR 62%

Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia

Guangyu Shen, Siyuan Cheng, Kaiyuan Zhang, Guanhong Tao, Shengwei An, Lu Yan, Zhuo Zhang, Shiqing Ma, Xiangyu Zhang

专题命中越狱攻击：safety(abstract)；分类 cs.CL、cs.AI

2402.02309 2024-02-07 cs.LG cs.CL cs.CR cs.CV 62%

Jailbreaking Attack against Multimodal Large Language Model

Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, Rong Jin

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.LG

2206.06854 2024-02-05 cs.AI cs.CR cs.CV cs.LG stat.ML 62%

On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective

Mathieu Serrurier, Franck Mamalet, Thomas Fel, Louis Béthune, Thibaut Boissin

专题命中越狱攻击：alignment(abstract)；分类 cs.AI、cs.LG

Journal ref Conference on Neural Information Processing Systems (NeurIPS), Neural Information Processing Systems Foundation, Dec 2023, New Orleans (Louisiana), United States

2311.17035 2023-11-29 cs.LG cs.CL cs.CR 62%

Scalable Extraction of Training Data from (Production) Language Models

Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramèr, Katherine Lee

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.LG

2310.12860 2023-10-31 cs.CL cs.CY 62%

Probing LLMs for hate speech detection: strengths and vulnerabilities

Sarthak Roy, Ashish Harshavardhan, Animesh Mukherjee, Punyajoy Saha

专题命中越狱攻击：jailbreak(abstract)；分类 cs.CL、cs.CY

Comments 13 pages, 9 figures, 7 tables, accepted to findings of EMNLP 2023

2309.11751 2023-10-17 cs.CV cs.AI cs.CR cs.LG 62%

How Robust is Google's Bard to Adversarial Image Attacks?

Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Technical report

2309.00614 2023-09-06 cs.LG cs.CL cs.CR 62%

Baseline Defenses for Adversarial Attacks Against Aligned Language Models

Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, Tom Goldstein

专题命中越狱攻击：alignment(abstract)；分类 cs.CL、cs.LG

Comments 12 pages

2308.15092 2023-08-30 math.NA cs.AI cs.LG cs.NA 62%

Can We Rely on AI?

Desmond J. Higham

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

2202.00091 2023-03-27 cs.LG cs.AI cs.CR cs.CV 62%

Query Efficient Decision Based Sparse Attacks Against Black-Box Deep Learning Models

Viet Quoc Vo, Ehsan Abbasnejad, Damith C. Ranasinghe

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at the International Conference on Learning Representations (ICLR 2022). Code is available at https://sparseevoattack.github.io/

2302.07953 2023-02-17 cs.CR cs.AI cs.LG cs.RO 62%

AI Security Threats against Pervasive Robotic Systems: A Course for Next Generation Cybersecurity Workforce

Sudip Mittal, Jingdao Chen

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

2209.03463 2022-09-12 cs.CY cs.AI cs.CR cs.SI 62%

Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbots

Wai Man Si, Michael Backes, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, Savvas Zannettou, Yang Zhang

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.CY

Journal ref Published in ACM CCS 2022. Please cite the CCS version

2203.15506 2022-03-30 cs.CR cs.AI cs.LG 62%

Trojan Horse Training for Breaking Defenses against Backdoor Attacks in Deep Learning

Arezoo Rajabi, Bhaskar Ramasubramanian, Radha Poovendran

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Submitted to conference

2203.15498 2022-03-30 cs.CR cs.AI cs.CV cs.LG 62%

Powerful Physical Adversarial Examples Against Practical Face Recognition Systems

Inderjeet Singh, Toshinori Araki, Kazuya Kakizaki

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Accepted at IEEE/CVF WACV 2022 MAP

2108.01124 2021-08-04 cs.CR cs.AI cs.LG stat.AP 62%

Efficacy of Statistical and Artificial Intelligence-based False Information Cyberattack Detection Models for Connected Vehicles

Sakib Mahmud Khan, Gurcan Comert, Mashrur Chowdhury

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 6 figures

2101.06704 2021-01-19 cs.AI cs.CR cs.CV cs.LG 62%

Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions

Nodens Koren, Qiuhong Ke, Yisen Wang, James Bailey, Xingjun Ma

专题命中越狱攻击：safety(abstract)；分类 cs.AI、cs.LG

Comments Preprint

2607.09787 2026-07-14 cs.CV cs.LG 新提交 61%

Adversarially Guided Diffusion for LiDAR Range Image Synthesis

用于激光雷达距离图像合成的对抗引导扩散

Stavros Bouras, Antonios Makris, Alexandros Gkillas, Aris S. Lalos, Konstantinos Tserpes

机构 * School of Electrical and Computer Engineering, National Technical University of Athens（雅典国立技术大学电气与计算机工程学院）； Industrial Systems Institute, Athena Research Center（雅典娜研究中心工业系统研究所）

专题命中越狱攻击：safety(abstract)；分类 cs.LG；trustworthy(comments)

AI总结研究针对二维距离图像分割的无限制对抗攻击，提出基于扩散并利用分割损失对抗引导的方法，在SemanticKITTI数据集实验，能跨架构转移，相比基线在有效性与现实性间有独特权衡，实现可控退化。

Comments Accepted at the 1st Workshop on Secure and Trustworthy AI (STAI 2026), co-located with the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2026)

2510.11837 2025-10-15 cs.CR cs.AI 61%

Countermind: A Multi-Layered Security Architecture for Large Language Models

Dominik Schwarz

机构 * Independent Researcher（独立研究者）

专题命中越狱攻击：prompt injection(abstract,comments)；分类 cs.AI

Comments 33 pages, 3 figures, 6 tables. Keywords: LLM security; defense-in-depth; prompt injection; activation steering; multimodal sandbox; threat modeling

2608.12273 2026-08-13 cs.CR cs.AI 新提交 57%

Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agents

收敛式绕路劫持：基于技能的大语言模型智能体中的任务保留型资源放大

Junliang Liu, Ruoyu Li, Wenxin Tang, Jingyu Xiao, Zhenyu Liu, Jingheng Xu, Laizhong Cui

专题命中越狱攻击：safety(abstract)；分类 cs.AI

AI总结该研究提出收敛式绕路劫持攻击，耦合LLM智能体技能选择与规划阶段，可放大任务资源消耗，在DeepSeek-V4-Pro上80.02%的任务会被选中攻击者控制的协调器，任务完成率不变但成本显著提升。

2608.10530 2026-08-12 cs.CR cs.AI 新提交 57%

On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models

关于智能体大语言模型漏洞的理解、识别与缓解

Md Jafrin Hossain, Mohammad Arif Hossain, Nirwan Ansari

专题命中越狱攻击：prompt injection(abstract)；分类 cs.AI

AI总结本研究针对智能体大语言模型安全开展系统文献综述，提出四层漏洞分类法，发现攻击研究多于防御研究、感知层漏洞研究占比偏高等现状，识别7个开放问题及架构耦合的核心不安全根源。

2608.10405 2026-08-12 cs.SD cs.AI 新提交 57%

Never Stop Speaking: a Denial-of-Service Attack on End-to-End Speech Language Models

永不停止说话：针对端到端语音语言模型的拒绝服务攻击

Shuozhe Cheng, Kunlan Xiang, Mingxuan Li, Ji Zhang, Dongxiao Liu, Wenbo Jiang

专题命中越狱攻击：alignment(abstract)；分类 cs.AI

AI总结本研究针对端到端语音语言模型提出基于扰动的拒绝服务攻击，通过优化声学扰动抑制EOS生成以延长解码，在三类开源模型上验证了其攻击有效性及安全风险。

2604.23341 2026-08-11 cs.CR cs.AI 版本更新 57%

Evaluating Jailbreaking Vulnerabilities in LLMs Deployed as Assistants for Smart Grid Operations: A Benchmark Against NERC Standards

评估部署于智能电网操作中的LLM jailbreaking漏洞：与NERC标准的基准测试

Taha Hammadia, Lucas Rea, Ahmad Mohammad Saber, Amr Youssef, Deepa Kundur

机构 * ECE Department, University of Toronto（多伦多大学电子工程系）； CIISE, Concordia University（麦吉尔大学CIISE）

专题命中越狱攻击：safety(abstract)；分类 cs.AI

AI总结本文评估了智能电网操作中部署LLM的jailbreaking漏洞，通过与NERC标准的基准测试，发现DeepInception方法攻击成功率最高，Claude 3.5 Haiku完全免疫，Gemini 2.0 Flash-Lite最易受攻击。

Comments \c{opyright} 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

2608.04322 2026-08-06 cs.CL 新提交 57%

DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning

DataRx：面向更安全的大语言模型任务特定微调的缺失感知采样方法

Junbo Zhang, Qianli Zhou, Xinyang Deng, Wen Jiang

机构 * Northwestern Polytechnical University（西北工业大学）

专题命中越狱攻击：safety(abstract)；分类 cs.CL

AI总结本文提出DataRx缺失感知采样方法，通过高维隐表示量化安全信号差距，仅用1% BeaverTails安全样本，即可大幅降低Llama3-8B-Instruct的攻击成功率，还可与现有安全数据合成方法结合提升防御效果。

2605.17480 2026-07-30 cs.AI 版本更新 57%

The Capability Paradox: How Smarter Auditors Make Multi-Agent Systems Less Secure

能力悖论：更聪明的审计员如何使多智能体系统更不安全

Qiqi Liu, Runhan Song, Shilin Ye

机构 * University of Chinese Academy of Sciences（中国科学院大学）； Max Planck Institute for Security and Privacy（马克斯·普朗克安全与隐私研究所）； Henan Yinzhu Safety Technology Co., Ltd.（河南亿众安全技术有限公司）； Harbin Institute of Technology, Faculty of Computing（哈尔滨工业大学计算机学院）

专题命中越狱攻击：safety(abstract)；分类 cs.AI

AI总结本文研究了多智能体系统中，随着工人能力的提升，系统级攻击成功率反而上升的现象，揭示了语言确定性在攻击传播中的作用，并提出异质性集成验证作为解决方案，以降低攻击成功率。

Comments 28 pages, 6 figures

2607.22716 2026-07-28 cs.CV cs.LG 新提交 57%

Visual Token Compression Enhances Robustness of MLLMs

视觉令牌压缩增强多模态大语言模型的鲁棒性

Shishen Gu, Jiequan Cui, Wenbo Hu, Zenglin Shi, Zhenzhen Hu, Richang Hong

机构 * Hefei University of Technology（合肥工业大学）

专题命中越狱攻击：jailbreak(abstract)；分类 cs.LG

AI总结研究如何增强多模态大语言模型的鲁棒性，核心方法是通过测量视觉令牌与语言特征空间的距离来识别并剪枝分布外视觉令牌，该方法能有效防御越狱攻击、减轻幻觉，提升模型在通用数据集上的表现。

Comments 20 pages, 16 figures. Accepted at ACM Multimedia 2026. Code: https://github.com/Eurek001/OOD-VTP

2605.02187 2026-07-23 cs.CR cs.AI 版本更新 57%

Rewriting the Response Path: Silent Tampering and Provider-Signed Defense in BYOK LLM Agents

重写响应路径：BYOK LLM 代理中的静默篡改与提供者签名防御

Mingyu Luo, Zihan Zhang, Zesen Liu, Yuchong Xie, Zhixiang Zhang, Dung Hiu Hilton Yeung, Wai Ip Lai, Ping Chen, Ming Wen, Dongdong She

机构 * Fudan University（复旦大学）； The Hong Kong University of Science and Technology（香港科学与技术大学）； Huazhong University of Science and Technology（华中科技大学）

专题命中越狱攻击：alignment(abstract)；分类 cs.AI

AI总结研究BYOK LLM代理响应路径完整性问题，发现中继可篡改响应。提出sign-c服务器端方案，能认证执行承载字段和查询，本地垫片验证，加密保护机密性，防御有效拒绝篡改响应，误拒率为零，延迟开销低。

2607.04334 2026-07-21 cs.AI 版本更新 57%

Do GUI Agents Believe Their Eyes? Diagnosing State-Belief Reliance on Pixels versus Structure

图形用户界面代理相信它们的眼睛吗？诊断状态信念对像素与结构的依赖

Guijia Zhang, Yuxun Chen, Yuheng Qi, Harry Yang

机构 * Shenzhen University（深圳大学）； The Hong Kong University of Science and Technology（香港科技大学）

专题命中越狱攻击：safety(abstract)；分类 cs.AI

AI总结研究多模态GUI代理状态信念来源，通过对310个真实探针进行单通道干预形式化视觉状态依赖并测量，核心指标是感知融合差距，发现文本状态信念依赖结构，图像精度高，错误会导致行动失败。

Comments 17 pages, 3 figures

2601.12349 2026-07-16 cs.CR cs.AI cs.SE 版本更新 57%

Mind the Gap: Action Rebinding Attacks against Android GUI Agents

零权限操纵：我们能否信任由大型多模态模型驱动的GUI代理？

Yi Qian, Kunwei Qian, Xingbang He, Ligeng Chen, Jikang Zhang, Tiantai Zhang, Haiyang Wei, Linzhang Wang, Hao Wu, Bing Mao

机构 * State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）； Hornor Device Co., Ltd（Hornor设备有限公司）； Institute of Dataspace, Hefei Comprehensive National Science Center（数据空间研究所，合肥综合性国家科学中心）

专题命中越狱攻击：alignment(abstract)；分类 cs.AI

AI总结研究揭示了由大型多模态模型驱动的GUI代理在Android中存在视觉原子性假设的漏洞，通过零权限攻击实现对代理执行的重新绑定，并利用意图对齐策略绕过验证门，展示了代理-操作系统集成中的安全缺陷。

2607.11751 2026-07-14 cs.CR cs.LG cs.MA 新提交 57%

When Local Monitors Miss Compositional Harm: Diagnosing Distributed Backdoors in Multi-Agent Systems

当局部监测器遗漏组合性危害时：诊断多智能体系统中的分布式后门

Yibo Hu, Ren Wang

专题命中越狱攻击：safety(abstract)；分类 cs.LG

AI总结研究多智能体系统中分布式后门问题及局部监测器漏洞，提出可观测性边界概念，通过实验证明局部监测器在局部无害时会失效，还展示了特定监测器和门的效果，指出找到能暴露负载的表示形式是未解决问题。

1. 越狱攻击 1717 篇

A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents

Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks

Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia

Jailbreaking Attack against Multimodal Large Language Model

On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective

Scalable Extraction of Training Data from (Production) Language Models

Probing LLMs for hate speech detection: strengths and vulnerabilities

How Robust is Google's Bard to Adversarial Image Attacks?

Baseline Defenses for Adversarial Attacks Against Aligned Language Models

Can We Rely on AI?

Query Efficient Decision Based Sparse Attacks Against Black-Box Deep Learning Models

AI Security Threats against Pervasive Robotic Systems: A Course for Next Generation Cybersecurity Workforce

Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbots

Trojan Horse Training for Breaking Defenses against Backdoor Attacks in Deep Learning

Powerful Physical Adversarial Examples Against Practical Face Recognition Systems

Efficacy of Statistical and Artificial Intelligence-based False Information Cyberattack Detection Models for Connected Vehicles

Adversarial Interaction Attack: Fooling AI to Misinterpret Human Intentions

Adversarially Guided Diffusion for LiDAR Range Image Synthesis

Countermind: A Multi-Layered Security Architecture for Large Language Models

Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agents

On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models

Never Stop Speaking: a Denial-of-Service Attack on End-to-End Speech Language Models

Evaluating Jailbreaking Vulnerabilities in LLMs Deployed as Assistants for Smart Grid Operations: A Benchmark Against NERC Standards

DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning

The Capability Paradox: How Smarter Auditors Make Multi-Agent Systems Less Secure

Visual Token Compression Enhances Robustness of MLLMs

Rewriting the Response Path: Silent Tampering and Provider-Signed Defense in BYOK LLM Agents

Do GUI Agents Believe Their Eyes? Diagnosing State-Belief Reliance on Pixels versus Structure

Mind the Gap: Action Rebinding Attacks against Android GUI Agents

When Local Monitors Miss Compositional Harm: Diagnosing Distributed Backdoors in Multi-Agent Systems