大模型对齐与安全 - arXivDaily 专题

2005.14137 2020-05-29 cs.LG cs.CV stat.ML 57%

QEBA: Query-Efficient Boundary-Based Blackbox Attack

Huichen Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, Bo Li

专题命中越狱攻击：safety(abstract)；分类 cs.LG

Comments Accepted by CVPR 2020

1909.08072 2019-10-10 cs.LG cs.CR stat.ML 57%

Adversarial Attacks and Defenses in Images, Graphs and Text: A Review

Han Xu, Yao Ma, Haochen Liu, Debayan Deb, Hui Liu, Jiliang Tang, Anil K. Jain

专题命中越狱攻击：safety(abstract)；分类 cs.LG

Comments survey, adversarial attacks, defenses

1907.11129 2019-07-26 cs.LG cs.CR cs.NE stat.AP stat.ML 57%

Semisupervised Adversarial Neural Networks for Cyber Security Transfer Learning

Casey Kneale, Kolia Sadeghi

专题命中越狱攻击：alignment(abstract)；分类 cs.LG

Comments 14 figures, 17 pages, Draft

1906.07077 2019-06-18 cs.LG cs.CR stat.ML 57%

The Attack Generator: A Systematic Approach Towards Constructing Adversarial Attacks

Felix Assion, Peter Schlicht, Florens Greßner, Wiebke Günther, Fabian Hüger, Nico Schmidt, Umair Rasheed

专题命中越狱攻击：safety(abstract)；分类 cs.LG

Comments CVPR SAIAD - Workshop 2019

1804.05810 2019-05-02 cs.CV cs.CR cs.LG stat.ML 57%

ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector

Shang-Tse Chen, Cory Cornelius, Jason Martin, Duen Horng Chau

专题命中越狱攻击：safety(abstract)；分类 cs.LG

Journal ref Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 52-68, 2018

1812.02575 2018-12-08 stat.ML cs.CR cs.LG 57%

Prior Networks for Detection of Adversarial Attacks

Andrey Malinin, Mark Gales

专题命中越狱攻击：safety(abstract)；分类 cs.LG

1709.03423 2018-02-09 stat.ML cs.LG 57%

Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks

Thilo Strauss, Markus Hanselmann, Andrej Junginger, Holger Ulmer

专题命中越狱攻击：safety(abstract)；分类 cs.LG

Comments 10 pages, 2 figures, 4 tables

1711.05934 2017-11-17 cs.CV cs.CR cs.LG 57%

Enhanced Attacks on Defensively Distilled Deep Neural Networks

Yujia Liu, Weiming Zhang, Shaohua Li, Nenghai Yu

专题命中越狱攻击：safety(abstract)；分类 cs.LG

1710.00814 2017-10-04 cs.CV cs.CR cs.LG 57%

Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight

Yen-Chen Lin, Ming-Yu Liu, Min Sun, Jia-Bin Huang

专题命中越狱攻击：safety(abstract)；分类 cs.LG

Comments Project page: http://yclin.me/RL_attack_detection/ Code: https://github.com/yenchenlin/rl-attack-detection

2407.01606 2024-07-03 cs.LG cs.AI cs.CL cs.CV stat.ML 56%

On Discrete Prompt Optimization for Diffusion Models

Ruochen Wang, Ting Liu, Cho-Jui Hsieh, Boqing Gong

专题命中越狱攻击：分类 cs.CL、cs.AI、cs.LG；DPO(comments)

Comments ICML 2024. Code available at https://github.com/ruocwang/dpo-diffusion

Journal ref Proceedings of the 41st International Conference on Machine Learning (ICML 2024)

2608.13671 2026-08-17 cs.CV 新提交 50%

PROVE: Training-Free Prompt Recovery using Verifiable Evidence

PROVE：基于可验证证据的无训练提示词恢复

Rupayan Mallick, Mahsa Khoshnoodi, Sarah Adel Bargal

专题命中越狱攻击：alignment(abstract)

AI总结该研究提出无训练的黑盒提示词反转攻击PROVE，通过可验证场景描述重建提示词，在多数据集上优于基线方法，可用于版权保护相关研究。

2603.22928 2026-08-12 cs.CR 版本更新 50%

SoK: The Attack Surface of Agentic AI - Tools and Autonomy

SoK：代理AI的攻击面——工具与自主性

Ali Dehghantanha, Sajad Homayoun

专题命中越狱攻击：prompt injection(abstract)

AI总结本文系统分析了代理AI的安全风险，提出攻击分类与评估指标，探讨防御措施及开放研究挑战，帮助研究人员和工程师应对代理AI的安全威胁。

2603.04469 2026-08-06 cs.CR cs.MA 版本更新 50%

Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systems

超越输入防护：为执行感知攻击检测重建跨代理语义流

Qizhi Cai, Yangyang Wei, Zhipeng Chen, Shouling Ji, Zhenyuan Li

专题命中越狱攻击：prompt injection(abstract)

AI总结 SysName通过重建跨代理语义流，实现执行感知的攻击检测，有效识别多种复合攻击向量，提升多代理系统安全性。

Comments 16 pages, 15 figures

2509.20851 2026-08-05 cs.CV 版本更新 50%

Poisoning Prompt-Guided Sampling in Video Large Language Models

针对视频大语言模型中提示引导采样的投毒攻击

Yuxin Cao, Wei Song, Jingling Xue, Jin Song Dong

机构 * National University of Singapore（新加坡国立大学）； University of New South Wales（新南威尔士大学）； CSIRO’s Data61（CSIRO数据61）

专题命中越狱攻击：safety(abstract)

AI总结该研究针对视频大语言模型的提示引导采样提出PoisonVID投毒攻击，在多种模型与采样器组合上实现高攻击成功率，揭示了PGS存在的结构性安全隐患。

Comments 16 pages, 5 figures

2608.01117 2026-08-04 cs.CR 新提交 50%

SoK: Intent-Oriented Systematization of Multi-Turn LLM Jailbreaks

SoK：面向意图的多轮大语言模型越狱系统化研究

Siyuan Li, Aodu Wulianghai, Zehao Liu, Xi Lin, Qinghua Mao, Haoyu Li, Xiang Chen, Siyuan Liang, Jun Wu, Jianhua Li, Dacheng Tao

专题命中越狱攻击：safety(abstract)

AI总结本研究提出面向意图的多轮LLM越狱分类法，发现攻击有效性取决于意图组织精细度，推动检测范围升级，表明轮级安全机制不足，需对应层级评估协议。

2607.25936 2026-07-29 cs.CR 新提交 50%

From Role Prompt to Infinite Thinking: Exploiting Persona Conditioning for Inference Cost Attacks in LLMs

从角色提示到无限思考：利用角色条件进行大语言模型推理成本攻击

Zhiyi Mou, Wangze Ni, Tianfang Xiao, Haoyang LI, Chen Jason Zhang, Hanzhi Ma, Yang Bai, Zhibo Wang, Kui Ren

专题命中越狱攻击：alignment(abstract)

AI总结研究LLMs推理成本问题，提出RolePlay框架利用角色条件诱导低效行为放大推理成本，实验证明该框架优于现有方法，为LLM推理效率攻击提供新视角。

Comments 17pages

2607.21243 2026-07-24 cs.CV 新提交 50%

Detectors Learn the Wrong Thing: Shortcut-Resistant Adversarial Training Against Physically Realizable Attacks

检测器学习到错误的东西：针对物理可实现攻击的抗捷径对抗训练

Yuanhao Huang, Yilong Ren, Jinlei Wang, Xuesong Bai, Zheng Zhang, Haiyang Yu

机构 * School of Transportation Science and Engineering, Beihang University（北京航空航天大学交通科学与工程学院）； State Key Lab of Intelligent Transportation System（智能交通系统国家重点实验室）； Zhongguancun Laboratory（中关村实验室）； School of Systems Science and Engineering, Sun Yat-Sen University（中山大学系统科学与工程学院）； Shandong Sino-Aisa Tire Proving Ground Co.,Ltd.（山东中亚洲轮胎试验场有限公司）

专题命中越狱攻击：safety(abstract)

AI总结研究针对物理可实现攻击，提出实例级对比对抗训练框架InsCAT，通过SICA、ROPO和Guard等方法防止检测器将对抗纹理用作独立决策线索，经实验验证其在多场景和检测器上效果良好，提升检测可靠性。

2607.19545 2026-07-23 cs.CR 新提交 50%

When HTTP 402 Meets the Blockchain: Risks on Emerging x402 Payments

当HTTP 402遇上区块链：新兴x402支付的风险

Qinying Wang, Yong Yang, Yuan Chen, Shouling Ji, Mathias Payer

专题命中越狱攻击：safety(abstract)

AI总结研究x402支付协议，通过系统研究确定促进者的安全规则，基于规则违规分析得出新攻击向量，提出半自动黑盒工具评估安全性，发现部署均有违规，还通过实证测量量化相关指标，揭示其安全风险。

Journal ref USENIX Security 2026

2602.06409 2026-07-22 cs.CR 版本更新 50%

VENOMREC: Cross-Modal Interactive Poisoning for Targeted Promotion in Multimodal LLM Recommender Systems

VENOMREC: 多模态交互性污染用于多模态大语言模型推荐系统中的定向推广

Guowei Guan, Yurong Hao, Jiaming Zhang, Tiantong Wu, Fuyao Zhang, Tianxiang Chen, Longtao Huang, Cyril Leung, Wei Yang Bryan Lim

专题命中越狱攻击：alignment(abstract)

AI总结 VENOMREC通过跨模态交互性污染技术，在多模态大语言模型推荐系统中实现定向推广，有效提升推荐性能。

2502.12497 2026-07-10 cs.CR 版本更新 50%

Demystifying LLM Supply Chain Vulnerabilities in the Wild: Distribution, Root Cause, and Real-World Impact

揭开大语言模型供应链在实际应用中的漏洞：分布、根源及现实世界影响

Shenao Wang, Yanjie Zhao, Zhao Liu, Quanchen Zou, Haoyu Wang

专题命中越狱攻击：safety(abstract)

AI总结该研究首次系统大规模实证分析大语言模型供应链漏洞，分析529个实际漏洞，发现漏洞集中在应用层等，特定漏洞源于独特特征，还研究了现实影响，提炼5条见解，为保障供应链安全提供数据基础与研究方向。

Comments Accepted by Internetware 2026

2603.21664 2026-07-08 cs.CV 版本更新 50%

HumanOmni-Speaker: Identifying Who said What and When

HumanOmni-Speaker: 识别说话者说了什么以及何时

Detao Bai, Zhiheng Ma, Xihan Wei

机构 * Tongyi Lab Alibaba Group（阿里云实验室）； Shenzhen University of Advanced Technology（深圳先进技术大学）

专题命中越狱攻击：alignment(abstract)

AI总结本文提出HumanOmni-Speaker，通过视觉注册说话人辨识与识别方法，解决多人物对话中说话者身份与时间的识别问题，实现端到端的多模态协同。

2510.02964 2026-07-08 cs.CR 版本更新 50%

External Data Extraction Attacks against Retrieval-Augmented Large Language Models

针对检索增强型大语言模型的外部数据提取攻击

Yu He, Yifei Chen, Yiming Li, Shuo Shao, Leyi Qi, Boheng Li, Dacheng Tao, Zhan Qin

专题命中越狱攻击：jailbreak(abstract)

AI总结研究针对检索增强型大语言模型的外部数据提取攻击，提出统一框架并开发SECRET攻击方法，该方法含自适应优化与聚类聚焦触发策略，实验显示其显著优于先前攻击，能有效提取数据，呼吁关注此新兴威胁。

Comments Accepted by IEEE TIFS

2602.04356 2026-07-07 cs.CV 版本更新 50%

Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models

用于对大型视觉语言模型进行对抗攻击的逐阶段注意力引导区域排序

Jaehyun Kwak, Nam Cao, Boryeong Cho, Segyu Lee, Sumyeong Ahn, Se-Young Yun

机构 * KAIST（韩国科学技术院）； KENTECH（KENTECH研究院）

专题命中越狱攻击：safety(abstract)

AI总结研究针对大型视觉语言模型的有针对性对抗攻击，基于注意力分析提出逐阶段注意力引导区域排序，介绍了黑盒区域排序框架SAGA，在多个模型上取得了最先进的攻击成功率和最佳不可感知性。

Comments Pre-print

2607.01356 2026-07-03 cs.CR cs.SY eess.SY 新提交 50%

Chameleon: Recovering Cyber-Physical Systems from Memory Corruption Attacks via ML Surrogates

Chameleon: 通过ML代理从内存破坏攻击中恢复信息物理系统

Mohsen Salehi, Karthik Pattabiraman

专题命中越狱攻击：safety(abstract)

AI总结提出Chameleon框架，利用基于机器学习的代理在检测到攻击时替换受损组件，实现信息物理系统从内存破坏攻击中的自动恢复，在多种机器人车辆上验证了高保真度和低开销。

2607.00361 2026-07-02 cs.CR 新提交 50%

ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision-Language Models

ReShift: 视觉-语言模型上基于“啊哈时刻”驱动的推理级后门攻击

Zhihao Dou, Qinjian Zhao, Zhiqiang Gao, Sumon Biswas

专题命中越狱攻击：safety(abstract)

AI总结提出ReShift框架，通过“啊哈时刻”驱动推理轨迹重定向，结合PRDC管道和SRJO策略实现隐蔽的后门攻击，理论保证熵反弹与轨迹发散的联系。

Comments ECCV 2026

2606.26377 2026-06-26 cs.CR 新提交 50%

Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats

验证意图与危害：针对LLM生成威胁的统一防御

Poojitha Thota, Yun Lei, Santhosh Thangaraj, Siddhartha Reddy Jonnalagadda, Shirin Nilizadeh

专题命中越狱攻击：prompt injection(abstract)

AI总结提出一种联合验证用户提示意图和模型响应危害的防御框架，通过专门分析器与裁决器协同工作，在五个威胁类别上平均F1提升至0.95，攻击成功率降至4.1%。

2606.21842 2026-06-23 cs.CR 新提交 50%

Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG

代理辅助的非前缀KV缓存侧信道攻击

He Sun, Shinan Liu, Siyuan Ma, Junhao Li, Mingjun Xiao, Wenhao Jiang

专题命中越狱攻击：alignment(abstract)

AI总结针对RAG系统中非前缀KV缓存融合的侧信道漏洞，提出SpliceLeak攻击，利用“阶梯波”时序特征提取私有提示长度和语义内容，并设计SpliceDefense防御机制。

Comments 15 pages, 8 figures

2606.01410 2026-06-16 cs.HC 版本更新 50%

What LLMs Must Forget to Teach Effectively: A DIY Approach to Premodern Japanese Language Pedagogy

LLM必须忘记什么才能有效教学：前现代日语教学法的DIY方法

Ariel Stilerman, Andrew Nelson, Alan Cheng, Caleb Langley, Sera Wang, Camilla Piana, Pelin Çılgın, Qianhe Qin, Teisha Nishimitsu, Liaoliao Zhang, Huiting Liu, Josh Eyre, Gavin Sherry

专题命中越狱攻击：alignment(abstract)

AI总结本文提出一种基于大型语言模型（LLM）的DIY教学框架，通过提示工程创建定制工具，避免LLM过度解释或产生幻觉，从而促进前现代日语文学和语言课程中的主动理解与教学对齐。

2603.15125 2026-06-08 cs.CR 版本更新 50%

From Storage to Steering: Memory Control Flow Attacks on LLM Agents

从存储到操控：针对LLM代理的内存控制流攻击

Zhenlin Xu, Xiaogang Zhu, Yu Yao, Minhui Xue, Yiliao Song

专题命中越狱攻击：safety(abstract)

AI总结本文研究了内存控制流攻击对LLM代理的影响，提出MEMFLOW框架以系统识别和量化此类攻击，发现超过90%的测试在严格安全约束下仍易受攻击，揭示了严重的安全风险。

2605.15741 2026-06-04 cs.CV 50%

HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusion

HyperDiT: 用于高保真像素空间扩散的超连接Transformer

Yu He, Lichen Ma, Zipeng Guo, Xinyuan Shan, Jingling Fu, Dong Chen, Junshi Huang, Yan Li

机构 * University of Science and Technology of China（中国科学技术大学）

专题命中越狱攻击：alignment(abstract)

AI总结针对像素空间扩散模型中全局语义与细粒度细节难以兼顾的粒度困境，提出HyperDiT框架，通过超连接跨尺度交互和尺度感知旋转位置编码，结合预训练视觉基础模型的密集语义，在像素空间实现高保真生成，在ImageNet 256×256上取得1.56的SoTA FID。

1. 越狱攻击 1721 篇

QEBA: Query-Efficient Boundary-Based Blackbox Attack

Adversarial Attacks and Defenses in Images, Graphs and Text: A Review

Semisupervised Adversarial Neural Networks for Cyber Security Transfer Learning

The Attack Generator: A Systematic Approach Towards Constructing Adversarial Attacks

ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector

Prior Networks for Detection of Adversarial Attacks

Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks

Enhanced Attacks on Defensively Distilled Deep Neural Networks

Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight

On Discrete Prompt Optimization for Diffusion Models

PROVE: Training-Free Prompt Recovery using Verifiable Evidence

SoK: The Attack Surface of Agentic AI - Tools and Autonomy

Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systems

Poisoning Prompt-Guided Sampling in Video Large Language Models

SoK: Intent-Oriented Systematization of Multi-Turn LLM Jailbreaks

From Role Prompt to Infinite Thinking: Exploiting Persona Conditioning for Inference Cost Attacks in LLMs

Detectors Learn the Wrong Thing: Shortcut-Resistant Adversarial Training Against Physically Realizable Attacks

When HTTP 402 Meets the Blockchain: Risks on Emerging x402 Payments

VENOMREC: Cross-Modal Interactive Poisoning for Targeted Promotion in Multimodal LLM Recommender Systems

Demystifying LLM Supply Chain Vulnerabilities in the Wild: Distribution, Root Cause, and Real-World Impact

HumanOmni-Speaker: Identifying Who said What and When

External Data Extraction Attacks against Retrieval-Augmented Large Language Models

Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models

Chameleon: Recovering Cyber-Physical Systems from Memory Corruption Attacks via ML Surrogates

ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision-Language Models

Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats

Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG

What LLMs Must Forget to Teach Effectively: A DIY Approach to Premodern Japanese Language Pedagogy

From Storage to Steering: Memory Control Flow Attacks on LLM Agents

HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusion