大模型对齐与安全 - arXivDaily 专题

2404.12038 2024-12-03 cs.CL 83%

Uncovering Safety Risks of Large Language Models through Concept Activation Vector

Zhihao Xu, Ruixuan Huang, Changyu Chen, Xiting Wang

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Comments 10 pages, accepted at NeurIPS 2024

2410.08660 2024-12-02 cs.CR cs.AI 83%

RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process

Peiran Wang, Xiaogeng Liu, Chaowei Xiao

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.AI

2410.15645 2024-11-05 cs.AI 83%

Boosting Jailbreak Transferability for Large Language Models

Hanqing Liu, Lifeng Zhou, Huanqian Yan

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.AI

2410.15641 2024-10-22 cs.CL 83%

SMILES-Prompting: A Novel Approach to LLM Jailbreak Attacks in Chemical Synthesis

Aidan Wong, He Cao, Zijing Liu, Yu Li

专题命中越狱攻击：jailbreak(title)；safety(abstract)；prompt injection(abstract)；分类 cs.CL

2410.11272 2024-10-16 cs.CL 83%

Cognitive Overload Attack:Prompt Injection for Long Context

Bibek Upadhayay, Vahid Behzadan, Amin Karbasi

专题命中越狱攻击：prompt injection(title)；safety(abstract)；jailbreak(abstract)；分类 cs.CL

Comments 40 pages, 31 Figures

2410.03857 2024-10-10 cs.CL 83%

You Know What I'm Saying: Jailbreak Attack via Implicit Reference

Tianyu Wu, Lingrui Mei, Ruibin Yuan, Lujun Li, Wei Xue, Yike Guo

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL

2410.04190 2024-10-08 cs.CR cs.CL 83%

Harnessing Task Overload for Scalable Jailbreak Attacks on Large Language Models

Yiting Dong, Guobin Shen, Dongcheng Zhao, Xiang He, Yi Zeng

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

2409.07353 2024-09-12 cs.CV cs.AI 83%

Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attacks

Md Zarif Hossain, Ahmed Imteaj

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.AI

2408.05094 2024-08-12 cs.CL 83%

Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts

Tingchen Fu, Yupeng Hou, Julian McAuley, Rui Yan

专题命中越狱攻击：alignment(title,abstract)；harmlessness(abstract)；分类 cs.CL

2405.20773 2024-06-13 cs.CR cs.AI 83%

Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character

Siyuan Ma, Weidi Luo, Yu Wang, Xiaogeng Liu

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.AI

2405.19358 2024-06-03 cs.CR cs.AI 83%

Robustifying Safety-Aligned Large Language Models through Clean Data Curation

Xiaoqun Liu, Jiacheng Liang, Muchao Ye, Zhaohan Xi

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；分类 cs.AI

2308.03825 2024-05-16 cs.CR cs.LG 83%

"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models

Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, Yang Zhang

专题命中越狱攻击：jailbreak(title,abstract)；prompt injection(abstract)；分类 cs.LG

2405.09055 2024-05-16 cs.CL 83%

A safety realignment framework via subspace-oriented model fusion for large language models

Xin Yi, Shunfan Zheng, Linlin Wang, Xiaoling Wang, Liang He

专题命中越狱攻击：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL

2405.04403 2024-05-08 cs.CV cs.CL 83%

Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks

Georgios Pantazopoulos, Amit Parekh, Malvina Nikandrou, Alessandro Suglia

专题命中越狱攻击：jailbreak(title)；alignment(abstract)；safety(abstract)；分类 cs.CL

2310.06474 2024-03-05 cs.CL 83%

Multilingual Jailbreak Challenges in Large Language Models

Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, Lidong Bing

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

Comments ICLR 2024

2312.04127 2024-02-26 cs.CL 83%

Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak

Yanrui Du, Sendong Zhao, Ming Ma, Yuhan Chen, Bing Qin

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL

2401.16765 2024-01-31 cs.CR cs.AI 83%

A Cross-Language Investigation into Jailbreak Attacks in Large Language Models

Jie Li, Yi Liu, Chongyang Liu, Ling Shi, Xiaoning Ren, Yaowen Zheng, Yang Liu, Yinxing Xue

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；分类 cs.AI

2307.14539 2023-10-12 cs.CR cs.CL 83%

Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models

Erfan Shayegani, Yue Dong, Nael Abu-Ghazaleh

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL

2410.15236 2026-05-29 cs.CR cs.AI cs.LG 82%

Jailbreaking and Mitigation of Vulnerabilities in Large Language Models

大语言模型的越狱与漏洞缓解

Benji Peng, Hanxuan Chen, Keyu Chen, Qian Niu, Ziqian Bi, Ming Liu, Pohsun Feng, Tianyang Wang, Lawrence K. Q. Yan, Yizhu Wen, Yichao Zhang, Caitlyn Heqi Yin, Xinyuan Song, Riyang Bao, Jiacheng Shi

机构 * Hunan University Changsha, PRC ； Georgia Institute of Technology Atlanta, USA ； Kyoto University Kyoto, Japan ； Purdue University West Lafayette, USA ； National Taiwan Normal University Taipei, ROC ； University of Liverpool Suzhou, PRC ； Hong Kong University of Science ； University of Hawaii Honolulu, USA ； The University of Texas at Dallas Dallas, USA ； University of Wisconsin-Madison Madison, USA ； Emory University Atlanta, USA ； College of William \& Mary Williamsburg, USA

专题命中越狱攻击：alignment(abstract)；safety(abstract)；jailbreak(abstract)；prompt injection(abstract)

AI总结本文综述了大语言模型在提示注入和越狱攻击下的漏洞，分类攻击方法并评估防御策略，指出研究空白与未来方向。

Journal ref Eureka 1(1) (2026) 26-61

2605.26158 2026-05-27 cs.CR cs.AI cs.LG 82%

Furina: Fragmented Uncertainty-Driven Refusal Instability Attack

Furina: 碎片化不确定性驱动的拒绝不稳定攻击

Tongxi Wu, Jian Zhang, Yang Gao

机构 * School of Intelligence Science and Technology（智能科学与技术学院）； State Key Laboratory for Novel Software Technology（新型软件技术国家重点实验室）； Nanjing University（南京大学）

专题命中越狱攻击：jailbreak(abstract,abstract_cn)；alignment(abstract)；safety(abstract)；分类 cs.AI、cs.LG

AI总结通过揭示大语言模型安全行为存在不稳定区域，提出多指标诊断框架并开发Furina攻击方法，利用碎片化场景提示诱导不确定性放大，实现高效越狱。

Comments This work is accepted as a regular paper at ICML 2026

2512.13741 2025-12-17 cs.LG cs.AI 82%

The Laminar Flow Hypothesis: Detecting Jailbreaks via Semantic Turbulence in Large Language Models

层流假说：通过大语言模型中的语义湍流检测对抗性突破

Md. Hasib Ur Rahman

机构 * Brac University（布拉克大学）

专题命中越狱攻击：alignment(abstract)；RLHF(abstract)；safety(abstract)；jailbreak(abstract)

AI总结本研究提出层流假说，通过检测大语言模型中的语义湍流，实现对抗性突破的实时检测与安全架构诊断。

2405.21018 2024-06-06 cs.LG cs.CL cs.CR 82%

Improved Techniques for Optimization-Based Jailbreaking on Large Language Models

Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jindong Gu, Yang Liu, Xiaochun Cao, Min Lin

专题命中越狱攻击：alignment(abstract)；safety(abstract)；jailbreak(abstract)；red teaming(abstract)

2310.10844 2023-10-18 cs.CL cs.CR cs.LG 82%

Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks

Erfan Shayegani, Md Abdullah Al Mamun, Yu Fu, Pedram Zaree, Yue Dong, Nael Abu-Ghazaleh

专题命中越狱攻击：alignment(abstract)；safety(abstract)；jailbreak(abstract)；trustworthy(abstract)

2601.09321 2026-08-13 cs.CR 版本更新 82%

SpatialJB: How Text Distribution Art Becomes the "Jailbreak Key" for LLM Guardrails

SpatialJB: 文本分布艺术如何成为LLM防护机制的'突破密钥'

Zhiyi Mou, Jingyuan Yang, Zeheng Qian, Wangze Ni, Tianfang Xiao, Ning Liu, Chen Zhang, Zhan Qin, Kui Ren

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)

AI总结 SpatialJB通过破坏LLM输出生成过程，利用空间结构扰动突破现有防护机制，实验显示其高效性，同时提出基础防御策略以应对此类攻击。

Journal ref ICML 2026

2608.04034 2026-08-06 cs.CR 新提交 82%

A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination

基于原子越狱策略解耦与组合的多模态自动红队评估

Shiji Zhao, Yuxuan Zhou, Chen Xiong, Dongxian Wu, Yang Bai, Xun Chen

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)

AI总结针对现有多模态越狱攻击的两大挑战，本文提出HACA方法，构建多模态原子越狱策略空间，对5种主流MLLMs平均攻击成功率达95.48%。

2607.01277 2026-07-03 cs.CR 新提交 82%

Cognitive Firewall: A Proactive, Zero-Trust, Multi-Gate Framework for LLM Safety

认知防火墙：一种主动、零信任、多门控的LLM安全框架

Michele Guida, Ruslan Shikhhamzayev, Sindhuja Penchala, Stefano Iannucci, Jiacheng Li, Shahram Rahimi, Noorbakhsh Amiri Golilarz

专题命中越狱攻击：safety(title,abstract)；jailbreak(abstract)

AI总结提出认知防火墙框架，通过意图、零信任上下文、一致性和输出风险四个门控进行对话级安全评估，有效降低单轮、多轮、基于权威和人工制作的越狱攻击成功率。

2510.15499 2026-06-16 cs.CR 版本更新 82%

HarmRLVR: Weaponizing Verifiable Rewards for Harmful LLM Alignment

HarmRLVR: 利用可验证奖励进行有害大模型对齐

Yuexiao Liu, Lijun Li, Xingjun Wang, Jing Shao

专题命中越狱攻击：alignment(title,abstract)；safety(abstract)

AI总结提出HarmRLVR，首次系统研究可验证奖励强化学习（RLVR）的对齐可逆性风险，通过仅64个有害提示的GRPO即可快速逆转安全对齐，攻击成功率高达96.01%。

2605.02958 2026-05-27 cs.CR cs.AI cs.CL cs.LG 82%

Tracing the Dynamics of Refusal: Exploiting Latent Refusal Trajectories for Robust Jailbreak Detection

追踪拒绝的动态：利用潜在拒绝轨迹进行鲁棒越狱检测

Xulin Hu, Che Wang, Wei Yang Bryan Lim, Jianbo Gao, Zhong Chen

机构 * Peking University（北京大学）； Nanyang Technological University（南洋理工大学）； Beijing Jiaotong University（北京交通大学）

专题命中越狱攻击：jailbreak(title,abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结通过因果追踪识别出稀疏的“拒绝轨迹”激活模式，并提出轻量级白盒检测器SALO，基于隐藏状态窗口实现鲁棒越狱检测。

Comments Accepted to the 43rd International Conference on Machine Learning (ICML 2026). Camera-ready version

2605.24552 2026-05-26 cs.CR 82%

Ellipsoid Control: A White-list Jailbreak Defense via Benign Latent Modeling

椭球控制：通过良性潜在建模的白名单越狱防御

Luoyu Chen, Weiqi Wang, Zhiyi Tian, Feng Wu, Ahmed Asiri, Shui Yu

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)

AI总结提出一种基于白名单视角的测试时防御方法Ellipsoid Control，通过拟合良性数据的各向异性椭球约束投影梯度下降，在任意输入上触发拒绝同时最小化良性潜在几何的扭曲，从而在增强安全性的同时保持模型效用。

Comments Under review by TIFS

2605.16471 2026-05-19 cs.CR 82%

From AI-Generated Content to Agentic Action: Security and Safety Threats in Generative AI

从AI生成内容到代理行为：生成式AI中的安全与安全威胁

Zelin Zhang, Qi Li, Jie Cao, Lingshuang Liu, Jianbing Ni

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)

AI总结研究探讨生成式AI从内容生成转向执行操作带来的安全与安全威胁，分析攻击者需求、系统自主性及潜在危害的变化，并评估检测、水印、对齐等技术对策，指出当前治理机制无法提供必要的协调。

Comments Accepted by Journal of Information and Intelligence (JII)

1. 越狱攻击 1715 篇

Uncovering Safety Risks of Large Language Models through Concept Activation Vector

RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process

Boosting Jailbreak Transferability for Large Language Models

SMILES-Prompting: A Novel Approach to LLM Jailbreak Attacks in Chemical Synthesis

Cognitive Overload Attack:Prompt Injection for Long Context

You Know What I'm Saying: Jailbreak Attack via Implicit Reference

Harnessing Task Overload for Scalable Jailbreak Attacks on Large Language Models

Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attacks

Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts

Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character

Robustifying Safety-Aligned Large Language Models through Clean Data Curation

"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models

A safety realignment framework via subspace-oriented model fusion for large language models

Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks

Multilingual Jailbreak Challenges in Large Language Models

Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak

A Cross-Language Investigation into Jailbreak Attacks in Large Language Models

Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models

Jailbreaking and Mitigation of Vulnerabilities in Large Language Models

Furina: Fragmented Uncertainty-Driven Refusal Instability Attack

The Laminar Flow Hypothesis: Detecting Jailbreaks via Semantic Turbulence in Large Language Models

Improved Techniques for Optimization-Based Jailbreaking on Large Language Models

Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks

SpatialJB: How Text Distribution Art Becomes the "Jailbreak Key" for LLM Guardrails

A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination

Cognitive Firewall: A Proactive, Zero-Trust, Multi-Gate Framework for LLM Safety

HarmRLVR: Weaponizing Verifiable Rewards for Harmful LLM Alignment

Tracing the Dynamics of Refusal: Exploiting Latent Refusal Trajectories for Robust Jailbreak Detection

Ellipsoid Control: A White-list Jailbreak Defense via Benign Latent Modeling

From AI-Generated Content to Agentic Action: Security and Safety Threats in Generative AI