大模型对齐与安全 - arXivDaily 专题

2502.12562 2025-06-04 cs.CL cs.CR cs.MM 88%

SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddings

Weikai Lu, Hao Peng, Huiping Zhuang, Cen Chen, Ziqian Zeng

机构 * South China University of Technology, China（华南理工大学）； Beihang University, China（北航）； Pazhou Laboratory, China（琶洲实验室）

专题命中越狱攻击：alignment(title,abstract)；safety(title,abstract)；分类 cs.CL

Comments Accepted in ACL 2025 Main Track

2506.00668 2025-06-03 cs.CL 88%

SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues

Martin Kuo, Jianyi Zhang, Aolin Ding, Louis DiValentin, Amin Hass, Benjamin F Morris, Isaac Jacobson, Randolph Linderman, James Kiessling, Nicolas Ramos, Bhavna Gopal, Maziyar Baran Pouyan, Changwei Liu, Hai Li, Yiran Chen

机构 * Center for Computational Evolutionary Intelligence, Duke University（计算进化智能中心，杜克大学）； Accenture, USA（Accenture美国公司）

专题命中越狱攻击：alignment(title,abstract)；safety(title,abstract)；分类 cs.CL

2502.15594 2025-05-26 cs.CL 88%

SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention

Jiaqi Wu, Chen Chen, Chunyan Hou, Xiaojie Yuan

专题命中越狱攻击：safety(title,abstract)；jailbreak(title,abstract)；分类 cs.CL

2502.12893 2025-02-28 cs.CL 88%

H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking

Martin Kuo, Jianyi Zhang, Aolin Ding, Qinsi Wang, Louis DiValentin, Yujia Bao, Wei Wei, Hai Li, Yiran Chen

专题命中越狱攻击：safety(title,abstract)；jailbreak(title,abstract)；分类 cs.CL

Comments Website: https://maliciouseducator.org/

2410.02298 2025-02-10 cs.CR cs.CL 88%

Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models

Guobin Shen, Dongcheng Zhao, Yiting Dong, Xiang He, Yi Zeng

专题命中越狱攻击：safety(title,abstract)；jailbreak(title,abstract)；分类 cs.CL

Comments Accepted by ICLR2025. url: https://openreview.net/forum?id=s20W12XTF8

2410.09760 2025-02-03 cs.LG 88%

Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation

Guozhi Liu, Weiwei Lin, Tiansheng Huang, Ruichao Mo, Qi Mu, Li Shen

专题命中越狱攻击：alignment(title,abstract)；safety(title,abstract)；分类 cs.LG

2412.00357 2024-12-03 cs.AI cs.CV 88%

Safety Alignment Backfires: Preventing the Re-emergence of Suppressed Concepts in Fine-tuned Text-to-Image Diffusion Models

Sanghyun Kim, Moonseok Choi, Jinwoo Shin, Juho Lee

专题命中越狱攻击：alignment(title,abstract)；safety(title,abstract)；分类 cs.AI

Comments 20 pages, 18 figures

2405.18641 2024-10-30 cs.LG 88%

Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack

Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, Ling Liu

专题命中越狱攻击：alignment(title,abstract)；safety(title,abstract)；分类 cs.LG

Comments Accepted by NeurIPS2024. arXiv admin note: substantial text overlap with arXiv:2402.01109

2312.06924 2024-06-10 cs.CL 88%

Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack

Yu Fu, Yufei Li, Wen Xiao, Cong Liu, Yue Dong

专题命中越狱攻击：alignment(title,abstract)；safety(title,abstract)；分类 cs.CL

Comments Accepted to ACL2024 main

2402.13494 2024-05-31 cs.CL cs.CR 88%

GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis

Yueqi Xie, Minghong Fang, Renjie Pi, Neil Gong

专题命中越狱攻击：safety(title,abstract)；jailbreak(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2024 Main

2512.15353 2025-12-18 cs.CL cs.AI 88%

Adversarial versification in portuguese as a jailbreak operator in LLMs

葡萄牙对抗性韵律作为LLMs中的对抗性操作符

Joao Queiroz

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；RLHF(abstract)；safety(abstract)

AI总结研究发现，将提示改写为诗歌能显著提高LLMs的安全漏洞，揭示当前对齐机制的不足，并指出葡萄牙语评估的缺失。

Comments 15 pages

2502.19820 2025-03-31 cs.CL cs.AI 88%

Foot-In-The-Door: A Multi-turn Jailbreak for LLMs

Zixuan Weng, Xiaolong Jin, Jinyuan Jia, Xiangyu Zhang

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；AI safety(abstract)

Comments 19 pages, 8 figures

2407.16686 2024-08-26 cs.CL 88%

Can Large Language Models Automatically Jailbreak GPT-4V?

Yuanwei Wu, Yue Huang, Yixin Liu, Xiang Li, Pan Zhou, Lichao Sun

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；RLHF(abstract)；safety(abstract)

Comments TrustNLP@NAACL2024 (Fourth Workshop on Trustworthy Natural Language Processing)

2607.19829 2026-07-23 cs.CR 新提交 88%

DARWIN: Evolving Jailbreak Adversary and Guardrail for LLM Safety Evaluation and Protection

达尔文：为大语言模型安全评估与保护演化越狱对手及防护措施

Weiwei Qi, Zefeng Wu, Zhilin Guo, Tianhang Zheng, Chaochao Lu, Liang He, Zhan Qin, Kui Ren

专题命中越狱攻击：safety(title,abstract)；jailbreak(title,abstract)

AI总结针对现有大语言模型安全评估与防御方法的静态局限，提出达尔文演化攻击-防御框架，含通过策略发现等扩展能力的达尔文攻击及从新对抗样本中迭代学习的达尔文防护，在攻击成功率和防御召回率上表现出色。

2604.19461 2026-06-05 cs.CR 88%

Involuntary In-Context Learning: Exploiting Few-Shot Pattern Completion to Bypass Safety Alignment in GPT-5.4

强制性上下文学习：利用少样本模式完成来绕过GPT-5.4的安全对齐

Alex Polyakov, Daniel Kuznetsov

专题命中越狱攻击：alignment(title,abstract)；safety(title,abstract)

AI总结本文提出了一种称为强制性上下文学习（IICL）的攻击方法，通过使用少量示例的抽象操作符框架来强制模式完成，从而绕过大型语言模型的安全对齐训练。通过在10个OpenAI模型上进行3479次测试，研究发现语义操作符命名可以实现100%的绕过率，抽象框架是攻击的关键，而示例顺序和温度参数对攻击效果有显著影响。

2602.16729 2026-04-24 cs.CR cs.AI cs.CL cs.LG 88%

Intent Laundering: AI Safety Datasets Are Not What They Seem

意图洗白：AI安全数据集并非看起来那样

Shahriar Golchin, Marc Wetter

机构 * Applied Machine Learning Research（应用机器学习研究）

专题命中越狱攻击：safety(title,abstract)；AI safety(title)；分类 cs.CL、cs.AI、cs.LG

AI总结研究评估了广泛使用的对抗安全数据集质量，发现其过度依赖触发线索，导致模型安全评估与真实攻击行为脱节。

Comments v2 preprint: updated with more models and a new dataset

2511.10692 2025-11-17 cs.SD 88%

StyleBreak: Revealing Alignment Vulnerabilities in Large Audio-Language Models via Style-Aware Audio Jailbreak

Hongyi Li, Chengxuan Zhou, Chu Wang, Sicheng Liang, Yanting Chen, Qinlin Xie, Jiawei Ye, Jie Wu

机构 * Hongyi Li, Chengxuan Zhou, Chu Wang, Sicheng Liang, Yanting Chen, Qinlin Xie, Jiawei Ye, Jie Wu（作者团队）

专题命中越狱攻击：alignment(title,abstract)；jailbreak(title,abstract)

Comments Accepted by AAAI 2026

2411.08410 2025-03-07 cs.CR cs.CV 88%

The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense

Yangyang Guo, Fangkai Jiao, Liqiang Nie, Mohan Kankanhalli

专题命中越狱攻击：safety(title,abstract)；jailbreak(title,abstract)

Comments Logic smoothing and language polishing

2603.14355 2026-07-21 cs.CL 版本更新 87%

Exposing Long-Tail Safety Failures in Large Language Models through Efficient Diverse Response Sampling

通过高效多样响应采样暴露大语言模型中的长尾安全故障

Suvadeep Hajra, Palash Nandi, Tanmoy Chakraborty

专题命中越狱攻击：safety(title,abstract)；jailbreak(abstract,abstract_cn)；RLHF(abstract)；分类 cs.CL

AI总结本文通过多样响应生成方法系统暴露大语言模型的安全故障，提出PDPS算法在降低计算成本的同时提高攻击成功率，并生成更多多样化的不安全输出。

Comments Accepted by Transactions on Machine Learning Research (TMLR)

2606.05614 2026-06-05 cs.AI 87%

Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack

安全悖论：增强的安全意识如何使LLM易受后验攻击

Long P. Hoang, Hai V. Le, Shaoyang Xu, Wei Lu, Wenxuan Zhang

机构 * Singapore University of Technology and Design（新加坡科技设计大学）； Nanyang Technological University（南洋理工大学）

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；jailbreak(abstract)；分类 cs.AI

AI总结本文揭示安全对齐增强的LLM因内部安全评估能力而面临后验攻击漏洞，通过实验和理论分析证明安全判断能力越强越易被利用，并提出因果干预验证。

2606.00150 2026-06-02 cs.CR cs.AI 87%

Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Models

人格攻击：针对大型语言模型的增量记忆注入越狱攻击

Junyoung Park, Seongyong Ju, Sunghwan Park, Jaewoo Lee

机构 * Chung-Ang University（Chung-Ang 大学）

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；prompt injection(abstract)

AI总结提出一种基于记忆注入的越狱方法Persona Attack，通过逐步操纵模型上下文窗口，使模型在记忆积累中优先处理注入指令，从而绕过安全对齐，在特定配置下攻击成功率可达95%。

2510.12672 2025-10-20 cs.LG 87%

Keep Calm and Avoid Harmful Content: Concept Alignment and Latent Manipulation Towards Safer Answers

Ruben Belo, Marta Guimaraes, Claudia Soares

机构 * Neuraspace（Neuraspace公司）

专题命中越狱攻击：alignment(title,abstract)；safety(abstract)；jailbreak(abstract)；AI safety(abstract)

2510.01088 2025-10-02 cs.AI 87%

Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defense

Guobin Shen, Dongcheng Zhao, Haibo Tong, Jindong Li, Feifei Zhao, Yi Zeng

机构 * Beijing Institute of AI Safety and Governance（北京人工智能安全与治理研究院）； Beijing Key Laboratory of Safe AI and Superalignment（北京安全人工智能与超对齐重点实验室）； BrainCog Lab, Institute of Automation, Chinese Academy of Sciences（脑认知实验室，中国科学院自动化研究所）

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；jailbreak(abstract)；AI safety(abstract)

2403.00867 2024-11-08 cs.CR cs.AI cs.CL cs.LG 87%

Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes

Xiaomeng Hu, Pin-Yu Chen, Tsung-Yi Ho

专题命中越狱攻击：jailbreak(title,abstract)；RLHF(abstract)；safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by NeurIPS 2024. Project page: https://huggingface.co/spaces/TrustSafeAI/GradientCuff-Jailbreak-Defense

2604.24542 2026-04-28 cs.CR cs.AI cs.CL 87%

Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models

用于大语言模型运行时恶意行为检测的分层收敛指纹

Nay Myat Min, Long H. Pham, Jun Sun

机构 * Singapore Management University（新加坡管理大学）

专题命中越狱攻击：jailbreak(abstract,abstract_cn)；prompt injection(abstract,abstract_cn)；alignment(abstract)；safety(abstract)

AI总结本文提出LCF方法，通过分析神经网络层间隐藏状态轨迹来检测运行时恶意行为，无需参考模型或重训练，有效降低攻击成功率并高检测率识别多种威胁。

Comments 34 pages, 5 figures. Code: https://github.com/NayMyatMin/LCF-LLM

2605.14152 2026-07-08 cs.CL cs.AI cs.CR cs.CY 版本更新 87%

ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safety

ROK-FORTRESS：衡量地缘政治转译对国家安全和公共安全的影响

Michael S. Lee, Yash Maurya, Drew Rein, Bert Herring, Jonathan Nguyen, Kyungho Song, Udari Madhushani Sehwag, Jiyeon Cho, Kaustubh Deshpande, Yeongkyun Jang, Jiyeon Joo, Minn Seok Choi, Evi Fuelle, Christina Q. Knight, Joseph Brandifino, Max Fenkell

机构 * Scale AI

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI、cs.CY

AI总结本文提出ROK-FORTRESS基准，通过英韩语言对和美韩地缘政治轴，评估语言与地缘政治交互对国家安全的影响，发现韩语版本存在压制效应，且模型间表现差异显著。

Comments 16 pages main text + appendix (74 pages total), 4 figures and 2 tables in main text; dataset at https://huggingface.co/datasets/ScaleAI/ROK-FORTRESS_public

2604.25102 2026-04-29 cs.CV 87%

One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations

一个扰动，两种失效模式：通过嵌入引导的字形扰动探测VLM安全性

Ravikumar Balakrishnan, Sanket Mendapara

机构 * Cisco Systems（思科系统）

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；red teaming(abstract)；prompt injection(abstract)

AI总结本文通过实证研究揭示多模态嵌入距离对VLM攻击成功率的预测作用，并提出基于嵌入引导的字形扰动方法，验证了可读性与安全对齐的交互影响。

2510.22085 2025-10-28 cs.CR cs.AI cs.CL cs.LG 87%

Jailbreak Mimicry: Automated Discovery of Narrative-Based Jailbreaks for Large Language Models

Pavlos Ntais

机构 * University of Athens（雅典大学）

专题命中越狱攻击：jailbreak(title,abstract)；alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 18 pages, 5 figures

2505.13500 2025-10-14 cs.CL cs.AI cs.LG 87%

Noise Injection Systemically Degrades Large Language Model Safety Guardrails

Prithviraj Singh Shahani, Kaveh Eskandari Miandoab, Matthias Scheutz

机构 * Tufts University（塔夫茨大学）

专题命中越狱攻击：safety(title,abstract)；alignment(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages,3 figures

2501.18628 2025-08-26 cs.CR cs.AI cs.CL cs.CY 87%

TombRaider: Entering the Vault of History to Jailbreak Large Language Models

Junchen Ding, Jiahao Zhang, Yi Liu, Ziqi Ding, Gelei Deng, Yuekang Li

机构 * UNSW（新南威尔士大学）； NTU（国立大学）

专题命中越狱攻击：jailbreak(title,abstract)；safety(abstract)；red teaming(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments Main Conference of EMNLP

1. 越狱攻击 1715 篇

SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddings

SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues

SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention

H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking

Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models

Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation

Safety Alignment Backfires: Preventing the Re-emergence of Suppressed Concepts in Fine-tuned Text-to-Image Diffusion Models

Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack

Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack

GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis

Adversarial versification in portuguese as a jailbreak operator in LLMs

Foot-In-The-Door: A Multi-turn Jailbreak for LLMs

Can Large Language Models Automatically Jailbreak GPT-4V?

DARWIN: Evolving Jailbreak Adversary and Guardrail for LLM Safety Evaluation and Protection

Involuntary In-Context Learning: Exploiting Few-Shot Pattern Completion to Bypass Safety Alignment in GPT-5.4

Intent Laundering: AI Safety Datasets Are Not What They Seem

StyleBreak: Revealing Alignment Vulnerabilities in Large Audio-Language Models via Style-Aware Audio Jailbreak

The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense

Exposing Long-Tail Safety Failures in Large Language Models through Efficient Diverse Response Sampling

Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack

Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Models

Keep Calm and Avoid Harmful Content: Concept Alignment and Latent Manipulation Towards Safer Answers

Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defense

Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes

Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models

ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safety

One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations

Jailbreak Mimicry: Automated Discovery of Narrative-Based Jailbreaks for Large Language Models

Noise Injection Systemically Degrades Large Language Model Safety Guardrails

TombRaider: Entering the Vault of History to Jailbreak Large Language Models