大模型对齐与安全

Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Space

Yao Huang, Yitong Sun, Shouwei Ruan, Yichi Zhang, Yinpeng Dong, Xingxing Wei

机构 * Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院） ； College of AI, Tsinghua University（清华大学人工智能学院） ； Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University（清华大学计算机科学与技术系） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 20 figures, accepted by ACL 2025, Findings

Beyond the Tip of Efficiency: Uncovering the Submerged Threats of Jailbreak Attacks in Small Language Models

Sibo Yi, Tianshuo Cong, Xinlei He, Qi Li, Jiaxing Song

机构 * Tsinghua University（清华大学） ； Zhongguancun Laboratory（中关村实验室） ； The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； Shandong Key Laboratory of Artificial Intelligence Security（山东省人工智能安全重点实验室） 纠错

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2025 findings

Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attacks

Yue Zhou, Henry Peng Zou, Barbara Di Eugenio, Yang Zhang

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments Accepted to the main conference of EMNLP 2024

NeuRel-Attack: Neuron Relearning for Safety Disalignment in Large Language Models

Yi Zhou, Wenpeng Xing, Dezhang Kong, Changting Lin, Meng Han

机构 * Binjiang Institute of Zhejiang University（浙江大学滨江研究院） ； South China University of Technology（华南理工大学） ； Zhejiang University（浙江大学） 纠错

专题命中 越狱攻击 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI、cs.LG

SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models

Junfeng Fang, Yukai Wang, Ruipeng Wang, Zijun Yao, Kun Wang, An Zhang, Xiang Wang, Tat-Seng Chua

专题命中 越狱攻击 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI、cs.LG

MetaSC: Test-Time Safety Specification Optimization for Language Models

Víctor Gallego

专题命中 越狱攻击 ：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Published at ICLR 2025 Workshop on Foundation Models in the Wild

Journal ref ICLR 2025 Workshop on Foundation Models in the Wild

Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processing

Johan Wahréus, Ahmed Hussain, Panos Papadimitratos

专题命中 越狱攻击 ：safety(title,abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

Comments 22 pages; 26 figures

Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring

Honglin Mu, Han He, Yuxin Zhou, Yunlong Feng, Yang Xu, Libo Qin, Xiaoming Shi, Zeming Liu, Xudong Han, Qi Shi, Qingfu Zhu, Wanxiang Che

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NAACL 2025

How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation

Zhuohang Long, Siyuan Wang, Shujun Liu, Yuhang Lai, Xuanjing Huang, Zhongyu Wei

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense

Yang Ouyang, Hengrui Gu, Shuhang Lin, Wenyue Hua, Jie Peng, Bhavya Kailkhura, Meijun Gao, Tianlong Chen, Kaixiong Zhou

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 4 figures, conference

JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models

Delong Ran, Jinyuan Liu, Yichen Gong, Jingyi Zheng, Xinlei He, Tianshuo Cong, Anyu Wang

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL、cs.AI

Comments This is the Extended Version for the Poster at NDSS Symposium 2025, Feb 24-28, 2025. Our code is available at https://github.com/ThuCCSLab/JailbreakEval

Can Safety Fine-Tuning Be More Principled? Lessons Learned from Cybersecurity

David Williams-King, Linh Le, Adam Oberman, Yoshua Bengio

专题命中 越狱攻击 ：safety(title,abstract)；jailbreak(abstract)；分类 cs.AI、cs.LG

Comments published at Neurips Safe Generative AI Workshop 2024

Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation

Qizhang Li, Xiaochen Yang, Wangmeng Zuo, Yiwen Guo

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.LG

MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialogue

Fengxiang Wang, Ranjie Duan, Peng Xiao, Xiaojun Jia, Shiji Zhao, Cheng Wei, YueFeng Chen, Chongwen Wang, Jialing Tao, Hang Su, Jun Zhu, Hui Xue

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Models

Ziwei Zheng, Junyao Zhao, Le Yang, Lijun He, Fan Li

专题命中 越狱攻击 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI、cs.LG

LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models

Miao Yu, Junfeng Fang, Yingjie Zhou, Xing Fan, Kun Wang, Shirui Pan, Qingsong Wen

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models

Muxi Diao, Rumei Li, Shiyang Liu, Guogang Liao, Jingang Wang, Xunliang Cai, Weiran Xu

专题命中 越狱攻击 ：safety(title,abstract)；red teaming(abstract)；分类 cs.CL、cs.AI

AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks

Yifan Zeng, Yiran Wu, Xiao Zhang, Huazheng Wang, Qingyun Wu

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL、cs.LG

Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue

Zhenhong Zhou, Jiuyang Xiang, Haopeng Chen, Quan Liu, Zherui Li, Sen Su

专题命中 越狱攻击 ：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments working in progress 23pages, 18 figures

Jailbreak Instruction-Tuned LLMs via end-of-sentence MLP Re-weighting

Yifan Luo, Zhennan Zhou, Meitan Wang, Bin Dong

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Distract Large Language Models for Automatic Jailbreak Attack

Zeguan Xiao, Yan Yang, Guanhua Chen, Yun Chen

专题命中 越狱攻击 ：jailbreak(title,abstract)；red teaming(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024

AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs

Lijia Lv, Weigang Zhang, Xuehai Tang, Jie Wen, Feng Liu, Jizhong Han, Songlin Hu

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL、cs.AI

Medical MLLM is Vulnerable: Cross-Modality Jailbreak and Mismatched Attacks on Medical Multimodal Large Language Models

Xijie Huang, Xinyuan Wang, Hantao Zhang, Yinghao Zhu, Jiawen Xi, Jingkun An, Hao Wang, Hao Liang, Chengwei Pan

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks

Kexin Chen, Yi Liu, Dongxia Wang, Jiaying Chen, Wenhai Wang

专题命中 越狱攻击 ：jailbreak(title,abstract)；trustworthy(abstract)；分类 cs.CL、cs.AI

ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages

Junjie Ye, Sixian Li, Guanyu Li, Caishuang Huang, Songyang Gao, Yilong Wu, Qi Zhang, Tao Gui, Xuanjing Huang

专题命中 越狱攻击 ：safety(title,abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 Main Conference

Journal ref Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics 2024 (Volume 1: Long Papers)

JailbreakHunter: A Visual Analytics Approach for Jailbreak Prompts Discovery from Large-Scale Human-LLM Conversational Datasets

Zhihua Jin, Shiyi Liu, Haotian Li, Xun Zhao, Huamin Qu

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.LG

Comments 18 pages, 9 figures

Poisoned LangChain: Jailbreak LLMs by LangChain

Ziqiu Wang, Jun Liu, Shengkai Zhang, Yang Yang

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.AI

Comments 6 pages,2 figures,This paper is a submission to ACM TURC. It has been accepted by the editor of the organizer

Emerging Safety Attack and Defense in Federated Instruction Tuning of Large Language Models

Rui Ye, Jingyi Chai, Xiangrui Liu, Yaodong Yang, Yanfeng Wang, Siheng Chen

专题命中 越狱攻击 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL、cs.AI

Comments 18 pages

Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast

Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, Min Lin

专题命中 越狱攻击 ：jailbreak(title,abstract)；safety(abstract)；分类 cs.CL、cs.LG

Comments ICML 2024

AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models

Xiaogeng Liu, Nan Xu, Muhao Chen, Chaowei Xiao

专题命中 越狱攻击 ：jailbreak(title,abstract)；alignment(abstract)；分类 cs.CL、cs.AI

Comments Published as a conference paper at ICLR 2024. Code is available at https://github.com/SheltonLiu-N/AutoDAN