大模型对齐与安全

CheatAgent: Attacking LLM-Empowered Recommender Systems via LLM Agent

Liang-bo Ning, Shijie Wang, Wenqi Fan, Qing Li, Xin Xu, Hao Chen, Feiran Huang

机构 * The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 越狱攻击 ：safety(abstract)；分类 cs.AI

Comments Accepted by KDD 2024;

Foundation Models in Federated Learning: Assessing Backdoor Vulnerabilities

Xi Li, Chen Wu, Jiaqi Wang

机构 * University of Alabama at Birmingham（阿拉巴马大学伯明翰分校） ； Meta ； The Pennsylvania State University（宾夕法尼亚州立大学） 纠错

专题命中 越狱攻击 ：safety(abstract)；分类 cs.LG

Comments Xi Li and Chen Wu are equal contribution. The corresponding author is Jiaqi Wang. This paper has been accepted by IJCNN 2025

Exploring Backdoor Attack and Defense for LLM-empowered Recommendations

Liangbo Ning, Wenqi Fan, Qing Li

专题命中 越狱攻击 ：safety(abstract)；分类 cs.AI

Ctrl-Z: Controlling AI Agents via Resampling

Aryan Bhatt, Cody Rushing, Adam Kaufman, Tyler Tracy, Vasil Georgiev, David Matolcsi, Akbir Khan, Buck Shlegeris

专题命中 越狱攻击 ：safety(abstract)；分类 cs.LG

Comments bashcontrol.com

StruPhantom: Evolutionary Injection Attacks on Black-Box Tabular Agents Powered by Large Language Models

Yang Feng, Xudong Pan

专题命中 越狱攻击 ：prompt injection(abstract)；分类 cs.AI

Comments Work in Progress

Let the Noise Speak: Harnessing Noise for a Unified Defense Against Adversarial and Backdoor Attacks

Md Hasan Shahriar, Ning Wang, Naren Ramakrishnan, Y. Thomas Hou, Wenjing Lou

专题命中 越狱攻击 ：safety(abstract)；分类 cs.LG

Comments 20 pages, 9 figures

Defending Large Language Models Against Attacks With Residual Stream Activation Analysis

Amelia Kawasaki, Andrew Davis, Houssam Abbas

专题命中 越狱攻击 ：safety(abstract)；分类 cs.LG

Comments Included in Proceedings of the Conference on Applied Machine Learning in Information Security (CAMLIS 2024), Arlington, Virginia, USA, October 24-25, 2024

Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors

Yuefeng Peng, Junda Wang, Hong Yu, Amir Houmansadr

专题命中 越狱攻击 ：prompt injection(abstract)；分类 cs.CL

Securing Virtual Reality Experiences: Unveiling and Tackling Cybersickness Attacks with Explainable AI

Ripan Kumar Kundu, Matthew Denton, Genova Mongalo, Prasad Calyam, Khaza Anuarul Hoque

专题命中 越狱攻击 ：safety(abstract)；分类 cs.AI

Comments This work has been submitted to the IEEE for possible publication

Jailbreaking Large Language Models in Infinitely Many Ways

Oliver Goldstein, Emanuele La Malfa, Felix Drinkall, Samuele Marro, Michael Wooldridge

专题命中 越狱攻击 ：safety(abstract)；分类 cs.LG

Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation

Wenlong Meng, Fan Zhang, Wendao Yao, Zhenyuan Guo, Yuwei Li, Chengkun Wei, Wenzhi Chen

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.CL

Comments 17 pages, 10 figures

Enhancing Network Security Management in Water Systems using FM-based Attack Attribution

Aleksandar Avdalovic, Joseph Khoury, Ahmad Taha, Elias Bou-Harb

专题命中 越狱攻击 ：safety(abstract)；分类 cs.LG

PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMs

Xueluan Gong, Mingzhe Li, Yilin Zhang, Fengyuan Ran, Chen Chen, Yanjiao Chen, Qian Wang, Kwok-Yan Lam

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.AI

Attacking Large Language Models with Projected Gradient Descent

Simon Geisler, Tom Wollschläger, M. H. I. Abdalla, Johannes Gasteiger, Stephan Günnemann

专题命中 越狱攻击 ：alignment(abstract)；分类 cs.LG

Efficient Jailbreaking of Large Models by Freeze Training: Lower Layers Exhibit Greater Sensitivity to Harmful Content

Hongyuan Shen, Min Zheng, Jincheng Wang, Yang Zhao

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.LG

Swallowing the Poison Pills: Insights from Vulnerability Disparity Among LLMs

Peng Yifeng, Wu Zhizheng, Chen Chen

专题命中 越狱攻击 ：safety(abstract)；分类 cs.AI

Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoders

Xuansheng Wu, Jiayi Yuan, Wenlin Yao, Xiaoming Zhai, Ninghao Liu

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.CL

Comments Pre-print. 20 pages, 5 figures

Interpreting Adversarial Attacks and Defences using Architectures with Enhanced Interpretability

Akshay G Rao, Chandrashekhar Lakshminarayanan, Arun Rajkumar

专题命中 越狱攻击 ：alignment(abstract)；分类 cs.LG

Comments Publication accepted at AAAI Deployable AI conference 2025 (proof - accepted-papers?authuser=0" target="_blank" rel="noopener">https://sites.google.com/view/dai-2025/accepted-papers?authuser=0) Total 17 pages

Data to Defense: The Role of Curation in Customizing LLMs Against Jailbreaking Attacks

Xiaoqun Liu, Jiacheng Liang, Luoxi Tang, Muchao Ye, Weicheng Ma, Zhaohan Xi

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.AI

G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems

Shilong Wang, Guibin Zhang, Miao Yu, Guancheng Wan, Fanci Meng, Chongye Guo, Kun Wang, Yang Wang

专题命中 越狱攻击 ：prompt injection(abstract)；分类 cs.LG

Prompt Inject Detection with Generative Explanation as an Investigative Tool

Jonathan Pan, Swee Liang Wong, Yidi Yuan, Xin Wei Chia

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.AI

Comments 5 pages, 4 tables, 3 diagrams

Self-Instruct Few-Shot Jailbreaking: Decompose the Attack into Pattern and Behavior Learning

Jiaqi Hua, Wanxu Wei

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.AI

Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreaking

Stav Cohen, Ron Bitton, Ben Nassi

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.AI

Comments for Github, see https://github.com/StavC/UnleashingWorms-ExtractingData . arXiv admin note: substantial text overlap with arXiv:2403.02817

Learning-based Detection of GPS Spoofing Attack for Quadrotors

Pengyu Wang, Zhaohua Yang, Jialu Li, Ling Shi

专题命中 越狱攻击 ：safety(abstract)；分类 cs.LG

Comments Accepted in IEEE Industrial Electronics Society Annual Online Conference

Jailbreaking? One Step Is Enough!

Weixiong Zheng, Peijian Zeng, Yiwei Li, Hongyan Wu, Nankai Lin, Junhao Chen, Aimin Yang, Yongmei Zhou

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.CL

Comments 17 pages

Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation

Xuying Li, Zhuo Li, Yuji Kosuga, Yasuhiro Yoshida, Victor Bian

专题命中 越狱攻击 ：safety(abstract)；分类 cs.AI

Recent Advances in Attack and Defense Approaches of Large Language Models

Jing Cui, Yishi Xu, Zhewei Huang, Shuchang Zhou, Jianbin Jiao, Junge Zhang

专题命中 越狱攻击 ：safety(abstract)；分类 cs.AI

Harnessing LLM to Attack LLM-Guarded Text-to-Image Models

Yimo Deng, Huangxun Chen

专题命中 越狱攻击 ：safety(abstract)；分类 cs.AI

Comments 10 pages, 7 figures, under review

Insights and Current Gaps in Open-Source LLM Vulnerability Scanners: A Comparative Analysis

Jonathan Brokman, Omer Hofman, Oren Rachmil, Inderjeet Singh, Vikas Pahuja, Rathina Sabapathy Aishvariya Priya, Amit Giloni, Roman Vainshtein, Hisashi Kojima

专题命中 越狱攻击 ：jailbreak(abstract)；分类 cs.LG

Comments 15 pages, 11 figures

Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Models

Pete Janowczyk, Linda Laurier, Ave Giulietta, Arlo Octavia, Meade Cleti

专题命中 越狱攻击 ：safety(abstract)；分类 cs.AI