大模型对齐与安全

Runtime Compliance Verification for AI Agents

AI代理的运行时合规性验证

Nafiseh Kahani, Masoud Barati, Diana Addae

专题命中 红队测试 ：red teaming(abstract)

AI总结 提出C-Trace框架，通过运行时监控和形式化策略谓词，确保AI代理在工具调用和对话中遵守GDPR规则，将攻击成功率降至12%以下。

From Rookie to Expert: Manipulating LLMs for Automated Vulnerability Exploitation in Enterprise Software

从新手到专家：利用LLMs实现企业软件自动漏洞利用

Moustapha Awwalou Diouf, Maimouna Tamah Diao, Iyiola Emmanuel Olatunji, Abdoul Kader Kaboré, Jordan Samhi, Gervais Mendy, Samuel Ouya, Jacques Klein, Tegawendé F. Bissyandé

专题命中 红队测试 ：safety(abstract)

AI总结 本文展示如何利用公开的LLMs将新手转变为具备攻击能力的用户，提出RSA策略以绕过安全机制，验证了LLMs在漏洞利用中的有效性，挑战了传统安全原则。

Diversifying Toxicity Search in Large Language Models Through Speciation

通过种群化扩大大型语言模型毒性搜索

Onkar Shelar, Travis Desell

专题命中 红队测试 ：red teaming(abstract)

AI总结 本文提出ToxSearch-S，通过种群化方法在大型语言模型中扩展毒性搜索，提高毒性峰值并扩大语义覆盖范围，同时在嵌入空间中形成行为差异化的niche。

Comments Preprint. 4 pages, Accepted at GECCO as short paper

Evil Vizier: Vulnerabilities of LLM-Integrated XR Systems

Yicheng Zhang, Zijian Huang, Sophie Chen, Erfan Shayegani, Jiasi Chen, Nael Abu-Ghazaleh

专题命中 红队测试 ：safety(abstract)

AEAS: Actionable Exploit Assessment System

Xiangmin Shen, Wenyuan Cheng, Yan Chen, Zhenyuan Li, Yuqiao Gu, Lingzhi Wang, Wencheng Zhao, Dawei Sun, Jiashui Wang

专题命中 红队测试 ：alignment(abstract)

Comments AEAS has been implemented in the planning agent of PentestAgent, our LLM-driven automated penetration testing framework. Check out our repository: https://github.com/nbshenxm/pentest-agent

Extending the OWASP Multi-Agentic System Threat Modeling Guide: Insights from Multi-Agent Security Research

Klaudia Krawiecka, Christian Schroeder de Witt

专题命中 红队测试 ：safety(abstract)

Autonomous Vehicle Security: A Deep Dive into Threat Modeling

Amal Yousseef, Shalaka Satam, Banafsheh Saber Latibari, Jesus Pacheco, Soheil Salehi, Salim Hariri, Partik Satam

专题命中 红队测试 ：safety(abstract)

Threat Modelling and Risk Analysis for Large Language Model (LLM)-Powered Applications

Stephen Burabari Tete

专题命中 红队测试 ：prompt injection(abstract)

Bayesian Methods for Trust in Collaborative Multi-Agent Autonomy

R. Spencer Hallyburton, Miroslav Pajic

专题命中 红队测试 ：safety(abstract)

Needle in a Haystack: Detecting Subtle Malicious Edits to Additive Manufacturing G-code Files

Caleb Beckwith, Harsh Sankar Naicker, Svara Mehta, Viba R. Udupa, Nghia Tri Nim, Varun Gadre, Hammond Pearce, Gary Mac, Nikhil Gupta

专题命中 红队测试 ：safety(abstract)

Comments To appear in IEEE Embedded Systems Letters