大模型对齐与安全

OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents

Thomas Kuntz, Agatha Duzan, Hao Zhao, Francesco Croce, Zico Kolter, Nicolas Flammarion, Maksym Andriushchenko

机构 * EPFL（苏黎世联邦理工学院） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；prompt injection(abstract)；分类 cs.LG

Comments NeurIPS 2025 Datasets & Benchmarks Track (Spotlight)

Adaptive Blockwise Search: Inference-Time Alignment for Large Language Models

Mohammad Atif Quamar, Mohammad Areeb, Nishant Sharma, Ananth Shreekumar, Jonathan Rosenthal, Muslum Ozgur Ozmen, Mikhail Kuznetsov, Z. Berkay Celik

机构 * Purdue University（普渡大学） ； Arizona State University（亚利桑那州立大学） ； Amazon（亚马逊） 纠错

专题命中 安全评测 ：alignment(title,abstract)；harmlessness(abstract)；分类 cs.CL

SAGE: A Generic Framework for LLM Safety Evaluation

Madhur Jindal, Hari Shrawgi, Parag Agrawal, Sandipan Dandapat

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Comments Accepted to EMNLP 2025

Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety

Yubin Kim, Hyewon Jeong, Chanwoo Park, Eugene Park, Haipeng Zhang, Xin Liu, Hyeonhoon Lee, Daniel McDuff, Marzyeh Ghassemi, Cynthia Breazeal, Samir Tulebaev, Hae Won Park

机构 * Massachusetts Institute of Technology（麻省理工学院） ； Google Research（谷歌研究） ； Harvard Medical School（哈佛医学院） ； Seoul National University Hospital（首尔国立大学医院） 纠错

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models

Leyi Pan, Zheyu Fu, Yunpeng Zhai, Shuchang Tao, Sheng Guan, Shiyu Huang, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Felix Henry, Aiwei Liu, Lijie Wen

机构 * Tsinghua University（清华大学） ； Tongyi Lab（通义实验室） ； Peking University（北京大学） ； OpenRL Lab（OpenRL实验室） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Comments 22 pages, 10 figures, 12 tables

Safety and Security Analysis of Large Language Models: Benchmarking Risk Profile and Harm Potential

Charankumar Akiri, Harrison Simpson, Kshitiz Aryal, Aarav Khanna, Maanak Gupta

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CY

AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Models

Sayantan Adak, Pratyush Chatterjee, Somnath Banerjee, Rima Hazra, Somak Aditya, Animesh Mukherjee

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.CL

Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Models

Wanying Wang, Zeyu Ma, Han Zheng, Xin Tan, Mingang Chen

机构 * Shanghai Key Laboratory of Computer Software Testing and Evaluating（上海软件测试与评估 key laboratory） ； Shanghai Normal University（上海Normal University） ； TrustAI Pte. Ltd. ； East China Normal University（东华师范大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Comments Accepted by ACM Multimedia 2025

ELITE: Enhanced Language-Image Toxicity Evaluation for Safety

Wonjun Lee, Doehyeon Lee, Eugene Choi, Sangyoon Yu, Ashkan Yousefpour, Haon Park, Bumsub Ham, Suhyun Kim

机构 * Yonsei University（延世大学） ； Kyung Hee University（庆熙大学） ； Korea Institute of Science（韩国科学研究院） ； Seoul National University（首尔国立大学） ； Sookmyung Women's University（_sookmyung女子大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Comments ICML 2025. Project page at https://velpegor.github.io/ELITE/

Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Framework

Rishane Dassanayake, Mario Demetroudi, James Walpole, Lindley Lentati, Jason R. Brown, Edward James Young

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Comments 24 pages (14 pages main text, 4 pages bibliography, 6 pages appendices), 3 figures

CALMA: A Process for Deriving Context-aligned Axes for Language Model Alignment

Prajna Soni, Deepika Raman, Dylan Hadfield-Menell

专题命中 安全评测 ：alignment(title,abstract)；harmlessness(abstract)；分类 cs.CY

Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework

Satyapriya Krishna, Ninareh Mehrabi, Abhinav Mohanty, Matteo Memelli, Vincent Ponzo, Payal Motwani, Rahul Gupta

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.CY

Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models

Artyom Kharinaev, Viktor Moskvoretskii, Egor Shvetsov, Kseniia Studenikina, Bykov Mikhail, Evgeny Burnaev

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency

Kathleen C. Fraser, Hillary Dawkins, Isar Nejadgholi, Svetlana Kiritchenko

机构 * National Research Council Canada（加拿大国家研究理事会） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Comments to appear at LLMSEC 2025

From Theory to Practice: Real-World Use Cases on Trustworthy LLM-Driven Process Modeling, Prediction and Automation

Peter Pfeiffer, Alexander Rombach, Maxim Majlatow, Nijat Mehdiyev

机构 * German Research Center for Artificial Intelligence (DFKI) and Saarland University（德国人工智能研究中心（DFKI）和萨尔兰州大学） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.LG

Comments Accepted to the Next Gen Data and Process Management: Large Language Models and Beyond workshop at SIGMOD 2025

Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap

Wenhan Yang, Spencer Stice, Ali Payani, Baharan Mirzasoleiman

机构 * Computer Science Department UCLA（计算机科学系，加州大学洛杉矶分校） ； Cisco Systems Inc.（思科系统公司） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It

Zheng-Xin Yong, Beyza Ermis, Marzieh Fadaee, Stephen H. Bach, Julia Kreutzer

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.CL

Reshaping Representation Space to Balance the Safety and Over-rejection in Large Audio Language Models

Hao Yang, Lizhen Qu, Ehsan Shareghi, Gholamreza Haffari

机构 * Department of Data Science & AI, Monash University（数据科学与人工智能系，莫纳什大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.CL

Mitigating Deceptive Alignment via Self-Monitoring

Jiaming Ji, Wenqi Chen, Kaile Wang, Donghai Hong, Sitong Fang, Boyuan Chen, Jiayi Zhou, Juntao Dai, Sirui Han, Yike Guo, Yaodong Yang

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.AI

From nuclear safety to LLM security: Applying non-probabilistic risk management strategies to build safe and secure LLM-powered systems

Alexander Gutfraind, Vicki Bier

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.AI

Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration

Honglong Yang, Shanshan Song, Yi Qin, Lehan Wang, Haonan Wang, Xinpeng Ding, Qixiang Zhang, Bodong Du, Xiaomeng Li

机构 * Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学） ； Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；alignment(abstract)；分类 cs.CL

aiXamine: Simplified LLM Safety and Security

Fatih Deniz, Dorde Popovic, Yazan Boshmaf, Euisuh Jeong, Minhaj Ahmad, Sanjay Chawla, Issa Khalil

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)；分类 cs.AI

Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation

Jiajun Shen, Tong Zhou, Yubo Chen, Delai Qiu, Shengping Liu, Kang Liu, Jun Zhao

机构 * The Key Laboratory of Cognition and Decision Intelligence for Complex Systems Institute of Automation, Chinese Academy of Sciences（认知与决策智能复杂系统重点实验室，中国科学院自动化研究所） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； University of Chinese Academy of Sciences（中国科学院大学） ； Unisound Al Technology Co,Ltd（Unisound人工智能技术有限公司） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；alignment(abstract)；分类 cs.CL

Comments 19 pages, 14 figures

SafeChat: A Framework for Building Trustworthy Collaborative Assistants and a Case Study of its Usefulness

Biplav Srivastava, Kausik Lakkaraju, Nitin Gupta, Vansh Nagpal, Bharath C. Muppasani, Sara E. Jones

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.CL

AI-University: An LLM-based platform for instructional alignment to scientific classrooms

Mostafa Faghih Shojaei, Rahul Gulati, Benjamin A. Jasperson, Shangshang Wang, Simone Cimolato, Dangli Cao, Willie Neiswanger, Krishna Garikipati

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Comments 10 pages, 3 figures

Routing to the Right Expertise: A Trustworthy Judge for Instruction-based Image Editing

Chenxi Sun, Hongzhi Zhang, Qi Wang, Fuzheng Zhang

专题命中 安全评测 ：trustworthy(title,abstract)；alignment(abstract)；分类 cs.AI

Augmenting deep neural networks with symbolic knowledge: Towards trustworthy and interpretable AI for education

Danial Hooshyar, Roger Azevedo, Yeongwook Yang

专题命中 安全评测 ：trustworthy(title,abstract)；safety(abstract)；分类 cs.AI

You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisation

Simon Pepin Lehalleur, Jesse Hoogland, Matthew Farrugia-Roberts, Susan Wei, Alexander Gietelink Oldenziel, George Wang, Liam Carroll, Daniel Murfet

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.LG

Safety case template for frontier AI: A cyber inability argument

Arthur Goemans, Marie Davidsen Buhl, Jonas Schuett, Tomek Korbak, Jessica Wang, Benjamin Hilton, Geoffrey Irving

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)；分类 cs.CY

Comments 21 pages, 6 figures, 1 table

HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment

Yannis Belkhiter, Giulio Zizzo, Sergio Maffeis

专题命中 安全评测 ：alignment(title,abstract)；safety(abstract)；分类 cs.CL

Comments NeurIPS 2024 Workshop on Safe Generative Artificial Intelligence (SafeGenAI)