大模型对齐与安全

MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programming

Chengqi Zheng, Jianda Chen, Yueming Lyu, Wen Zheng Terence Ng, Haopeng Zhang, Yew-Soon Ong, Ivor Tsang, Haiyan Yin

专题命中 安全评测 ：safety(title,abstract)；分类 cs.LG

SAGE-Eval: Evaluating LLMs for Systematic Generalizations of Safety Facts

Chen Yueh-Han, Guy Davidson, Brenden M. Lake

机构 * New York Univeristy（纽约大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.AI

MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems

Kai Chen, Taihang Zhen, Hewei Wang, Kailai Liu, Xinfeng Li, Jing Huo, Tianpei Yang, Jinfeng Xu, Wei Dong, Yang Gao

专题命中 安全评测 ：safety(title,abstract)；分类 cs.AI

Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making

Yejin Son, Minseo Kim, Sungwoong Kim, Seungju Han, Jian Kim, Dongju Jang, Youngjae Yu, Chanyoung Park

机构 * Yonsei University（延世大学） ； Stanford University（斯坦福大学） ； University of Washington, Seattle WA（华盛顿大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.AI

Comments 37 pages, 13 tables, 6 figures

CODE-DITING: A Reasoning-Based Metric for Functional Alignment in Code Evaluation

Guang Yang, Yu Zhou, Xiang Chen, Wei Zheng, Xing Hu, Xin Zhou, David Lo, Taolue Chen

机构 * College of Computer Science and Technology（计算机科学与技术学院） ； School of Artificial Intelligence and Computer Science（人工智能与计算机科学学院） ； School of Software（软件学院） ； School of Software Technology（软件技术学院） ； School of Computing and Information Systems（计算与信息系统学院） ； School of Computing and Mathematical Sciences（计算与数学科学学院） 纠错

专题命中 安全评测 ：alignment(title)；trustworthy(abstract)；分类 cs.AI

SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models

Seanie Lee, Dong Bok Lee, Dominik Wagner, Minki Kang, Haebin Seong, Tobias Bocklet, Juho Lee, Sung Ju Hwang

机构 * KAIST（韩国科学技术院） ； Technische Hochschule Nürnberg Georg Simon Ohm（图林根工业大学诺伊堡分校） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL

Comments ACL 2025 findings

Agent-SafetyBench: Evaluating the Safety of LLM Agents

Zhexin Zhang, Shiyao Cui, Yida Lu, Jingzhuo Zhou, Junxiao Yang, Hongning Wang, Minlie Huang

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL

Comments 26 pages

IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment

Chenlin Ming, Chendi Qu, Mengzhang Cai, Qizhi Pei, Zhuoshi Pan, Yu Li, Xiaoming Duan, Lijun Wu, Conghui He

机构 * Shanghai Jiao Tong University（上海交通大学） ； Shanghai AI Laboratory（上海人工智能实验室） ； Tsinghua University（清华大学） 纠错

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.AI

CellTypeAgent: Trustworthy cell type annotation with Large Language Models

Jiawen Chen, Jianghao Zhang, Huaxiu Yao, Yun Li

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI

The Value of Disagreement in AI Design, Evaluation, and Alignment

Sina Fazelpour, Will Fleisher

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CY

Comments Accepted to ACM Conference on Fairness, Accountability, and Transparency (FAccT) 2025

Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineering

Krishna Ronanki

机构 * Chalmers University of Technology — University of Gothenburg（查尔姆斯理工大学——戈尔达堡大学） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI

A Trustworthy Multi-LLM Network: Challenges,Solutions, and A Use Case

Haoxiang Luo, Gang Sun, Yinqiu Liu, Dusit Niyato, Hongfang Yu, Mohammed Atiquzzaman, Schahram Dustdar

机构 * University of Electronic Science and Technology of China（电子科技大学） ； Nanyang Technological University（南洋理工大学） ； University of Oklahoma（俄克拉荷马大学） ； TU Wien（维也纳技术大学） ； Universitat Pompeu Fabra（庞培法布拉大学） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI

Handling Imbalanced Pseudolabels for Vision-Language Models with Concept Alignment and Confusion-Aware Calibrated Margin

Yuchen Wang, Xuefeng Bai, Xiucheng Li, Weili Guan, Liqiang Nie, Xinyang Chen

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.LG

Comments Accepted to ICML 2025

OET: Optimization-based prompt injection Evaluation Toolkit

Jinsheng Pan, Xiaogeng Liu, Chaowei Xiao

机构 * University of Rochester（罗切斯特大学） ； University of Wisconsin-Madison（威斯康星大学麦迪逊分校） 纠错

专题命中 安全评测 ：prompt injection(title,abstract)；分类 cs.AI

Safety-Critical Traffic Simulation with Guided Latent Diffusion Model

Mingxing Peng, Ruoyu Yao, Xusen Guo, Yuting Xie, Xianda Chen, Jun Ma

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； School of Computer Science and Engineering（计算机科学与工程学院） ； Sun Yat-sen University（中山大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.AI

Comments 7 pages, 3 figures

Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trends

Mohammad Abu Tami, Mohammed Elhenawy, Huthaifa I. Ashqar

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL

Are Vision LLMs Road-Ready? A Comprehensive Benchmark for Safety-Critical Driving Video Understanding

Tong Zeng, Longfeng Wu, Liang Shi, Dawei Zhou, Feng Guo

机构 * Department of Computer Science, Virginia Tech（弗吉尼亚理工学院计算机科学系） ； Virginia Tech Transportation Institute, Virginia Tech（弗吉尼亚理工学院交通研究所） ； Department of Statistics, Virginia Tech（弗吉尼亚理工学院统计学系） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL

ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Models

Hengxiang Zhang, Hongfu Gao, Qiang Hu, Guanhua Chen, Lili Yang, Bingyi Jing, Hongxin Wei, Bing Wang, Haifeng Bai, Lei Yang

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL

Towards Responsible and Trustworthy Educational Data Mining: Comparing Symbolic, Sub-Symbolic, and Neural-Symbolic AI Methods

Danial Hooshyar, Eve Kikas, Yeongwook Yang, Gustav Šír, Raija Hämäläinen, Tommi Kärkkäinen, Roger Azevedo

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI

Evaluating Retrieval Augmented Generative Models for Document Queries in Transportation Safety

Chad Melton, Alex Sorokine, Steve Peterson

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL

Comments 14 pages, 3 Figures, 3 tables

Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs

Ariba Khan, Stephen Casper, Dylan Hadfield-Menell

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CY

Dynamic Evaluation Framework for Personalized and Trustworthy Agents: A Multi-Session Approach to Preference Adaptability

Chirag Shah, Hideo Joho, Kirandeep Kaur, Preetam Prabhu Srikar Dammu

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI

CliME: Evaluating Multimodal Climate Discourse on Social Media and the Climate Alignment Quotient (CAQ)

Abhilekh Borah, Hasnat Md Abdullah, Kangda Wei, Ruihong Huang

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL

Comments 16 pages, 9 figures

AI Alignment: A Comprehensive Survey

Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Lukas Vierling, Donghai Hong, Jiayi Zhou, Zhaowei Zhang, Fanzhi Zeng, Juntao Dai, Xuehai Pan, Kwan Yee Ng, Aidan O'Gara, Hua Xu, Brian Tse, Jie Fu, Stephen McAleer, Yaodong Yang, Yizhou Wang, Song-Chun Zhu, Yike Guo, Wen Gao

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.AI

Comments Continually updated

Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNN

Massimiliano Lupo Pasini, Jong Youl Choi, Kshitij Mehta, Pei Zhang, David Rogers, Jonghyun Bae, Khaled Z. Ibrahim, Ashwin M. Aji, Karl W. Schulz, Jorda Polo, Prasanna Balaprakash

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.LG

Comments 51 pages, 32 figures

The Greatest Good Benchmark: Measuring LLMs' Alignment with Utilitarian Moral Dilemmas

Giovanni Franco Gabriel Marraffini, Andrés Cotton, Noe Fabian Hsueh, Axel Fridman, Juan Wisznia, Luciano Del Corro

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL

Hierarchical Contextual Manifold Alignment for Structuring Latent Representations in Large Language Models

Meiquan Dong, Haoran Liu, Yan Huang, Zixuan Feng, Jianhong Tang, Ruoxi Wang

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL

Comments arXiv admin note: This paper has been withdrawn by arXiv due to disputed and unverifiable authorship

Enhanced OoD Detection through Cross-Modal Alignment of Multi-Modal Representations

Jeonghyeon Kim, Sangheum Hwang

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.AI

Comments CVPR 2025

Safety Aware Task Planning via Large Language Models in Robotics

Azal Ahmad Khan, Michael Andrev, Muhammad Ali Murtaza, Sergio Aguilera, Rui Zhang, Jie Ding, Seth Hutchinson, Ali Anwar

专题命中 安全评测 ：safety(title,abstract)；分类 cs.AI

FiVL: A Framework for Improved Vision-Language Alignment through the Lens of Training, Evaluation and Explainability

Estelle Aflalo, Gabriela Ben Melech Stan, Tiep Le, Man Luo, Shachar Rosenman, Sayak Paul, Shao-Yen Tseng, Vasudev Lal

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.AI