大模型对齐与安全

SciTrust 2.0: A Comprehensive Framework for Evaluating Trustworthiness of Large Language Models in Scientific Applications

Emily Herron, Junqi Yin, Feiyi Wang

机构 * Oak Ridge National Laboratory（橡树岭国家实验室） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI

Comments Preprint Submitted to ACM Transactions on AI for Science (TAIS)

OpenGuardrails: A Configurable, Unified, and Scalable Guardrails Platform for Large Language Models

Thomas Wang, Haowen Li

机构 * The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 安全评测 ：safety(abstract)；prompt injection(abstract)；分类 cs.CL

A Pragmatic Way to Measure Chain-of-Thought Monitorability

Scott Emmons, Roland S. Zimmermann, David K. Elson, Rohin Shah

机构 * Google（谷歌） 纠错

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.LG

Comments The first two authors contributed equally

Can Large Language Models Unlock Novel Scientific Research Ideas?

Sandeep Kumar, Tirthankar Ghosal, Vinayak Goyal, Asif Ekbal

机构 * Department of Computer Science and Engineering, Indian Institute of Technology Patna（计算机科学与工程系，印度理工学院帕纳布分校） ； National Center for Computational Sciences, Oak Ridge National Laboratory（计算科学国家中心，橡树岭国家实验室） 纠错

专题命中 安全评测 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments EMNLP 2025 (Main)

S-Chain: Structured Visual Chain-of-Thought For Medicine

Khai Le-Duc, Duy M. H. Nguyen, Phuong T. H. Trinh, Tien-Phat Nguyen, Nghiem T. Diep, An Ngo, Tung Vu, Trinh Vuong, Anh-Tien Nguyen, Mau Nguyen, Van Trung Hoang, Khai-Nguyen Nguyen, Hy Nguyen, Chris Ngo, Anji Liu, Nhat Ho, Anne-Christin Hauschild, Khanh Xuan Nguyen, Thanh Nguyen-Tang, Pengtao Xie, Daniel Sonntag, James Zou, Mathias Niepert, Anh Totti Nguyen

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.LG

Comments First version

EU-Agent-Bench: Measuring Illegal Behavior of LLM Agents Under EU Law

Ilija Lichkovski, Alexander Müller, Mariam Ibrahim, Tiwai Mhundwa

机构 * AI Safety Initiative Groningen（格罗宁根人工智能安全倡议） 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments Accepted at the Workshop on Regulatable ML at the 39th Conference on Neural Information Processing Systems (NeurIPS 2025)

Quantifying CBRN Risk in Frontier Models

Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa, Sahil Agarwal, Prashanth Harshangi

机构 * Enkrypt AI 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.AI

TRUST: A Decentralized Framework for Auditing Large Language Model Reasoning

Morris Yu-Chao Huang, Zhen Tan, Mohan Zhang, Pingzhi Li, Zhuo Zhang, Tianlong Chen

专题命中 安全评测 ：harmlessness(abstract)；trustworthy(abstract)；分类 cs.AI

Subliminal Corruption: Mechanisms, Thresholds, and Interpretability

Reya Vir, Sarvesh Bhatnagar

机构 * Department of Computer Science, Columbia University, New York, NY, USA（哥伦比亚大学计算机科学系） ； Department of Computer Science（计算机科学系） ； Engineering, University of Michigan, MI, USA（密歇根大学工程学院） 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.LG

A Vision for Access Control in LLM-based Agent Systems

Xinfeng Li, Dong Huang, Jie Li, Hongyi Cai, Zhenhong Zhou, Wei Dong, XiaoFeng Wang, Yang Liu

机构 * Nanyang Technological University, Singapore（南洋理工大学，新加坡） ； National University of Singapore, Singapore（国立新加坡大学，新加坡） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI

Comments 11 pages, 1 figure

Taming a Retrieval Framework to Read Images in Humanlike Manner for Augmenting Generation of MLLMs

Suyang Xi, Chenxi Yang, Hong Ding, Yiqing Ni, Catherine C. Liu, Yunhao Liu, Chengqi Zhang

机构 * Emory University（埃默里大学） ； University of Electronic Science and Technology of China（电子科技大学） ； University of Illinois Chicago（伊利诺伊大学香槟分校） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.AI

Comments 12 pages, 5 figures

Optimizing Ethical Risk Reduction for Medical Intelligent Systems with Constraint Programming

Clotilde Brayé, Aurélien Bricout, Arnaud Gotlieb, Nadjib Lazaar, Quentin Vallet

机构 * Enovacom ； Simula Research Laboratory ； LISN, Université Paris-Saclay（LISN，巴黎萨克雷大学） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI

Sherkala-Chat: Building a State-of-the-Art LLM for Kazakh in a Moderately Resourced Setting

Fajri Koto, Rituraj Joshi, Nurdaulet Mukhituly, Yuxia Wang, Zhuohan Xie, Rahul Pal, Daniil Orel, Parvez Mullah, Diana Turmakhan, Maiya Goloburda, Mohammed Kamran, Samujjwal Ghosh, Bokang Jia, Jonibek Mansurov, Mukhammed Togmanov, Debopriyo Banerjee, Nurkhan Laiyk, Akhmed Sakip, Xudong Han, Ekaterina Kochmar, Alham Fikri Aji, Aaryamonvikram Singh, Alok Anil Jadhav, Satheesh Katipomu, Samta Kamboj, Monojit Choudhury, Gurpreet Gosal, Gokulakrishnan Ramakrishnan, Biswajit Mishra, Sarath Chandran, Avraham Sheinin, Natalia Vassilieva, Neha Sengupta, Preslav Nakov

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） ； Inception ； Cerebras Systems（Cerebras系统） 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.CL

Comments Accepted at COLM 2025

Autonomous Data Agents: A New Opportunity for Smart Data

Yanjie Fu, Dongjie Wang, Wangyang Ying, Xinyuan Wang, Xiangliang Zhang, Huan Liu, Jian Pei

机构 * Arizona State University（亚利桑那州立大学） ； University of Kansas（堪萨斯大学） ； University of Notre Dame（圣母大学） ； Duke University（杜克大学） 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.AI

Say One Thing, Do Another? Diagnosing Reasoning-Execution Gaps in VLM-Powered Mobile-Use Agents

Lingzhong Dong, Ziqi Zhou, Shuaibo Yang, Haiyue Sheng, Pengzhou Cheng, Zongru Wu, Zheng Wu, Gongshen Liu, Zhuosheng Zhang

机构 * Shanghai Jiao Tong University（上海交通大学） ； Beijing Institute of Technology（北京理工大学） 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.CL

Human-AI Teaming Co-Learning in Military Operations

Clara Maathuis, Kasper Cools

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI

Comments Submitted to Sensors + Imaging; presented on 18th of September (Artificial Intelligence for Security and Defence Applications III)

Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness

Erfan Shayegani, Keegan Hines, Yue Dong, Nael Abu-Ghazaleh, Roman Lutz, Spencer Whitehead, Vidhisha Balachandran, Besmira Nushi, Vibhav Vineet

机构 * Microsoft Research AI Frontiers（微软研究院人工智能前沿） ； Microsoft AI Red Team（微软AI红色团队） ； University of California, Riverside（加州大学河滨分校） ； NVIDIA（英伟达） 纠错

专题命中 安全评测 ：safety(abstract)；分类 cs.CL、cs.AI、cs.CY

Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundation

Wei Liu, Haomei Xu, Bingqing Liu, Zhiying Deng, Haozhao Wang, Jun Wang, Ruixuan Li, Yee Whye Teh, Wee Sun Lee

机构 * National University of Singapore（新加坡国立大学） ； Huazhong University of Science and Technology（华中科技大学） ； Central China Normal University（中国地质大学） ； iWudao Tech（iWudao科技） ； Oxford（牛津大学） ； Google Deepmind（谷歌DeepMind） 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments This is a work in progress. Comments and suggestions are welcome

CORE: Full-Path Evaluation of LLM Agents Beyond Final State

Panagiotis Michelakis, Yiannis Hadjiyiannis, Dimitrios Stamoulis

机构 * Synkrasis Labs（Synkrasis实验室） ； Harbin Institute of Technology（哈尔滨工业大学） 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.AI

Comments Accepted: LAW 2025 Workshop NeurIPS 2025

EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models

Botai Yuan, Yutian Zhou, Yingjie Wang, Fushuo Huo, Yongcheng Jing, Li Shen, Ying Wei, Zhiqi Shen, Ziwei Liu, Tianwei Zhang, Jie Yang, Dacheng Tao

机构 * Nanyang Technological University（南洋理工大学） ； Shanghai Jiao Tong University（上海交通大学） ； Fudan University（复旦大学） ； Sun Yat-sen University（中山大学） ； Zhejiang University（浙江大学） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI

Comments 29 pages, 6 figures

How Much of Your Data Can Suck? Thresholds for Domain Performance and Emergent Misalignment in LLMs

Jian Ouyang, Arman T, Ge Jin

机构 * Invisible Technologies（隐形技术） 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.CL

LLMZ+: Contextual Prompt Whitelist Principles for Agentic LLMs

Tom Pawelek, Raj Patel, Charlotte Crowell, Noorbakhsh Amiri, Sudip Mittal, Shahram Rahimi, Andy Perkins

机构 * Department of Computer Science Mississippi State University（计算机科学系密苏里州立大学） ； Department of Computer Science The University of Alabama（计算机科学系阿拉巴马大学） ； Mississippi State University, Mississippi State, MS, USA（密苏里州立大学） ； The University of Alabama, Tuscaloosa, AL, USA（阿拉巴马大学） 纠错

专题命中 安全评测 ：jailbreak(abstract)；prompt injection(abstract)；分类 cs.AI

Comments 7 pages, 5 figures, to be published and presented at ICMLA 2025

Medical AI Consensus: A Multi-Agent Framework for Radiology Report Generation and Evaluation

Ahmed T. Elboardy, Ghada Khoriba, Essam A. Rashed

机构 * Graduate School of Information Science, University of Hyogo（京都大学垣田学园信息科学研究生院） ； Center for Informatics Science, School of Information Technology and Computer Science, Nile University（尼罗大学信息科学中心） 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.AI

Comments NeurIPS2025 Workshop: Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling

Audited Reasoning Refinement: Fine-Tuning Language Models via LLM-Guided Step-Wise Evaluation and Correction

Sumanta Bhattacharyya, Sara Riazi, Pedram Rooshenas

专题命中 安全评测 ：alignment(abstract)；DPO(abstract)；分类 cs.CL

MedicalOS: An LLM Agent based Operating System for Digital Healthcare

Jared Zhu, Junde Wu

机构 * University of Oxford（牛津大学） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI

GeoGPT-RAG Technical Report

Fei Huang, Fan Wu, Zeqing Zhang, Qihao Wang, Long Zhang, Grant Michael Boquet, Hongyang Chen

机构 * GeoGPT Team（GeoGPT团队） ； Zhejiang Lab（浙江实验室） 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.AI

Comments 19 pages, 10 figures, 10 tables

Triadic Fusion of Cognitive, Functional, and Causal Dimensions for Explainable LLMs: The TAXAL Framework

David Herrera-Poyatos, Carlos Peláez-González, Cristina Zuheros, Virilo Tejedor, Rosana Montes, Francisco Herrera

机构 * Department of Computer Science and Artificial Intelligence（计算机科学与人工智能系） ； Andalusian Institute of Data Science and Computational Intelligence（安达卢西亚数据科学与计算智能研究所） ； University of Granada（格拉纳达大学） 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.CL

Comments 27 pages, 9 tables and 2 figures

Internet 3.0: Architecture for a Web-of-Agents with it's Algorithm for Ranking Agents

Rajesh Tembarai Krishnamachari, Srividya Rajesh

机构 * NYU（纽约大学） ； Independent Researcher（独立研究者） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI

MAD Chairs: A new tool to evaluate AI

Chris Santos-Lang

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.CY

Comments 17 pages, 1 figure, reproduced with permission from Springer Nature from Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XVIII (COINE 2025)

LLM Embedding-based Attribution (LEA): Quantifying Source Contributions to Generative Model's Response for Vulnerability Analysis

Reza Fayyazi, Michael Zuzak, Shanchieh Jay Yang

机构 * Rochester Institute of Technology（罗切斯特技术学院） ； Gonzaga University（戈兹加大学） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI