大模型对齐与安全

FormalAlign: Automated Alignment Evaluation for Autoformalization

Jianqiao Lu, Yingjia Wan, Yinya Huang, Jing Xiong, Zhengying Liu, Zhijiang Guo

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages, 13 tables, 3 figures

Toward General Instruction-Following Alignment for Retrieval-Augmented Generation

Guanting Dong, Xiaoshuai Song, Yutao Zhu, Runqi Qiao, Zhicheng Dou, Ji-Rong Wen

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Working in progress

Revisiting the Superficial Alignment Hypothesis

Mohit Raghavendra, Vaskar Nath, Sean Hendryx

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Improving Concept Alignment in Vision-Language Concept Bottleneck Models

Nithish Muthuchamy Selvaraj, Xiaobao Guo, Adams Wai-Kin Kong, Alex Kot

专题命中 安全评测 ：alignment(title,abstract)；trustworthy(abstract)

AlignBench: Benchmarking Chinese Alignment of Large Language Models

Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Zhuoer Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2024

Emergence in Multi-Agent Systems: A Safety Perspective

Philipp Altmann, Julian Schönberger, Steffen Illium, Maximilian Zorn, Fabian Ritz, Tom Haider, Simon Burton, Thomas Gabor

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)

Comments 18 pages, 3 figures, accepted for publication at the International Symposium on Leveraging Applications of Formal Methods (ISoLA 2024)

FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcare

Karim Lekadir, Aasa Feragen, Abdul Joseph Fofanah, Alejandro F Frangi, Alena Buyx, Anais Emelie, Andrea Lara, Antonio R Porras, An-Wen Chan, Arcadi Navarro, Ben Glocker, Benard O Botwe, Bishesh Khanal, Brigit Beger, Carol C Wu, Celia Cintas, Curtis P Langlotz, Daniel Rueckert, Deogratias Mzurikwao, Dimitrios I Fotiadis, Doszhan Zhussupov, Enzo Ferrante, Erik Meijering, Eva Weicken, Fabio A González, Folkert W Asselbergs, Fred Prior, Gabriel P Krestin, Gary Collins, Geletaw S Tegenaw, Georgios Kaissis, Gianluca Misuraca, Gianna Tsakou, Girish Dwivedi, Haridimos Kondylakis, Harsha Jayakody, Henry C Woodruf, Horst Joachim Mayer, Hugo JWL Aerts, Ian Walsh, Ioanna Chouvarda, Irène Buvat, Isabell Tributsch, Islem Rekik, James Duncan, Jayashree Kalpathy-Cramer, Jihad Zahir, Jinah Park, John Mongan, Judy W Gichoya, Julia A Schnabel, Kaisar Kushibar, Katrine Riklund, Kensaku Mori, Kostas Marias, Lameck M Amugongo, Lauren A Fromont, Lena Maier-Hein, Leonor Cerdá Alberich, Leticia Rittner, Lighton Phiri, Linda Marrakchi-Kacem, Lluís Donoso-Bach, Luis Martí-Bonmatí, M Jorge Cardoso, Maciej Bobowicz, Mahsa Shabani, Manolis Tsiknakis, Maria A Zuluaga, Maria Bielikova, Marie-Christine Fritzsche, Marina Camacho, Marius George Linguraru, Markus Wenzel, Marleen De Bruijne, Martin G Tolsgaard, Marzyeh Ghassemi, Md Ashrafuzzaman, Melanie Goisauf, Mohammad Yaqub, Mónica Cano Abadía, Mukhtar M E Mahmoud, Mustafa Elattar, Nicola Rieke, Nikolaos Papanikolaou, Noussair Lazrak, Oliver Díaz, Olivier Salvado, Oriol Pujol, Ousmane Sall, Pamela Guevara, Peter Gordebeke, Philippe Lambin, Pieta Brown, Purang Abolmaesumi, Qi Dou, Qinghua Lu, Richard Osuala, Rose Nakasi, S Kevin Zhou, Sandy Napel, Sara Colantonio, Shadi Albarqouni, Smriti Joshi, Stacy Carter, Stefan Klein, Steffen E Petersen, Susanna Aussó, Suyash Awate, Tammy Riklin Raviv, Tessa Cook, Tinashe E M Mutsvangwa, Wendy A Rogers, Wiro J Niessen, Xènia Puig-Bosch, Yi Zeng, Yunusa G Mohammed, Yves Saint James Aquino, Zohaib Salahuddin, Martijn P A Starmans

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY、cs.LG

MM-Instruct: Generated Visual Instructions for Large Multimodal Model Alignment

Jihao Liu, Xin Huang, Jinliang Zheng, Boxiao Liu, Jia Wang, Osamu Yoshie, Yu Liu, Hongsheng Li

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Dataset and models are available at https://github.com/jihaonew/MM-Instruct

Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models

Abhishek Kumar, Robert Morabito, Sanzhar Umbet, Jad Kabbara, Ali Emami

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages (excluding references), accepted to ACL 2024 Main Conference

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, Jing Shao

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ACL 2024 Findings

Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models

Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, Ge Li

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL

Heterogeneous Value Alignment Evaluation for Large Language Models

Zhaowei Zhang, Ceyao Zhang, Nian Liu, Siyuan Qi, Ziqi Rong, Song-Chun Zhu, Shuguang Cui, Yaodong Yang

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments v3 is the camera-ready version for AAAI-24 Workshop on Public Sector LLMs: Algorithmic and Sociotechnical Design. 7 pages of main content, 1 page of references, 3 pages of appendices, and 7 figures. Our full prompts are released in the repo: https://github.com/zowiezhang/HVAE

JADE: A Linguistics-based Safety Evaluation Platform for Large Language Models

Mi Zhang, Xudong Pan, Min Yang

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments A preprint work. Benchmark link: https://github.com/whitzard-ai/jade-db. Website link: https://whitzard-ai.github.io/jade.html

Unveiling Safety Vulnerabilities of Large Language Models

George Kour, Marcel Zalmanovici, Naama Zwerdling, Esther Goldbraich, Ora Nova Fandina, Ateret Anaby-Tavor, Orna Raz, Eitan Farchi

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To be published in GEM workshop. Conference on Empirical Methods in Natural Language Processing (EMNLP). 2023

Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI

Muhammad Aurangzeb Ahmad, Ilker Yaramis, Taposh Dutta Roy

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Sociotechnical Safety Evaluation of Generative AI Systems

Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, William Isaac

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Comments main paper p.1-29, 5 figures, 2 tables

From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models

Jing Yao, Xiaoyuan Yi, Xiting Wang, Jindong Wang, Xing Xie

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Comments 20 pages, 5 figures

Guideline for Trustworthy Artificial Intelligence -- AI Assessment Catalog

Maximilian Poretschkin, Anna Schmitz, Maram Akila, Linara Adilova, Daniel Becker, Armin B. Cremers, Dirk Hecker, Sebastian Houben, Michael Mock, Julia Rosenzweig, Joachim Sicking, Elena Schulz, Angelika Voss, Stefan Wrobel

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY、cs.LG

DICES Dataset: Diversity in Conversational AI Evaluation for Safety

Lora Aroyo, Alex S. Taylor, Mark Diaz, Christopher M. Homan, Alicia Parrish, Greg Serapio-Garcia, Vinodkumar Prabhakaran, Ding Wang

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)

Trustworthy AI Inference Systems: An Industry Research View

Rosario Cammarota, Matthias Schunter, Anand Rajan, Fabian Boemer, Ágnes Kiss, Amos Treiber, Christian Weinert, Thomas Schneider, Emmanuel Stapf, Ahmad-Reza Sadeghi, Daniel Demmler, Joshua Stock, Huili Chen, Siam Umar Hussain, Sadegh Riazi, Farinaz Koushanfar, Saransh Gupta, Tajan Simunic Rosing, Kamalika Chaudhuri, Hamid Nejatollahi, Nikil Dutt, Mohsen Imani, Kim Laine, Anuj Dubey, Aydin Aysu, Fateme Sadat Hosseini, Chengmo Yang, Eric Wallace, Pamela Norton

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY、cs.LG

BERTMap: A BERT-based Ontology Alignment System

Yuan He, Jiaoyan Chen, Denvar Antonyrajah, Ian Horrocks

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Full version (with appendix) of the accepted paper in 36th AAAI Conference on Artificial Intelligence 2022

The relationship between trust in AI and trustworthy machine learning technologies

Ehsan Toreini, Mhairi Aitken, Kovila Coopamootoo, Karen Elliott, Carlos Gonzalez Zelaya, Aad van Moorsel

专题命中 安全评测 ：trustworthy(title)；safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Comments This submission has been accepted in ACM FAT* 2020 Conference

Alignment Verifiability in Large Language Models: Normative Indistinguishability under Behavioral Evaluation

大语言模型对齐的可验证性：行为评估下的规范不可区分性

Igor Santos-Grueiro

机构 * Igor Santos-Grueiro（独立研究者） 纠错

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.AI、cs.LG

AI总结 本文研究了大语言模型对齐的可验证性，指出行为评估无法唯一确定潜在对齐，提出了规范不可区分性概念，并通过实验验证了在评估意识下行为基准的局限性。

Comments 10 pages. Theoretical analysis of behavioral alignment evaluation

MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues

MTMCS-Bench: 多轮对话中多模态大语言模型上下文安全性的评估

Zheyuan Liu, Dongwhi Kim, Yixin Wan, Xiangchi Yuan, Zhaoxuan Tan, Fengran Mo, Meng Jiang

机构 * University of Notre Dame（诺丁汉大学） ； University of California, Los Angeles（加州大学洛杉矶分校） ； Georgia Institute of Technology（佐治亚理工学院） ； University of Montreal（蒙特利尔大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI

AI总结 MTMCS-Bench评估多模态大语言模型在多轮对话中的上下文安全性，揭示了安全与效用之间的权衡及现有防护措施的不足。

Comments A benchmark of realistic images and multi-turn conversations that evaluates contextual safety in MLLMs under two complementary settings

Trustworthy AI Must Account for Interactions

Jesse C. Cresswell

机构 * Jesse C. Cresswell（独立研究者） 纠错

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.LG；alignment(comments)

Comments Presented at the ICLR 2025 Workshop on Bidirectional Human-AI Alignment

Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments

Lorenz Linhardt, Tom Neuhäuser, Lenka Tětková, Oliver Eberle

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.LG

Comments ICLR 2025 Workshop on Representational Alignment (Re-Align)

GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abuse

Hongzhan Lin, Ziyang Luo, Bo Wang, Ruichao Yang, Jing Ma

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI

Comments The first work to benchmark Large Multimodal Models in safety insight on social media

The Journey to Trustworthy AI: Pursuit of Pragmatic Frameworks

Mohamad M Nasr-Azadani, Jean-Luc Chatelain

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY

Comments Updates: Added disclaimer about USA's recent U-turn on Trustworthy AI Executive Order. Improved Fairness and Group size in section 6.5. Fixed typos. Added a few new references. Updated title

SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety

Paul Röttger, Fabio Pernisi, Bertie Vidgen, Dirk Hovy

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI；alignment(comments)

Comments Accepted at AAAI 2025 (Special Track on AI Alignment)

RE-centric Recommendations for the Development of Trustworthy(er) Autonomous Systems

Krishna Ronanki, Beatriz Cabrero-Daniel, Jennifer Horkoff, Christian Berger

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY

Comments Accepted at [TAS '23]{First International Symposium on Trustworthy Autonomous Systems}