Comments62 pages (31-page article and 31-page supplementary information), 8 figures, 4 tables. v3: corrects the author metadata to the sole author, Kwan Soo Shin; revised title and abstract; adds cross-vendor and flagship validation, signal-detection and specified-task controls, and dual-process probes. Reproducibility deposit: doi:10.5281/zenodo.20826823
MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applications
MEDIC:对LLM在临床应用中的安全性和实用性领先指标的综合评估
Praveenkumar Kanithi, Clément Christophe, Marco AF Pimentel, Tathagata Raha, Prateek Munjal, Nada Saadi, Hamza A Javed, Svetlana Maslenkova, Nasir Hayat, Ronnie Rajan, Shadab Khan
First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations
首先,不伤害:迈向临床安全的大语言模型
David Wu, Fateme Nateghi Haredasht, Saloni Kumar Maharaj, Priyank Jain, Jessica Tran, Matthew Gwiazdon, Arjun Rustagi, Jenelle Jindal, Jacob M. Koshy, Vinay Kadiyala, Anup Agarwal, Bassman Tappuni, Brianna French, Sirus Jesudasen, Christopher V. Cosgriff, Rebanta Chakraborty, Jillian Caldwell, Susan Ziolkowski, David J. Iberri, Robert Diep, Rahul S. Dalal, Kira L. Newman, Kristin Galetta, J. Carl Pallais, Nancy Wei, Kathleen M. Buchheit, David I. Hong, Vartan Pahalyants, Ernest Y. Lee, Allen Shih, Tamara B. Kaplan, Vishnu Ravi, Sarita Khemani, Thomas A. Buckley, April S. Liang, Daniel Shirvani, Advait Patil, Nicholas Marshall, Kanav Chopra, Joel Koh, Adi Badhwar, Anastasia Perez, Austin J. Schoeffler, Mahbuba Tusty, Chase M. Walton, Liam G. McCoy, David J. H. Wu, Yingjie Weng, Sumant Ranji, Kevin Schulman, Nigam H. Shah, Jason Hom, Arnold Milstein, Arjun K. Manrai, Adam Rodman, Jonathan H. Chen, Ethan Goh
机构
*
Harvard Combined Dermatology Program(哈佛联合皮肤科项目)
;
Department of Dermatology, Mass General Brigham(麻省总医院皮肤科)
;
Harvard Medical School(哈佛医学院)
;
Stanford Center for Biomedical Informatics Research(斯坦福生物医学信息学研究中心)
;
Stanford University(斯坦福大学)
;
Division of Hospital Medicine, Department of Medicine, Stanford University School of Medicine(斯坦福大学医学院医院医学科)
;
Department of Medicine, Cambridge Health Alliance(剑桥健康联盟医学科)
;
Beth Israel Deaconess Hospital–Plymouth(贝塞斯达德acons医院-普利茅斯)
;
Department of Medicine, University of California, San Francisco(加州大学旧金山分校医学科)
;
Department of Neurology, Stanford University School of Medicine(斯坦福大学医学院神经科)
;
Department of Medicine, Beth Israel Deaconess Medical Center(贝塞斯达德acons医学中心医学科)
;
Division of Cardiology, Department of Medicine, Cambridge Health Alliance(剑桥健康联盟心脏病科)
;
Department of Cardiovascular Medicine, Summa Health System(Summa健康系统心血管医学科)
;
Division of Allergy, Pulmonary, and Critical Care Medicine, Department of Medicine, University of Wisconsin-Madison(威斯康星大学麦迪逊分校医学科过敏、呼吸科和危重医学科)
;
Division of Pulmonary and Critical Care Medicine, Department of Medicine, Massachusetts General Hospital(麻省总医院呼吸科和危重医学科)
;
Center for Immunology and Inflammatory Diseases, Department of Medicine, Massachusetts General Hospital(麻省总医院免疫和炎症疾病中心)
;
Broad Institute of MIT and Harvard(MIT和哈佛Broad研究所)
;
Division of Pulmonary, Critical Care, and Sleep Medicine, Cambridge Health Alliance(剑桥健康联盟呼吸科、危重医学科和睡眠医学科)
Safety from Honesty in a Disinterested AI Predictor
无兴趣AI预测器中的诚实安全性
Yoshua Bengio, Oliver Richardson, Tomáš Gavenčiak, Michael Cohen, Rory Svarc, Damiano Fornasiere, Gael Gendron, David Hyland, Aton Kamanda, Adam Oberman, Francis Rhys Ward, Anna Gavenčiak, Jacob Livingston Slosser, Vincent Mai, Iulian Serban, Joumana Ghosn
机构
*
LawZero
;
Université de Montréal(蒙特利尔大学)
;
Mila
;
University of California Berkeley(加州大学伯克利分校)
;
McGill University(麦吉尔大学)
;
Arb Research
;
Center for Theoretical Study Charles University in Prague(布拉格查理大学理论研究中心)
;
University of Oxford(牛津大学)
;
Sapien Institute(Sapien研究所)
ForesightSafety-SAGE:A Fully Automated Scenario Generation and Safety Evaluation Framework for LLM Agents
VESTA: 一种全自动的LLM智能体场景生成与安全评估框架
Lu Jia, Haibo Tong, Feifei Zhao, Jindong Li, Dongqi Liang, Ping Wu, Qian Zhang, Yi Zeng
机构
*
BrainCog AI Lab, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所类脑人工智能实验室)
;
Beijing Institute of AI Safety and Governance (Beijing-AISI)(北京人工智能安全与治理研究院)
;
Beijing Key Laboratory of Safe AI and Superalignment(北京市安全人工智能与超级对齐重点实验室)
;
School of Artificial Intelligence, UCAS(中国科学院大学人工智能学院)
;
Long-term AI(长期人工智能)
机构
*
University of Southern California(南加州大学)
;
Carnegie Mellon University(卡内基梅隆大学)
;
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
;
Stanford University(斯坦福大学)
VLAFlow: A Unified Training Framework for Vision-Language-Action Models via Co-training and Future Latent Alignment
VLAFlow:通过协同训练和未来潜在对齐的视觉-语言-动作模型统一训练框架
Guoyang Xia, Fengfa Li, Hongjin Ji, Lei Ren, Fangxiang Feng, Kun Zhan, Yan Xie
机构
*
Li Auto Inc.(理想汽车)
;
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院)
;
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
机构
*
Ant Group(蚂蚁集团)
;
Nanyang Technological University(南洋理工大学)
;
CFAR and IHPC, Agency for Science, Technology and Research (A*STAR), Singapore(新加坡科技研究局(A*STAR)计算与先进机器人中心及高性能计算研究所)
MedBeads: An AI-Native Clinical Context Graph Built from Immutable Beads and Reconstructable Clinical Links
MedBeads:面向可信医疗AI的智能体原生不可变数据基底
Takahito Nakajima
机构
*
Diagnostic Imaging and Interventional Radiology, Institute of Medicine, University of Tsukuba(东京大学医学研究院诊断影像与介入放射学部)
;
Center for Cyber Medicine Research, University of Tsukuba(东京大学计算机医学研究中心)
机构
*
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所)
;
Pengcheng Laboratory(鹏城实验室)
;
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
;
Data Science and Artificial Intelligence Research Institute, China United Network Communications Group Co., Ltd.(中国联合网络通信集团有限公司数据科学与人工智能研究院)
QEDBENCH: Quantifying the Alignment Gap in Automated Evaluation of University-Level Mathematical Proofs
QEDBENCH:量化大学水平数学证明自动评估中的对齐差距
Santiago Gonzalez, Alireza Amiri Bavandpour, Peter Ye, Edward Zhang, Ruslans Aleksejevs, Todor Antić, Polina Baron, Sujeet Bhalerao, Shubhrajit Bhattacharya, Zachary Burton, John Byrne, Hyungjun Choi, Nujhat Ahmed Disha, Koppany István Encz, Yuchen Fang, Robert Joseph George, Ebrahim Ghorbani, Alan Goldfarb, Jing Guo, Meghal Gupta, Stefano Huber, Annika Kanckos, Minjung Kang, Hyun Jong Kim, Dino Lorenzini, Levi Lorenzo, Tianyi Mao, Giovanni Marzenta, Ariane M. Masuda, Lukas Mauth, Ana Mickovic, Andres Miniguano-Trujillo, Antoine Moulin, Wenqi Ni, Tomos Parry, Kevin Ren, Hossein Roodbarani, Mathieu Rundström, Manjil Saikia, Detchat Samart, Rebecca Steiner, Connor Stewart, Dhara Thakkar, Jeffrey Tse, Vasiliki Velona, Yunhai Xiang, Sibel Yalçın, Jun Yan, Ji Zeng, Arman Cohan, Quanquan C. Liu
MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models
MedLayBench-V:面向医学视觉语言模型中专家与普通人语义对齐的大规模基准
Han Jang, Junhyeok Lee, Heeseong Eum, Kyu Sung Choi
机构
*
Seoul National University(首尔国立大学)
;
Seoul National University College of Medicine(首尔国立大学医学院)
;
Department of Radiology, Seoul National University Hospital(首尔国立大学医院放射科)
;
Healthcare AI Research Institute, Seoul National University Hospital(首尔国立大学医院健康人工智能研究所)
;
The Advanced Imaging and Computational Neuroimaging (AICON) Laboratory(先进影像与计算神经影像实验室)
Intent-Handover: Grounding Language in Human-Usage Regions for Trustworthy Robot-to-Human Handovers
Intent-Handover:在人类使用区域中接地语言以实现可信的机器人到人交接
Hanxin Zhang, Abdulqader Dhafer, Hongbiao Dong, Zhou Daniel Hao
机构
*
DANiLab, University of Leicester(莱斯特大学DANiLab)
;
University of Leicester(莱斯特大学)
;
School of Computing and Mathematical Sciences, University of Leicester(莱斯特大学计算与数学科学学院)
;
School of Metallurgy and Materials, University of Birmingham(伯明翰大学冶金与材料学院)