First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations
首先,不伤害:迈向临床安全的大语言模型
机构 * Harvard Combined Dermatology Program(哈佛联合皮肤科项目) ; Department of Dermatology, Mass General Brigham(麻省总医院皮肤科) ; Harvard Medical School(哈佛医学院) ; Stanford Center for Biomedical Informatics Research(斯坦福生物医学信息学研究中心) ; Stanford University(斯坦福大学) ; Division of Hospital Medicine, Department of Medicine, Stanford University School of Medicine(斯坦福大学医学院医院医学科) ; Department of Medicine, Cambridge Health Alliance(剑桥健康联盟医学科) ; Beth Israel Deaconess Hospital–Plymouth(贝塞斯达德acons医院-普利茅斯) ; Department of Medicine, University of California, San Francisco(加州大学旧金山分校医学科) ; Department of Neurology, Stanford University School of Medicine(斯坦福大学医学院神经科) ; Department of Medicine, Beth Israel Deaconess Medical Center(贝塞斯达德acons医学中心医学科) ; Division of Cardiology, Department of Medicine, Cambridge Health Alliance(剑桥健康联盟心脏病科) ; Department of Cardiovascular Medicine, Summa Health System(Summa健康系统心血管医学科) ; Division of Allergy, Pulmonary, and Critical Care Medicine, Department of Medicine, University of Wisconsin-Madison(威斯康星大学麦迪逊分校医学科过敏、呼吸科和危重医学科) ; Division of Pulmonary and Critical Care Medicine, Department of Medicine, Massachusetts General Hospital(麻省总医院呼吸科和危重医学科) ; Center for Immunology and Inflammatory Diseases, Department of Medicine, Massachusetts General Hospital(麻省总医院免疫和炎症疾病中心) ; Broad Institute of MIT and Harvard(MIT和哈佛Broad研究所) ; Division of Pulmonary, Critical Care, and Sleep Medicine, Cambridge Health Alliance(剑桥健康联盟呼吸科、危重医学科和睡眠医学科)
AI总结 提出NOHARM基准,包含1100个初级到专科咨询案例,评估28个LLM的医疗建议安全性,发现高达22.6%的案例存在严重危害风险,其中遗漏错误占80%以上。