SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models
SciCustom: 一个用于大型语言模型科学能力定制评估的框架
Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang
机构
*
State Key Laboratory for Multimedia Information Processing, School of Computer Science, PKU-Anker LLM Lab, Peking University(多媒体信息处理国家重点实验室,计算机学院,PKU-Anker LLM实验室,北京大学)
;
Zhongguancun Academy(中关村学院)
;
IDEA
;
Xidian University(西安电子科技大学)
;
Peking University(北京大学)
;
University of Washington(华盛顿大学)
;
University of Wisconsin–Madison(威斯康星大学麦迪逊分校)
;
University of Illinois Chicago(伊利诺伊大学芝加哥分校)
CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows?
CHI-Bench: 能否让AI代理自动化端到端、长周期、政策丰富的医疗工作流程?
Haolin Chen, Deon Metelski, Leon Qi, Tao Xia, Joonyul Lee, Steve Brown, Kevin Riley, Frank Wang, T. Y. Alvin Liu, Hank Capps MD, Zeyu Tang, Xiangchen Song, Lingjing Kong, Fan Feng, Tianyi Zeng, Zhiwei Liu, Zixian Ma, Hang Jiang, Fangli Geng, Yuan Yuan, Chenyu You, Qingsong Wen, Hua Wei, Yanjie Fu, Yue Zhao, Carl Yang, Biwei Huang, Kun Zhang, Caiming Xiong, Sanmi Koyejo, Eric P. Xing, Philip S. Yu, Weiran Yao
机构
*
Johns Hopkins Medicine(约翰霍普金斯医学中心)
;
Wellstar Health System(Wellstar健康系统)
;
Stanford University(斯坦福大学)
;
CMU(卡内基梅隆大学)
;
UCSD(加州大学圣地亚哥分校)
;
Yale School of Medicine(耶鲁医学院)
;
Salesforce AI Research(Salesforce人工智能研究)
;
University of Washington(华盛顿大学)
;
Northeastern University(东北大学)
;
Brown University(布朗大学)
;
Boston College(波士顿学院)
;
Stony Brook University(史泰森布里克大学)
;
University of Oxford(牛津大学)
;
Arizona State University(亚利桑那州立大学)
;
University of Southern California(南加州大学)
;
Emory University(埃默里大学)
;
MBZUAI
;
Recursive Superintelligence(递归超级智能)
;
University of Illinois at Chicago(伊利诺伊大学芝加哥分校)
Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior
重写历史:一种用于干预分析以研究数据对模型行为影响的配方
Rahul Nadkarni, Yanai Elazar, Hila Gonen, Noah A. Smith
机构
*
Paul G. Allen School of Computer Science & Engineering, University of Washington(保罗·G·阿伦计算机科学与工程学院,华盛顿大学)
;
Department of Computer Science, Bar-Ilan University(巴伊兰大学计算机科学系)
;
Department of Computer Science, University of British Columbia(不列颠哥伦比亚大学计算机科学系)
;
Allen Institute for Artificial Intelligence(人工智能阿伦研究所)
Aritra Ghosh, Drew Oldag, Michael Tauraso, Andrew J. Connolly, Peter Ferguson, Derek Jones, Gourav Khullar, Argyro Sasli, Samarth Venkatesh, Gracia Wang, Maxine West, Dylan Berry, Neven Caplar, Colin Orion Chandler, Tanawan Chatchadanoraset, Michael W. Coughlin, Melissa DeLucchi, Alexandra Junell, Diego Miura, Felipe Fontinele Nunes, Wilson Beebe, Doug Branton, Sandro Campos, Liam Cunningham, Mi Dai, Jeremy Kubica, Konstantin Malanchev, Rachel Mandelbaum, Sean McGuire, Imad Pasha, Dan S. Taranu, Tianqing Zhang
机构
*
Dept. of Astronomy \& the DiRAC Institute, University of Washington, Box 351580, Seattle, WA 98195, USA
;
School of Physics
;
Astronomy, University of Minnesota, Minneapolis, MN 55455, USA
;
Department of Astronomy
;
Planetary Science, Northern Arizona University, Flagstaff, USA
;
McWilliams Center for Cosmology
;
Astrophysics, Department of Physics, Carnegie Mellon University, Pittsburgh, PA 15213, USA