2605.01417
2026-05-05
cs.CL
cs.AI
86%
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks
Medmarks:面向医疗任务的综合性开源大语言模型评估套件
Benjamin Warner, Ratna Sagari Grandhi, Max Kieffer, Aymane Ouraq, Saurav Panigrahi, Geetu Ambwani, Kunal Bagga, Nikhil Khandekar, Arya Hariharan, Nishant Mishra, Manish Ram, Shamus Sim Zi Yang, Ahmed Essouaied, Adepoju Jeremiah Moyondafoluwa, Robert Scholz, Bofeng Huang, Molly Beavers, Srishti Gureja, Anish Mahishi, Sameed Khan, Maxime Griot, Hunar Batra, Jean-Benoit Delbrouck, Siddhant Bharadwaj, Ronald Clark, Ashish Vashist, Anas Zafar, Leema Krishna Murali, Harsh Deshpande, Ameen Patel, William Brown, Johannes Hagemann, Connor Lane, Paul Steven Scotti, Tanishq Mathew Abraham
专题命中
评测与基准
:LLM(title,abstract);large language model(abstract);language model(abstract);分类 cs.CL、cs.AI
AI总结
本文提出Medmarks,一个包含30个医疗任务基准的开源评估套件,系统评估61个模型,揭示前沿模型在医疗推理中的优势及模型对答案顺序偏见的敏感性。