ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding
ClinFusion:用于整体医学理解的以视觉为中心的多模态大语言模型系统
机构 * DAMO Academy, Alibaba Group(达摩院,阿里巴巴集团) ; Hupan Laboratory(湖畔实验室) ; College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院) ; Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系) ; Department of Radiology, The Affiliated Yangming Hospital of Ningbo University(宁波大学附属阳明医院放射科) ; Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University(浙江大学伊利诺伊大学厄巴纳香槟校区联合学院,浙江大学) ; Hepato-Pancreato-Biliary Center, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University(清华长庚医院肝胆胰中心,清华大学临床医学院,清华医学,清华大学) ; School of Software, Tsinghua University(清华大学软件学院) ; Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心)
AI总结 研究针对多模态大语言模型在医学领域部署的挑战,提出ClinFusion,采用组合级联视觉编码器架构和视觉基础评估框架,在多模态医学基准测试中表现优异,超越开源和专有模型,经专家盲评验证效果良好。
Comments Code: https://github.com/alibaba-damo-academy/ClinFusion Models: https://huggingface.co/collections/Alibaba-DAMO-Academy/clinfusion