Context-driven Missing-Modality Learning for Robust Medical Diagnosis with Image-Tabular Data
基于上下文驱动的缺失模态学习用于图像-表格数据的鲁棒医学诊断
机构 * College of Intelligence and Computing, Tianjin University, Tianjin 300350, China.(智能与计算学院,天津大学,天津300350,中国) ; Department of Statistics and Actuarial Science, School of Computing and Data Science, The University of Hong Kong, Hong Kong.(统计与精算系,计算与数据科学学院,香港大学,香港) ; Department of Radiology, Tianjin Huanhu Hospital, Tianjin 300350, China.(放射科,天津华和医院,天津300350,中国) ; Tianjin Key Laboratory of Cerebral Vascular and Neurodegenerative Diseases, Tianjin 300350, China.(天津脑血管与神经退行性疾病重点实验室,天津300350,中国) ; Medical School of Tianjin University, Tianjin 300072, China.(天津大学医学院,天津300072,中国)
AI总结 提出CMML框架,通过级联残差变换器自编码器合成缺失模态并利用上下文令牌进行语义对齐,在三种医学数据集上超越现有方法。
Comments 12 pages, 8 figures