医学 AI - arXivDaily 专题

2405.17459 2024-05-29 cs.LG cs.AI cs.CL cs.CV 62%

Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysis

Ziyan Yao, Fei Lin, Sheng Chai, Weijie He, Lu Dai, Xinghui Fei

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、cs.LG

2403.19880 2024-04-01 eess.IV cs.CV 62%

Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks

Pooria Ashrafian, Milad Yazdani, Moein Heidari, Dena Shahriari, Ilker Hacihaliloglu

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、eess.IV

Comments Submitted as a conference paper to MICCAI 2024

2305.11490 2024-03-19 cs.CV cs.AI cs.CL cs.LG 62%

LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation

Suhyeon Lee, Won Jun Kim, Jinho Chang, Jong Chul Ye

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、cs.LG

Comments 21 pages, 8 figures; ICLR 2024 (poster)

2402.14815 2024-02-23 cs.CY cs.AI cs.CV cs.LG 62%

Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging

Yuzhe Yang, Yujia Liu, Xin Liu, Avanti Gulhane, Domenico Mastrodicasa, Wei Wu, Edward J Wang, Dushyant W Sahani, Shwetak Patel

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG

Comments Code and data are available at https://github.com/YyzHarry/vlm-fairness

2402.02460 2024-02-13 cs.LG cs.AI cs.CV 62%

Review of multimodal machine learning approaches in healthcare

Felix Krones, Umar Marikkar, Guy Parsons, Adam Szmul, Adam Mahdi

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG

Comments 5 figures, 5 tables

2402.05294 2024-02-09 cs.LG cs.AI cs.CL cs.CV 62%

Examining Modality Incongruity in Multimodal Federated Learning for Medical Vision and Language-based Disease Detection

Pramit Saha, Divyanshu Mishra, Felix Wagner, Konstantinos Kamnitsas, J. Alison Noble

专题命中医疗多模态：radiology(abstract)；分类 cs.CV、cs.LG

Comments 42 pages

2402.00375 2024-02-02 eess.IV cs.CV 62%

Disentangled Multimodal Brain MR Image Translation via Transformer-based Modality Infuser

Jihoon Cho, Xiaofeng Liu, Fangxu Xing, Jinsong Ouyang, Georges El Fakhri, Jinah Park, Jonghye Woo

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、eess.IV

Comments 6 pages

2401.01179 2024-01-03 cs.CV cs.AI cs.LG 62%

Freeze the backbones: A Parameter-Efficient Contrastive Approach to Robust Medical Vision-Language Pre-training

Jiuming Qin, Che Liu, Sibo Cheng, Yike Guo, Rossella Arcucci

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、cs.LG

Comments Accepted by ICASSP 2024

2302.12247 2023-12-12 cs.LG cs.AI cs.CL cs.CV cs.IT math.IT 62%

Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework

Paul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling, Suzanne Nie, Richard Chen, Zihao Deng, Nicholas Allen, Randy Auerbach, Faisal Mahmood, Ruslan Salakhutdinov, Louis-Philippe Morency

专题命中医疗多模态：pathology(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2023. Code available at: https://github.com/pliang279/PID

2311.07033 2023-11-14 eess.IV cs.CV 62%

TTMFN: Two-stream Transformer-based Multimodal Fusion Network for Survival Prediction

Ruiquan Ge, Xiangyang Hu, Rungen Huang, Gangyong Jia, Yaqi Wang, Renshu Gu, Changmiao Wang, Elazab Ahmed, Linyan Wang, Juan Ye, Ye Li

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、eess.IV

2304.06286 2023-11-07 eess.SP cs.CV 62%

Automated Cardiovascular Record Retrieval by Multimodal Learning between Electrocardiogram and Clinical Report

Jielin Qiu, Jiacheng Zhu, Shiqi Liu, William Han, Jingqi Zhang, Chaojing Duan, Michael Rosenberg, Emerson Liu, Douglas Weber, Ding Zhao

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、eess.SP

Comments Accepted to the ML4H 2023 Proceedings track

2304.00379 2023-10-02 eess.IV cs.CV 62%

Improved Multimodal Fusion for Small Datasets with Auxiliary Supervision

Gregory Holste, Douwe van der Wal, Hans Pinckaers, Rikiya Yamashita, Akinori Mitani, Andre Esteva

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、eess.IV

Comments IEEE ISBI 2023 (see http://2023.biomedicalimaging.org/en/)

2307.08347 2023-07-20 cs.CV cs.AI cs.LG 62%

M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization

Che Liu, Sibo Cheng, Chen Chen, Mengyun Qiao, Weitong Zhang, Anand Shah, Wenjia Bai, Rossella Arcucci

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、cs.LG

Comments Accepted by MICCAI 2023

2307.07246 2023-07-18 cs.CV cs.LG 62%

Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Training

Xiaofei Chen, Yuting He, Cheng Xue, Rongjun Ge, Shuo Li, Guanyu Yang

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG

Comments accepted by MICCAI 2023

2206.14579 2023-04-12 cs.CL cs.CV cs.LG 62%

Competence-based Multimodal Curriculum Learning for Medical Report Generation

Fenglin Liu, Shen Ge, Yuexian Zou, Xian Wu

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、cs.LG

Comments Accepted by ACL 2021 (Oral)

1908.03009 2022-12-16 eess.SP cs.LG cs.NE 62%

A Multimodal Deep Network for the Reconstruction of T2W MR Images

Antonio Falvo, Danilo Comminiello, Simone Scardapane, Michele Scarpiniti, Aurelio Uncini

专题命中医疗多模态：diagnosis(abstract)；分类 cs.LG、eess.SP

Comments 29th Italian Neural Networks Workshop (WIRN 2019)

Journal ref Progresses in Artificial Intelligence and Neural Systems. Smart Innovation, Systems and Technologies, vol 184. Springer, Singapore, Jul. 2020

2211.12737 2022-11-24 cs.CV cs.AI cs.CL cs.LG 62%

RoentGen: Vision-Language Foundation Model for Chest X-ray Generation

Pierre Chambon, Christian Bluethgen, Jean-Benoit Delbrouck, Rogier Van der Sluijs, Małgorzata Połacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P. Langlotz, Akshay Chaudhari

专题命中医疗多模态：radiology(abstract)；分类 cs.CV、cs.LG

Comments 19 pages

2206.14355 2022-06-30 cs.CV cs.CL cs.LG 62%

EBMs vs. CL: Exploring Self-Supervised Visual Pretraining for Visual Question Answering

Violetta Shevchenko, Ehsan Abbasnejad, Anthony Dick, Anton van den Hengel, Damien Teney

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG

2111.13424 2021-11-29 cs.CV cs.AI cs.LG 62%

ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics

Aiham Taleb, Matthias Kirchler, Remo Monti, Christoph Lippert

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、cs.LG

2111.04318 2021-11-16 cs.LG cs.AI cs.CL cs.CV 62%

Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation

Fenglin Liu, Chenyu You, Xian Wu, Shen Ge, Sheng Wang, Xu Sun

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、cs.LG

2005.14117 2021-10-26 eess.IV cs.LG stat.ML 62%

Multimodal Feature Fusion and Knowledge-Driven Learning via Experts Consult for Thyroid Nodule Classification

Danilo Avola, Luigi Cinque, Alessio Fagioli, Sebastiano Filetti, Giorgio Grani, Emanuele Rodolà

专题命中医疗多模态：diagnosis(abstract)；分类 cs.LG、eess.IV

2105.13864 2021-05-31 cs.LG eess.SP 62%

SalientSleepNet: Multimodal Salient Wave Detection Network for Sleep Staging

Ziyu Jia, Youfang Lin, Jing Wang, Xuehui Wang, Peiyi Xie, Yingbin Zhang

专题命中医疗多模态：diagnosis(abstract)；分类 cs.LG、eess.SP

2011.00569 2020-11-03 cs.CV cs.AI cs.CL cs.LG cs.MM 62%

DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation

Jia-Hong Huang, Chao-Han Huck Yang, Fangyu Liu, Meng Tian, Yi-Chieh Liu, Ting-Wei Wu, I-Hung Lin, Kang Wang, Hiromasa Morikawa, Hernghua Chang, Jesper Tegner, Marcel Worring

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG

Comments Accepted to IEEE WACV 2021

2007.02790 2020-09-22 eess.IV cs.CV 62%

Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration

Zhe Xu, Jie Luo, Jiangpeng Yan, Ritvik Pulya, Xiu Li, William Wells, Jayender Jagadeesan

专题命中医疗多模态：CT(abstract)；分类 cs.CV、eess.IV

Comments accepted by MICCAI 2020

1905.12008 2019-05-30 cs.CV cs.AI cs.CL cs.LG 62%

Leveraging Medical Visual Question Answering with Supporting Facts

Tomasz Kornuta, Deepta Rajan, Chaitanya Shivade, Alexis Asseman, Ahmet S. Ozcan

专题命中医疗多模态：radiology(abstract)；分类 cs.CV、cs.LG

Comments Working notes from the ImageCLEF 2019 VQA-Med competition

1811.07407 2018-11-20 cs.CV cs.AI cs.LG 62%

Multimodal Densenet

Faisal Mahmood, Ziyun Yang, Thomas Ashley, Nicholas J. Durr

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、cs.LG

Comments 10 pages, 6 figures

1807.03232 2018-07-10 eess.SP cs.CV physics.med-ph 62%

Robust Heartbeat Detection from Multimodal Data via CNN-based Generalizable Information Fusion

B S Chandra, C S Sastry, S Jana

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV、eess.SP

1711.01666 2018-06-05 cs.CV cs.LG 62%

Label-driven weakly-supervised learning for multimodal deformable image registration

Yipeng Hu, Marc Modat, Eli Gibson, Nooshin Ghavami, Ester Bonmati, Caroline M. Moore, Mark Emberton, J. Alison Noble, Dean C. Barratt, Tom Vercauteren

专题命中医疗多模态：medical image(abstract)；分类 cs.CV、cs.LG

Comments Accepted to ISBI 2018

1702.06151 2017-02-22 cs.CV cs.IR cs.LG cs.MM 62%

Developing a comprehensive framework for multimodal feature extraction

Quinten McNamara, Alejandro de la Vega, Tal Yarkoni

专题命中医疗多模态：MRI(abstract)；分类 cs.CV、cs.LG

2607.19624 2026-07-23 cs.CV 新提交 61%

Pathologist Attention-Aligned Report Generation for Prostate Histopathology

用于前列腺组织病理学的病理学家注意力对齐报告生成

Ruoyu Xue, Suryakant Singh, Souradeep Chakraborty, Pierre Marza, Oksana Yaskiv, Constantin Friedman, Natallia Sheuka, Paul Friedman, Bharat Ramlal, Beatrice Knudsen, Rajarsi Gupta, Joel Saltz, Prateek Prasanna, Gregory Zelinsky, Dimitris Samaras

机构 * Department of Computer Science, Stony Brook University（纽约州立大学石溪分校计算机科学系）； Department of Biomedical Informatics, Stony Brook University（纽约州立大学石溪分校生物医学信息学系）； Université Paris-Saclay, CentraleSupélec, Gustave Roussy, INSERM, IHU PRISM, Cancer Data Science Unit（巴黎萨克雷大学、中央理工高等电力学院、古斯塔夫·鲁西研究所、法国国家健康与医学研究院、PRISM综合大学医院、癌症数据科学单元）； Université Paris-Saclay, CentraleSupélec, MICS Laboratory（巴黎萨克雷大学、中央理工高等电力学院、MICS实验室）； Department of Pathology and Laboratory Medicine, Northwell Health Laboratories（诺斯韦尔健康实验室病理与检验医学部）； Department of Pathology, University of Utah School of Medicine（犹他大学医学院病理系）； Department of Psychology, Stony Brook University（纽约州立大学石溪分校心理学系）

专题命中医疗多模态：diagnosis(abstract)；分类 cs.CV；medical image(comments)

AI总结研究将人类注意力引入前列腺病理报告生成模型训练，收集多模态数据集，通过注意力对齐损失微调模型，在两个模型上评估，在报告生成和视觉问答任务中取得较好效果，模型注意力图与病理学家注意力更对齐。

Comments 11 pages, 4 figures, accepted for publication at the 29th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2026)

1. 医疗多模态 966 篇

Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysis

Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks

LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation

Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging

Review of multimodal machine learning approaches in healthcare

Examining Modality Incongruity in Multimodal Federated Learning for Medical Vision and Language-based Disease Detection

Disentangled Multimodal Brain MR Image Translation via Transformer-based Modality Infuser

Freeze the backbones: A Parameter-Efficient Contrastive Approach to Robust Medical Vision-Language Pre-training

Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework

TTMFN: Two-stream Transformer-based Multimodal Fusion Network for Survival Prediction

Automated Cardiovascular Record Retrieval by Multimodal Learning between Electrocardiogram and Clinical Report

Improved Multimodal Fusion for Small Datasets with Auxiliary Supervision

M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization

Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Training

Competence-based Multimodal Curriculum Learning for Medical Report Generation

A Multimodal Deep Network for the Reconstruction of T2W MR Images

RoentGen: Vision-Language Foundation Model for Chest X-ray Generation

EBMs vs. CL: Exploring Self-Supervised Visual Pretraining for Visual Question Answering

ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics

Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation

Multimodal Feature Fusion and Knowledge-Driven Learning via Experts Consult for Thyroid Nodule Classification

SalientSleepNet: Multimodal Salient Wave Detection Network for Sleep Staging

DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation

Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration

Leveraging Medical Visual Question Answering with Supporting Facts

Multimodal Densenet

Robust Heartbeat Detection from Multimodal Data via CNN-based Generalizable Information Fusion

Label-driven weakly-supervised learning for multimodal deformable image registration

Developing a comprehensive framework for multimodal feature extraction

Pathologist Attention-Aligned Report Generation for Prostate Histopathology