医学 AI

Multimodal Lego: Model Merging and Fine-Tuning Across Topologies and Modalities in Biomedicine

Konstantin Hemker, Nikola Simidjievski, Mateja Jamnik

专题命中 医疗多模态 ：biomedical(abstract)；分类 cs.LG

Integrating Vision and Location with Transformers: A Multimodal Deep Learning Framework for Medical Wound Analysis

Ramin Mousa, Hadis Taherinia, Khabiba Abdiyeva, Amir Ali Bengari, Mohammadmahdi Vahediahmar

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Derm1M: A Million-scale Vision-Language Dataset Aligned with Clinical Ontology Knowledge for Dermatology

Siyuan Yan, Ming Hu, Yiwen Jiang, Xieji Li, Hao Fei, Philipp Tschandl, Harald Kittler, Zongyuan Ge

专题命中 医疗多模态 ：medical AI(abstract)；分类 cs.CV

Comments Our dataset and code will be publicly available at https://github.com/SiyuanYan1/Derm1M

A Multimodal Vision Foundation Model for Clinical Dermatology

Siyuan Yan, Zhen Yu, Clare Primiero, Cristina Vico-Alonso, Zhonghua Wang, Litao Yang, Philipp Tschandl, Ming Hu, Lie Ju, Gin Tan, Vincent Tang, Aik Beng Ng, David Powell, Paul Bonnington, Simon See, Elisabetta Magnaterra, Peter Ferguson, Jennifer Nguyen, Pascale Guitera, Jose Banuls, Monika Janda, Victoria Mar, Harald Kittler, H. Peter Soyer, Zongyuan Ge

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Comments 74 pages; Preprint; The code can be found at https://github.com/SiyuanYan1/PanDerm

DiN: Diffusion Model for Robust Medical VQA with Semantic Noisy Labels

Erjian Guo, Zhen Zhao, Zicheng Wang, Tong Chen, Yunyi Liu, Luping Zhou

专题命中 医疗多模态 ：medical image(abstract)；分类 cs.CV

Robust Multimodal Survival Prediction with the Latent Differentiation Conditional Variational AutoEncoder

Junjie Zhou, Jiao Tang, Yingli Zuo, Peng Wan, Daoqiang Zhang, Wei Shao

专题命中 医疗多模态 ：pathology(abstract)；分类 cs.CV

Comments Accepted by CVPR2025

STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classification

Siyi Du, Xinzhe Luo, Declan P. O'Regan, Chen Qin

专题命中 医疗多模态 ：medical image(abstract)；分类 cs.CV

Comments 16 pages (including 5 pages of supplementary materials), accepted by CVPR 2025

EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery

Guankun Wang, Long Bai, Junyi Wang, Kun Yuan, Zhen Li, Tianxu Jiang, Xiting He, Jinlin Wu, Zhen Chen, Zhen Lei, Hongbin Liu, Jiazheng Wang, Fan Zhang, Nicolas Padoy, Nassir Navab, Hongliang Ren

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Anatomy-Aware Conditional Image-Text Retrieval

Meng Zheng, Jiajin Zhang, Benjamin Planche, Zhongpai Gao, Terrence Chen, Ziyan Wu

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Comments 16 pages, 10 figures

R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interest

Xupeng Chen, Zhixin Lai, Kangrui Ruan, Shichu Chen, Jiaxiang Liu, Zuozhu Liu

专题命中 医疗多模态 ：biomedical(abstract)；分类 cs.CV

RCRank: Multimodal Ranking of Root Causes of Slow Queries in Cloud Database Systems

Biao Ouyang, Yingying Zhang, Hanyin Cheng, Yang Shu, Chenjuan Guo, Bin Yang, Qingsong Wen, Lunting Fan, Christian S. Jensen

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.LG

Comments Accepted by VLDB 2025

VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge

Vishwesh Nath, Wenqi Li, Dong Yang, Andriy Myronenko, Mingxin Zheng, Yao Lu, Zhijian Liu, Hongxu Yin, Yucheng Tang, Pengfei Guo, Can Zhao, Ziyue Xu, Yufan He, Greg Heinrich, Yee Man Law, Benjamin Simon, Stephanie Harmon, Stephen Aylward, Marc Edgar, Michael Zephyr, Song Han, Pavlo Molchanov, Baris Turkbey, Holger Roth, Daguang Xu

专题命中 医疗多模态 ：radiology(abstract)；分类 cs.CV

PaliGemma-CXR: A Multi-task Multimodal Model for TB Chest X-ray Interpretation

Denis Musinguzi, Andrew Katumba, Sudi Murindanyi

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation

Yue Jiang, Jiawei Chen, Dingkang Yang, Mingcheng Li, Shunli Wang, Tong Wu, Ke Li, Lihua Zhang

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Explainable, Multi-modal Wound Infection Classification from Images Augmented with Generated Captions

Palawat Busaranuvong, Emmanuel Agu, Reza Saadati Fard, Deepak Kumar, Shefalika Gautam, Bengisu Tulu, Diane Strong

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Prompt as Knowledge Bank: Boost Vision-language model via Structural Representation for zero-shot medical detection

Yuguang Yang, Tongfei Chen, Haoyu Huang, Linlin Yang, Chunyu Xie, Dawei Leng, Xianbin Cao, Baochang Zhang

专题命中 医疗多模态 ：medical image(abstract)；分类 cs.CV

Comments Accepted as ICLR 2025 conference paper

Artificial intelligence-enabled detection and assessment of Parkinson's disease using multimodal data: A survey

Aite Zhao, Yongcan Liu, Xinglin Yu, Xinyue Xing

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.LG

Enhancing medical vision-language contrastive learning via inter-matching relation modelling

Mingjian Li, Mingyuan Meng, Michael Fulham, David Dagan Feng, Lei Bi, Jinman Kim

专题命中 医疗多模态 ：medical image(abstract)；分类 cs.CV

Comments Published at IEEE Transactions on Medical Imaging

LMOD: A Large Multimodal Ophthalmology Dataset and Benchmark for Large Vision-Language Models

Zhenyue Qin, Yu Yin, Dylan Campbell, Xuansheng Wu, Ke Zou, Yih-Chung Tham, Ninghao Liu, Xiuzhen Zhang, Qingyu Chen

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Comments 2025 NAACL: Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics Project Page: https://kfzyqin.github.io/lmod/

MoColl: Agent-Based Specific and General Model Collaboration for Image Captioning

Pu Yang, Bin Dong

专题命中 医疗多模态 ：radiology(abstract)；分类 cs.CV

DAMPER: A Dual-Stage Medical Report Generation Framework with Coarse-Grained MeSH Alignment and Fine-Grained Hypergraph Matching

Xiaofei Huang, Wenting Chen, Jie Liu, Qisheng Lu, Xiaoling Luo, Linlin Shen

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Journal ref AAAI 2025

Unlocking the Potential of Weakly Labeled Data: A Co-Evolutionary Learning Framework for Abnormality Detection and Report Generation

Jinghan Sun, Dong Wei, Zhe Xu, Donghuan Lu, Hong Liu, Hong Wang, Sotirios A. Tsaftaris, Steven McDonagh, Yefeng Zheng, Liansheng Wang

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.CV

Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions

Jiarui Zhang, Ollie Liu, Tianyu Yu, Jinyi Hu, Willie Neiswanger

专题命中 医疗多模态 ：medical image(abstract)；分类 cs.CV

Comments 33 pages, 22 figures, 5 tables, 7 algorithms

A Self-guided Multimodal Approach to Enhancing Graph Representation Learning for Alzheimer's Diseases

Zhepeng Wang, Runxue Bao, Yawen Wu, Guodong Liu, Lei Yang, Liang Zhan, Feng Zheng, Weiwen Jiang, Yanfu Zhang

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.LG

MaintAGT:Sim2Real-Guided Multimodal Large Model for Intelligent Maintenance with Chain-of-Thought Reasoning

Hongliang He, Jinfeng Huang, Qi Li, Xu Wang, Feibin Zhang, Kangding Yang, Li Meng, Fulei Chu

专题命中 医疗多模态 ：diagnosis(abstract)；分类 eess.SP

Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?

Daniel P. Jeong, Saurabh Garg, Zachary C. Lipton, Michael Oberst

专题命中 医疗多模态 ：biomedical(abstract)；分类 cs.LG

Comments This version was published at EMNLP 2024 Main Conference as a Long Paper (Oral). See the extended version (arXiv:2411.08870) for additional results on QA tasks based on clinical notes and evaluations in the supervised fine-tuning regime

Multi-Task Adversarial Variational Autoencoder for Estimating Biological Brain Age with Multimodal Neuroimaging

Muhammad Usman, Azka Rehman, Abdullah Shahid, Abd Ur Rehman, Sung-Min Gho, Aleum Lee, Tariq M. Khan, Imran Razzak

专题命中 医疗多模态 ：MRI(abstract)；分类 cs.CV

Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer's Disease

Francesco Chiumento, Mingming Liu

专题命中 医疗多模态 ：radiology(abstract)；分类 eess.IV

Comments The paper has been accepted by the conference: "2024 International Conference on Big Data (IEEE Big Data 2024)"

Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance

Zhangwei Gao, Zhe Chen, Erfei Cui, Yiming Ren, Weiyun Wang, Jinguo Zhu, Hao Tian, Shenglong Ye, Junjun He, Xizhou Zhu, Lewei Lu, Tong Lu, Yu Qiao, Jifeng Dai, Wenhai Wang

专题命中 医疗多模态 ：medical image(abstract)；分类 cs.CV

Comments Technical report

Cognitive Insights Across Languages: Enhancing Multimodal Interview Analysis

David Ortiz-Perez, Jose Garcia-Rodriguez, David Tomás

专题命中 医疗多模态 ：diagnosis(abstract)；分类 cs.LG

Comments GitHub repository: https://github.com/davidorp/taukadial

Journal ref Interspeech 2024, pages 952-956