Why Does Grounding Hurt Medical VQA? Benchmarking, Diagnosis, and Fine-Tuning of Vision-Language Models
对前沿视觉-语言模型进行审计以实现可信的医学视觉问答:定位失败、格式崩溃和领域适应
Xupeng Chen, Binbin Shi, Chenqian Le, Qifu Yin, Lang Lin, Haowei Ni, Ran Gong, Panfeng Li
机构
*
New York University, New York, USA(纽约大学)
;
Tsinghua University, Beijing, China(清华大学)
;
Columbia University, New York, USA(哥伦比亚大学)
;
University of Michigan, Ann Arbor, USA(密歇根大学)
VisRAG2.0: Mitigating Visual Hallucinations via Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation
VisRAG2.0:通过视觉检索增强生成中的证据引导多图像推理减轻视觉幻觉
Yubo Sun, Chunyi Peng, Yukun Yan, Shi Yu, Zhenghao Liu, Sen Mei, Chi Chen, Maosong Sun
机构
*
School of Software and Microelectronics, Peking University, China(北京大学软件与微电子学院)
;
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院)
;
Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学人工智能研究院计算机科学与技术系)
ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding
ClinFusion:用于整体医学理解的以视觉为中心的多模态大语言模型系统
Hangjie Yuan, Yichen Qian, Zhiwei Tang, Xianzhe Xu, Lirong Wu, Sicheng Yang, Jinwang Wang, Pengju Wang, Zhitao Zeng, Yizeng Han, Yan Xing, Shengxuan Luo, Tao Feng, Qing Xie, Weigen Yao, Yi Yang, Zuozhu Liu, Jiasheng Tang, Shaocheng Wang, Jitao Wang, Jiahong Dong, Weihua Chen, Feng Xu, Fan Wang
机构
*
DAMO Academy, Alibaba Group(达摩院,阿里巴巴集团)
;
Hupan Laboratory(湖畔实验室)
;
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院)
;
Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
;
Department of Radiology, The Affiliated Yangming Hospital of Ningbo University(宁波大学附属阳明医院放射科)
;
Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University(浙江大学伊利诺伊大学厄巴纳香槟校区联合学院,浙江大学)
;
Hepato-Pancreato-Biliary Center, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University(清华长庚医院肝胆胰中心,清华大学临床医学院,清华医学,清华大学)
;
School of Software, Tsinghua University(清华大学软件学院)
;
Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心)
专题命中
视觉问答
:visual question answering(abstract);multimodal large language model(abstract);MLLM(abstract);分类 cs.CV、cs.AI
Towards Reliable Stain Transfer: An Iterative Data-Model Co-Optimization Framework Based on Multimodal Expert-Guided Assessment
迈向可靠的染色转移:基于多模态专家指导评估的迭代数据-模型协同优化框架
Siyuan Xu, Yan Wang, Haofei Song, Lili Gao, Jiansheng Wang, Qing Zhang, Dan Huang, Boxiang Yun, Hongkai Xiong, Qingli Li
机构
*
East China Normal University(华东师范大学)
;
Ruijin Hospital, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属瑞金医院)
;
Hangzhou Hyperspectral Imaging Technology Co., Ltd.(杭州高光谱成像技术有限公司)
;
Fudan University Shanghai Cancer Center(复旦大学附属肿瘤医院)
NeuroSymActive: Differentiable Neural-Symbolic Reasoning with Active Exploration for Knowledge Graph Question Answering
NeuroSymActive:基于主动探索的可微神经符号推理用于知识图谱问答
Rong Fu, Yang Li, Zeyu Zhang, Jiekai Wu, Yaohua Liu, Shuaishuai Cao, Yangchen Zeng, Yuhang Zhang, Xiaojing Du, Simon Fong
机构
*
University of Macau(澳门大学)
;
University of Chinese Academy of Sciences(中国科学院大学)
;
The Australian National University(澳大利亚国立大学)
;
Juntendo University(静冈大学)
;
Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院)
;
Central South University(中南大学)
;
Southeast University(东南大学)
;
China Agricultural University(中国农业大学)
;
Adelaide University(阿德莱德大学)
机构
*
College of Geodesy and Geomatics, Shandong University of Science and Technology(山东科技大学测绘与地理信息学院)
;
School of Environmental Science and Spatial Informatics, China University of Mining and Technology(中国矿业大学环境与测绘学院)
;
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University(武汉大学测绘遥感信息工程国家重点实验室)
;
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
;
School of Automation, Southeast University(东南大学自动化学院)
;
Department of Geography, National University of Singapore(新加坡国立大学地理系)
专题命中
视觉推理
:MLLM(abstract,abstract_cn);multimodal large language model(abstract);分类 cs.CV、cs.AI
Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding
利用记忆进行推理:一种用于免训练长视频理解的时间粒度自适应框架
Linghao Meng, Qiankun Li, Junyuan Mao, Pujin Liao, Zhicheng He, Enbo Zhang, Kun Wang, Yang Liu, Huazhu Fu, Yueming Jin
机构
*
National University of Singapore(新加坡国立大学)
;
Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)(高性能计算研究所,科学、技术与研究机构(A*STAR))
;
Nanyang Technological University(南洋理工大学)
;
University of Science and Technology of China(中国科学技术大学)
;
Jilin University(吉林大学)
专题命中
视觉推理
:LLaVA(abstract,abstract_cn);multimodal large language model(abstract);分类 cs.CV、cs.AI
GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understanding
GeoAnchor:通过潜在分解进行协作推理以实现3D空间理解
Hao Li, Han Fang, Zixin Pan, Xin Wei, Hongbo Sun, Jinglin Xu, Zhiyu Lin, Ye Yuan, Zhongjiang He, Yu Yu, Hao Sun
机构
*
Shanghai Jiao Tong University(上海交通大学)
;
Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd(星辰通用人工智能实验室,中国电信人工智能技术(北京)有限公司)
;
University of Science and Technology Beijing(北京科技大学)
;
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
专题命中
视觉推理
:grounding(abstract);multimodal large language model(abstract);分类 cs.CV、cs.AI
Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases
评估具有挑战性的真实世界临床病例中的多轮多模态诊断推理
Rui Yang, Weihao Xuan, Yi Lin, Zhuhan Bao, Jonathan Chong Kai Liew, Matthew Yu Heng Wong, Nicolás Lescano, Nikita R. Paripati, Emily Ling-Lin Pai, Jiarui Liu, Heli Qi, Heng-Jui Chang, Benny Kai Guo Loo, Huitao Li, Kunyu Yu, Yufan Wang, Chuan Hong, Shijian Lu, Douglas Teodoro, Naoto Yokoya, Ross Koppel, Mona Diab, Hua Xu, David W. Bates, Nan Liu, Yifan Peng
机构
*
Center for Biomedical Data Science, Duke-NUS Medical School(生物医学数据科学中心,杜克 - 新加坡国立大学医学院)
;
Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School(杜克 - 新加坡国立大学人工智能与医学科学计划,杜克 - 新加坡国立大学医学院)
;
Department of Population Health Sciences, Weill Cornell Medicine(人口健康科学系,威尔康奈尔医学院)
;
System Engineering, College of Engineering, Cornell University(系统工程,康奈尔大学工程学院)
;
Graduate School of Frontier Sciences, The University of Tokyo(前沿科学研究生院,东京大学)
;
RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心)
;
Department of Biostatistics and Bioinformatics, Duke University(生物统计学与生物信息学系,杜克大学)
;
Perelman School of Medicine, University of Pennsylvania(佩雷尔曼医学院,宾夕法尼亚大学)
;
School of Clinical Medicine, University of Cambridge(临床医学学院,剑桥大学)
;
Hospital of the University of Pennsylvania(宾夕法尼亚大学医院)
;
Children’s Hospital of Philadelphia (CHOP)(费城儿童医院)
;
Department of Anatomic Pathology and Laboratory Medicine, Hospital of the University of Pennsylvania(解剖病理学与检验医学系,宾夕法尼亚大学医院)
;
Department of Pathology and Laboratory Medicine, University of California, San Francisco(病理学与检验医学系,加州大学旧金山分校)
;
Language Technologies Institute, Carnegie Mellon University(语言技术研究所,卡内基梅隆大学)
专题命中
视觉推理
:multimodal large language model(abstract);分类 cs.AI
Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScore
超越事实准确性:使用逻辑得分评估RAG系统中的全球推理完整性
Zhichao Yan, Yunxiao Zhao, Jiapu Wang, Jiaoyan Chen, Xiaoli Li, Ru Li, Jeff Z. Pan
机构
*
Shanxi University(山西大学)
;
Nanjing University of Science and Technology(南京理工大学)
;
University of Manchester(曼彻斯特大学)
;
Singapore University of Technology and Design(新加坡科技设计大学)
;
University of Edinburgh(爱丁堡大学)
MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
MEDIC-AD:迈向医疗视觉-语言模型的临床智能
Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho, Pa Hong, Wookyoung Jeong, Yoojin Nam, Namjoon Kim, Ginny Y. Wong, Ka Chun Cheung, Jaeyoung Do
机构
*
AIDAS Laboratory, Seoul National University(首尔大学AIDAS实验室)
;
Samsung Changwon Hospital(三星昌原医院)
;
Samsung Medical Center(三星医疗中心)
;
NVIDIA, Santa Clara, USA(英伟达(美国圣克拉拉))
RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation
RRS-10K:用于罕见遥感图像解释的多任务视觉语言模型基准测试
Yuqiao Lai, Jiancheng Qi, Fei Wang, Yuxin Liu, Kun Li, Ye Chen, Yan Gao, Yanyan Wei
机构
*
Laboratory of Intelligent Language Processing, National University of Defense Technology(国防科技大学智能语言处理实验室)
;
Hefei University of Technology(合肥工业大学)
;
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)
;
United Arab Emirates University(阿联酋大学)