多模态大模型 - arXivDaily 专题

2505.00422 2025-05-02 cs.LG cs.CL 83%

Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training

Yu Han, Aaron Ceross, Jeroen H. M. Bergmann

机构 * Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford（生物医学工程研究所，工程科学系，牛津大学）； Birmingham Law School, University of Birmingham（伯明翰法学院，伯明翰大学）

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

2504.07957 2025-04-29 cs.CV 83%

MM-IFEngine: Towards Multimodal Instruction Following

Shengyuan Ding, Shenxi Wu, Xiangyu Zhao, Yuhang Zang, Haodong Duan, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Dahua Lin, Jiaqi Wang

机构 * Fudan University（复旦大学）； Shanghai AI Laboratory（上海人工智能实验室）； Shanghai Jiaotong University（上海交通大学）； The Chinese University of Hong Kong（香港中文大学）； CPII under InnoHK（创新香港下的CPII）； Shanghai Innovation Institute（上海创新研究院）

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2408.13632 2025-04-28 cs.CV 83%

FungiTastic: A multi-modal dataset and benchmark for image categorization

Lukas Picek, Klara Janouskova, Vojtech Cermak, Jiri Matas

机构 * University of West Bohemia & Inria, CTU in Prague（西波西米亚大学及Inria、布拉格CTU）

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments FGVC workshop, CVPR 2025

2504.16520 2025-04-24 cs.CV q-bio.NC 83%

A Few-Shot Metric Learning Method with Dual-Channel Attention for Cross-Modal Same-Neuron Identification

Wenwei Li, Liyi Cai, Wu Chen, Anan Li

专题命中多模态评测：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments 23 pages, 9 figures, submitted to arXiv for public access

2504.09809 2025-04-23 cs.HC cs.AI 83%

See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMs

Zhimin Li, Haichao Miao, Xinyuan Yan, Valerio Pascucci, Matthew Berger, Shusen Liu

机构 * Vanderbilt University（范德比大学）； Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）； University of Utah（犹他大学）

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.AI

2402.04178 2025-04-22 cs.CV 83%

SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models

Yichen Shi, Yuhao Gao, Yingxin Lai, Hongyang Wang, Jun Feng, Lei He, Jun Wan, Changsheng Chen, Zitong Yu, Xiaochun Cao

机构 * School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）； Department of Artificial Intelligence, Xiamen University（厦门大学人工智能系）； School of Information Science and Technology, Shijiazhuang Tiedao University（石家庄铁道大学信息科学与技术学院）； Electrical Engineering Department, UCLA（加州大学洛杉矶分校电子工程系）； New Laboratory of Pattern Recognition(NLPR), Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别新实验室）； College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）； School of Computing and Information Technology, Great Bay University（广东东莞Great Bay大学计算与信息科技学院）； School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络科学与技术学院）

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by Visual Intelligence

2504.11509 2025-04-18 cs.IR cs.CV 83%

PATFinger: Prompt-Adapted Transferable Fingerprinting against Unauthorized Multimodal Dataset Usage

Wenyi Zhang, Ju Jia, Xiaojun Jia, Yihao Huang, Xinfeng Li, Cong Wu, Lina Wang

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2504.11914 2025-04-17 cs.CV 83%

AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detection

Yuhao Chao, Jie Liu, Jie Tang, Gangshan Wu

专题命中多模态评测：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV

2504.06606 2025-04-10 cs.CV 83%

Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program

Minghe Gao, Xuqi Liu, Zhongqi Yue, Yang Wu, Shuang Chen, Juncheng Li, Siliang Tang, Fei Wu, Tat-Seng Chua, Yueting Zhuang

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2410.18387 2025-04-08 cs.CV 83%

Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Tasks

Lehan Wang, Haonan Wang, Honglong Yang, Jiaji Mao, Zehong Yang, Jun Shen, Xiaomeng Li

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted in ICLR 2025

2503.23330 2025-04-01 cs.CV 83%

EagleVision: Object-level Attribute Multimodal LLM for Remote Sensing

Hongxiang Jiang, Jihao Yin, Qixiong Wang, Jiaqi Feng, Guo Chen

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Under Review

2411.18499 2025-04-01 cs.CV 83%

OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation

Pengfei Zhou, Xiaopeng Peng, Jiajun Song, Chuanhao Li, Zhaopan Xu, Yue Yang, Ziyao Guo, Hao Zhang, Yuqi Lin, Yefei He, Lirui Zhao, Shuo Liu, Tianhua Li, Yuxuan Xie, Xiaojun Chang, Yu Qiao, Wenqi Shao, Kaipeng Zhang

专题命中多模态评测：image-text(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments 53 pages, 19 figures, accepted by CVPR 2025

2503.17261 2025-03-24 eess.IV cs.CV 83%

Cross-Modal Interactive Perception Network with Mamba for Lung Tumor Segmentation in PET-CT Images

Jie Mei, Chenyu Lin, Yu Qiu, Yaonan Wang, Hui Zhang, Ziyang Wang, Dong Dai

专题命中多模态评测：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted to CVPR 2025

2503.16185 2025-03-21 cs.CV 83%

MapGlue: Multimodal Remote Sensing Image Matching

Peihao Wu, Yongxiang Yao, Wenfei Zhang, Dong Wei, Yi Wan, Yansheng Li, Yongjun Zhang

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments The dataset and code are available at https://github.com/PeihaoWu/MapGlue

2503.14478 2025-03-20 cs.CV 83%

Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM

Xinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen, Dahua Lin

专题命中多模态评测：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Evaluation Code and dataset see https://github.com/open-compass/Creation-MMBench

2503.10079 2025-03-14 cs.CL 83%

Information Density Principle for MLLM Benchmarks

Chunyi Li, Xiaozhe Li, Zicheng Zhang, Yuan Tian, Ziheng Jia, Xiaohong Liu, Xiongkuo Min, Jia Wang, Haodong Duan, Kai Chen, Guangtao Zhai

专题命中多模态评测：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CL

2501.02509 2025-03-14 cs.CV 83%

Facial Attractiveness Prediction in Live Streaming: A New Benchmark and Multi-modal Method

Hui Li, Xiaoyu Ren, Hongjiu Yu, Huiyu Duan, Kai Li, Ying Chen, Libo Wang, Xiongkuo Min, Guangtao Zhai, Xu Liu

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Section 3 in Images Collection has description errors about data cleaning. The compared methods data of Table 3 lacks other metrics

2503.07094 2025-03-11 cs.CL 83%

A Novel Ophthalmic Benchmark for Evaluating Multimodal Large Language Models with Fundus Photographs and OCT Images

Xiaoyi Liang, Mouxiao Bian, Moxin Chen, Lihao Liu, Junjun He, Jie Xu, Lin Li

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

2503.07002 2025-03-11 cs.CV 83%

Taking Notes Brings Focus? Towards Multi-Turn Multimodal Dialogue Learning

Jiazheng Liu, Sipeng Zheng, Börje F. Karlsson, Zongqing Lu

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2502.11903 2025-03-11 cs.CL 83%

MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation

Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, Chengzhi Liu, Zhongxing Xu, Chong Zhang, Chun-Mei Feng, Yutong Xie, Imran Razzak, Zongyuan Ge, Jionglong Su, Junjun He, Yu Qiao

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CL

2503.04643 2025-03-07 eess.IV cs.CV 83%

Adaptive Prototype Learning for Multimodal Cancer Survival Analysis

Hong Liu, Haosen Yang, Federica Eduati, Josien P. W. Pluim, Mitko Veta

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 10 pages, 3 figures

2503.01887 2025-03-05 cs.LG cs.AI 83%

When Continue Learning Meets Multimodal Large Language Model: A Survey

Yukang Huo, Hao Tang

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.AI

Comments 42 pages, 6 figures, 37 tables

2503.00501 2025-03-04 cs.IR cs.CL 83%

Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions

Jia Chen, Qian Dong, Haitao Li, Xiaohui He, Yan Gao, Shaosheng Cao, Yi Wu, Ping Yang, Chen Xu, Yao Hu, Qingyao Ai, Yiqun Liu

专题命中多模态评测：multimodal(title,abstract)；image-text(abstract)；分类 cs.CL

Comments 11 pages

2412.16410 2025-02-27 cs.CL 83%

Application of Multimodal Large Language Models in Autonomous Driving

Md Robiul Islam

专题命中多模态评测：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CL

Comments 9 pages, 9 figures

2502.16486 2025-02-27 cs.CV 83%

MQADet: A Plug-and-Play Paradigm for Enhancing Open-Vocabulary Object Detection via Multimodal Question Answering

Caixiong Li, Xiongwei Zhao, Jinhang Zhang, Xing Zhang, Qihao Sun, Zhou Wu

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2410.04780 2025-02-19 cs.CV 83%

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Guanyu Zhou, Yibo Yan, Xin Zou, Kun Wang, Aiwei Liu, Xuming Hu

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by The Thirteenth International Conference on Learning Representations (ICLR 2025)

2502.05641 2025-02-11 cs.RO cs.AI 83%

Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputs

Aayam Shrestha, Pan Liu, German Ros, Kai Yuan, Alan Fern

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.AI

Journal ref The European Conference on Computer Vision (ECCV), 2024

2502.03999 2025-02-07 eess.IV cs.CV 83%

A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastoma

Ahmed Gomaa, Yixing Huang, Pluvio Stephan, Katharina Breininger, Benjamin Frey, Arnd Dörfler, Oliver Schnell, Daniel Delev, Roland Coras, Charlotte Schmitter, Jenny Stritzelberger, Sabine Semrau, Andreas Maier, Siming Bayer, Stephan Schönecker, Dieter H Heiland, Peter Hau, Udo S. Gaipl, Christoph Bert, Rainer Fietkau, Manuel A. Schmidt, Florian Putz

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2501.15795 2025-01-28 cs.CV 83%

Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?

Zhiling Chen, Hanning Chen, Mohsen Imani, Farhad Imani

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 16 pages, 11 figures

2501.11535 2025-01-22 cs.CV 83%

A baseline for machine-learning-based hepatocellular carcinoma diagnosis using multi-modal clinical data

Binwu Wang, Isaac Rodriguez, Leon Breitinger, Fabian Tollens, Timo Itzel, Dennis Grimm, Andrei Sirazitdinov, Matthias Frölich, Stefan Schönberg, Andreas Teufel, Jürgen Hesser, Wenzhao Zhao

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

1. 多模态评测 9087 篇

Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training

MM-IFEngine: Towards Multimodal Instruction Following

FungiTastic: A multi-modal dataset and benchmark for image categorization

A Few-Shot Metric Learning Method with Dual-Channel Attention for Cross-Modal Same-Neuron Identification

See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMs

SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models

PATFinger: Prompt-Adapted Transferable Fingerprinting against Unauthorized Multimodal Dataset Usage

AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detection

Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program

Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Tasks

EagleVision: Object-level Attribute Multimodal LLM for Remote Sensing

OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation

Cross-Modal Interactive Perception Network with Mamba for Lung Tumor Segmentation in PET-CT Images

MapGlue: Multimodal Remote Sensing Image Matching

Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLM

Information Density Principle for MLLM Benchmarks

Facial Attractiveness Prediction in Live Streaming: A New Benchmark and Multi-modal Method

A Novel Ophthalmic Benchmark for Evaluating Multimodal Large Language Models with Fundus Photographs and OCT Images

Taking Notes Brings Focus? Towards Multi-Turn Multimodal Dialogue Learning

MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation

Adaptive Prototype Learning for Multimodal Cancer Survival Analysis

When Continue Learning Meets Multimodal Large Language Model: A Survey

Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions

Application of Multimodal Large Language Models in Autonomous Driving

MQADet: A Plug-and-Play Paradigm for Enhancing Open-Vocabulary Object Detection via Multimodal Question Answering

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputs

A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastoma

Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?

A baseline for machine-learning-based hepatocellular carcinoma diagnosis using multi-modal clinical data