多模态大模型

RoDE: Linear Rectified Mixture of Diverse Experts for Food Large Multi-Modal Models

Pengkun Jiao, Xinlan Wu, Bin Zhu, Jingjing Chen, Chong-Wah Ngo, Yugang Jiang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models

Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xucheng Yin, Cheng-lin Liu, Lianwen Jin, Xiang Bai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Piecing It All Together: Verifying Multi-Hop Multimodal Claims

Haoran Wang, Aman Rangapur, Xiongxiao Xu, Yueqing Liang, Haroon Gharwi, Carl Yang, Kai Shu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments COLING 2025

A Dual-Module Denoising Approach with Curriculum Learning for Enhancing Multimodal Aspect-Based Sentiment Analysis

Nguyen Van Doan, Dat Tran Nguyen, Cam-Van Thi Nguyen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted at PACLIC 2024

Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models

Sherzod Hakimov, Yerkezhan Abdullayeva, Kushal Koshti, Antonia Schmidt, Yan Weiser, Anne Beyer, David Schlangen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at COLING 2025

Biological Brain Age Estimation using Sex-Aware Adversarial Variational Autoencoder with Multimodal Neuroimages

Abd Ur Rehman, Azka Rehman, Muhammad Usman, Abdullah Shahid, Sung-Min Gho, Aleum Lee, Tariq M. Khan, Imran Razzak

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Visual Error Patterns in Multi-Modal AI: A Statistical Approach

Ching-Yi Wang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

MMAUD: A Comprehensive Multi-Modal Anti-UAV Dataset for Modern Miniature Drone Threats

Shenghai Yuan, Yizhuo Yang, Thien Hoang Nguyen, Thien-Minh Nguyen, Jianfei Yang, Fen Liu, Jianping Li, Han Wang, Lihua Xie

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICRA 2024

VisScience: An Extensive Benchmark for Evaluating K12 Educational Multi-modal Scientific Reasoning

Zhihuan Jiang, Zhen Yang, Jinhao Chen, Zhengxiao Du, Weihan Wang, Bin Xu, Jie Tang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 89 pages, 70 figures

MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model

Zhen Yang, Jinhao Chen, Zhengxiao Du, Wenmeng Yu, Weihan Wang, Wenyi Hong, Zhihuan Jiang, Bin Xu, Jie Tang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 30 pages,19 figures

Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey

Jiayi Kuang, Jingyou Xie, Haohao Luo, Ronghao Li, Zhe Xu, Xianfeng Cheng, Yinghui Li, Xika Lin, Ying Shen

专题命中 多模态评测 ：MLLM(title)；multimodal(abstract)；分类 cs.CV、cs.CL

Exploring Large Language Models for Multimodal Sentiment Analysis: Challenges, Benchmarks, and Future Directions

Shezheng Song

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Multimodal large language model for wheat breeding: a new exploration of smart breeding

Guofeng Yang, Yu Li, Yong He, Zhenjiang Zhou, Lingzhen Ye, Hui Fang, Yiqi Luo, Xuping Feng

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks

Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, Chaowei Xiao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Localizing Events in Videos with Multimodal Queries

Gengyuan Zhang, Mang Ling Ada Fok, Jialu Ma, Yan Xia, Daniel Cremers, Philip Torr, Volker Tresp, Jindong Gu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 20 pages (including references and appendix); for the project homepage, see https://icq-benchmark.github.io/

MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis

Yingjie Zhou, Zicheng Zhang, Jiezhang Cao, Jun Jia, Yanwei Jiang, Farong Wen, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment

Arushi Gupta, Rafal Kocielnik, Jiayun Wang, Firdavs Nasriddinov, Cherine Yang, Elyssa Wong, Anima Anandkumar, Andrew Hung

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted as a spotlight proceedings paper at Machine Learning for Health 2024

Multi-modal Situated Reasoning in 3D Scenes

Xiongkun Linghu, Jiangyong Huang, Xuesong Niu, Xiaojian Ma, Baoxiong Jia, Siyuan Huang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024 Datasets and Benchmarks Track. Project page: https://msr3d.github.io/

Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment

Danqing Ma, Meng Wang, Ao Xiang, Zongqing Qi, Qin Yang

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Multi-Task Adversarial Variational Autoencoder for Estimating Biological Brain Age with Multimodal Neuroimaging

Muhammad Usman, Azka Rehman, Abdullah Shahid, Abd Ur Rehman, Sung-Min Gho, Aleum Lee, Tariq M. Khan, Imran Razzak

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

KnowPhish: Large Language Models Meet Multimodal Knowledge Graphs for Enhancing Reference-Based Phishing Detection

Yuexin Li, Chengyu Huang, Shumin Deng, Mei Lin Lock, Tri Cao, Nay Oo, Hoon Wei Lim, Bryan Hooi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by USENIX Security 2024

Journal ref 33rd USENIX Security Symposium (USENIX Security 2024), 793--810

MAN TruckScenes: A multimodal dataset for autonomous trucking in diverse conditions

Felix Fent, Fabian Kuttenreich, Florian Ruch, Farija Rizwin, Stefan Juergens, Lorenz Lechermann, Christian Nissler, Andrea Perl, Ulrich Voll, Min Yan, Markus Lienkamp

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024 Datasets and Benchmarks Track

An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models

Fatemeh Shiri, Xiao-Yu Guo, Mona Golestan Far, Xin Yu, Gholamreza Haffari, Yuan-Fang Li

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models

Chuhan Li, Ziyao Shangguan, Yilun Zhao, Deyuan Li, Yixin Liu, Arman Cohan

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

First Multi-Dimensional Evaluation of Flowchart Comprehension for Multimodal Large Language Models

Enming Zhang, Ruobing Yao, Huanyong Liu, Junhui Yu, Jiale Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Commonsense Knowledge Distillation for Visual Question Answering

Shuo Yang, Siwen Luo, Soyeon Caren Han

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments AAAI 2025 (Accepted, Oral)

Cross-Fundus Transformer for Multi-modal Diabetic Retinopathy Grading with Cataract

Fan Xiao, Junlin Hou, Ruiwei Zhao, Rui Feng, Haidong Zou, Lina Lu, Yi Xu, Juzhao Zhang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 4 figures

M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data

Matthew J Allen, Francisco Dorr, Joseph Alejandro Gallego Mejia, Laura Martínez-Ferrer, Anna Jungbluth, Freddie Kalaitzis, Raúl Ramos-Pollán

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 5 figures

MangaUB: A Manga Understanding Benchmark for Large Multimodal Models

Hikaru Ikuta, Leslie Wöhler, Kiyoharu Aizawa

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments This work has been submitted to the IEEE for possible publication

LiMTR: Time Series Motion Prediction for Diverse Road Users through Multimodal Feature Integration

Camiel Oerlemans, Bram Grooten, Michiel Braat, Alaa Alassi, Emilia Silvas, Decebal Constantin Mocanu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted at the NeurIPS 2024 workshop Time Series in the Age of Large Models. Code available at https://github.com/Cing2/LiMTR