多模态大模型

WikiMixQA: A Multimodal Benchmark for Question Answering over Tables and Charts

Negar Foroutan, Angelika Romanou, Matin Ansaripour, Julian Martin Eisenschlos, Karl Aberer, Rémi Lebret

机构 * EPFL（苏黎世联邦理工学院） ； Google DeepMind（谷歌DeepMind） ； Universidad Nacional de Córdoba（国家科隆大学） 纠错

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025 (Findings)

SemIRNet: A Semantic Irony Recognition Network for Multimodal Sarcasm Detection

Jingxuan Zhou, Yuehao Wu, Yibo Zhang, Yeyubei Zhang, Yunchong Liu, Bolin Huang, Chunhong Yuan

机构 * University of New South Wales（新南威尔士大学） ； University of Sydney（悉尼大学） ； University of Pennsylvania（宾夕法尼亚大学） ； University of Southern California（南加州大学） ； Kazan Federal University（卡扎那夫联邦大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 5 pages, 3 figures

Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models

Qianqi Yan, Yue Fan, Hongquan Li, Shan Jiang, Yang Zhao, Xinze Guan, Ching-Chen Kuo, Xin Eric Wang

机构 * University of California, Santa Cruz（加州大学圣克ruz分校） ； eBay（eBay公司） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Cross-View Multi-Modal Segmentation @ Ego-Exo4D Challenges 2025

Yuqian Fu, Runze Wang, Yanwei Fu, Danda Pani Paudel, Luc Van Gool

机构 * Fudan University（复旦大学） 纠错

专题命中 多模态评测 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments The 2nd Price Award of EgoExo4D Relations, Second Joint EgoVis Workshop with CVPR2025, technical report paper is accepted by CVPRW 25

MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learning

Yiqing Liang, Jielin Qiu, Wenhao Ding, Zuxin Liu, James Tompkin, Mengdi Xu, Mengzhou Xia, Zhengzhong Tu, Laixi Shi, Jiacheng Zhu

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments Project Webpage: https://modomodo-rl.github.io/

Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark

Ziming Cheng, Binrui Xu, Lisheng Gong, Zuhe Song, Tianshuo Zhou, Shiqi Zhong, Siyu Ren, Mingxiang Chen, Xiangchao Meng, Yuxin Zhang, Yanlin Li, Lei Ren, Wei Chen, Zhiyuan Huang, Mingjie Zhan, Xiaojie Wang, Fangxiang Feng

机构 * BUPT China（北京邮电大学） ； YSU China（云南大学） ； NUS Singapore（新加坡国立大学） ； Li Auto Inc. China（利汽车公司） ； SenseTime Research China（商汤研究） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 18 pages

Evaluating and Advancing Multimodal Large Language Models in Perception Ability Lens

Feng Chen, Chenhui Gou, Jing Liu, Yang Yang, Zhaoyang Li, Jiyuan Zhang, Zhenbang Sun, Bohan Zhuang, Qi Wu

机构 * AIML, University of Adelaide（AIML，阿德莱德大学） ； Monash University（墨尔本大学） ； Australian National University（澳大利亚国立大学） ； Tiktok, Australia（Tiktok，澳大利亚） ； Zhejiang University（浙江大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments Code repository: https://github.com/Chenfeng1271/AbilityLens/tree/main

GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking

Yufei Zhan, Ziheng Wu, Yousong Zhu, Rongkun Xue, Ruipu Luo, Zhenghao Chen, Can Zhang, Yifan Li, Zhentao He, Zheming Yang, Ming Tang, Minghui Qiu, Jinqiao Wang

机构 * Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（基础模型研究中心，自动化研究所，中国科学院） ； ByteDance（字节跳动） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学） ； Peng Cheng Laboratory（鹏城实验室） ； Wuhan AI Research（武汉人工智能研究所） ； Xi’an Jiaotong University（西安交通大学） ； Renmin University of China（中国人民大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Tech report

Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models

Qiao Liang, Yanjiang Liu, Weixiang Zhou, Ben He, Yaojie Lu, Hongyu Lin, Jia Zheng, Xianpei Han, Le Sun, Yingfei Sun

机构 * University of Chinese Academy of Sciences（中国科学院大学） ； Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所信息处理实验室） 纠错

专题命中 多模态评测 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluator

Fan Yang, Ru Zhen, Jianing Wang, Yanhao Zhang, Haoxiang Chen, Haonan Lu, Sicheng Zhao, Guiguang Ding

机构 * Tsinghua University（清华大学） ； BNRist ； OPPO AI Center（OPPO人工智能中心） ； Peking University（北京大学） ； Hangzhou Zhuoxi Institute of Brain and Intelligence（杭州智行脑科学与人工智能研究所） 纠错

专题命中 多模态评测 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2025

Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Models

Andrés Villa, Juan Carlos León Alcázar, Alvaro Soto, Bernard Ghanem

机构 * King Abdullah University of Science and Technology（卡斯土尼亚大学） 纠错

专题命中 多模态评测 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 18 pages, 10 figures, 6 tables

Multimodal Causal Reasoning Benchmark: Challenging Vision Large Language Models to Discern Causal Links Across Modalities

Zhiyuan Li, Heng Wang, Dongnan Liu, Chaoyi Zhang, Ao Ma, Jieting Long, Weidong Cai

机构 * School of Computer Science, The University of Sydney（计算机科学学院，悉尼大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments ACL2025 Findings

MT$^{3}$: Scaling MLLM-based Text Image Machine Translation via Multi-Task Reinforcement Learning

Zhaopeng Feng, Yupu Liang, Shaosheng Cao, Jiayuan Su, Jiahan Ren, Zhe Xu, Yao Hu, Wenxuan Huang, Jian Wu, Zuozhu Liu

机构 * Zhejiang University（浙江大学） ； University of Chinese Academy of Sciences（中国科学院大学） ； Xiaohongshu Inc.（小红书公司） ； East China Normal University（华东师范大学） 纠错

专题命中 多模态评测 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CL、cs.AI

Comments Work in progress

EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models

Jiamin Su, Yibo Yan, Fangteng Fu, Han Zhang, Jingheng Ye, Xiang Liu, Jiahao Huo, Huiyu Zhou, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； Beijing Future Brain Education Technology Co., Ltd.（北京未来脑教育科技有限公司） ； The Hong Kong University of Science and Technology（香港科技大学） ； Tsinghua University（清华大学） ； Guangxi Zhuang Autonomous Region Big Data Research Institute（广西壮族自治区大数据研究院） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL Findings 2025

AOR: Anatomical Ontology-Guided Reasoning for Medical Large Multimodal Model in Chest X-Ray Interpretation

Qingqiu Li, Zihang Cui, Seongsu Bae, Jilan Xu, Runtian Yuan, Yuejie Zhang, Rui Feng, Quanli Shen, Xiaobo Zhang, Junjun He, Shujun Wang

机构 * Fudan University（复旦大学） ； Xidian University（西安电子科技大学） ； KAIST（韩国科学技术院） ； Children’s Hospital of Fudan University（复旦大学附属儿童医院） ； Shanghai AI Laboratory（上海人工智能实验室） ； Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Multimodal Situational Safety

Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Anderson Compalas, Dawn Song, Xin Eric Wang

机构 * University of California, Santa Cruz（加州大学圣克ruz分校） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2025 Camera Ready

IV-Bench: A Benchmark for Image-Grounded Video Perception and Reasoning in Multimodal LLMs

David Ma, Yuanxing Zhang, Jincheng Ren, Jarvis Guo, Yifan Yao, Zhenlin Wei, Zhenzhu Yang, Zhongyuan Peng, Boyu Feng, Jun Ma, Xiao Gu, Zhoufutu Wen, King Zhu, Yancheng He, Meng Cao, Shiwen Ni, Jiaheng Liu, Wenhao Huang, Ge Zhang, Xiaojie Jin

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL

MedPromptX: Grounded Multimodal Prompting for Chest X-ray Diagnosis

Mai A. Shaaban, Adnan Khan, Mohammad Yaqub

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

A Multimodal Vision Foundation Model for Clinical Dermatology

Siyuan Yan, Zhen Yu, Clare Primiero, Cristina Vico-Alonso, Zhonghua Wang, Litao Yang, Philipp Tschandl, Ming Hu, Lie Ju, Gin Tan, Vincent Tang, Aik Beng Ng, David Powell, Paul Bonnington, Simon See, Elisabetta Magnaterra, Peter Ferguson, Jennifer Nguyen, Pascale Guitera, Jose Banuls, Monika Janda, Victoria Mar, Harald Kittler, H. Peter Soyer, Zongyuan Ge

专题命中 多模态评测 ：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

Comments 74 pages; Preprint; The code can be found at https://github.com/SiyuanYan1/PanDerm

MLLM-as-a-Judge for Image Safety without Human Labeling

Zhenting Wang, Shuming Hu, Shiyu Zhao, Xiaowen Lin, Felix Juefei-Xu, Zhuowei Li, Ligong Han, Harihar Subramanyam, Li Chen, Jianfa Chen, Nan Jiang, Lingjuan Lyu, Shiqing Ma, Dimitris N. Metaxas, Ankit Jain

专题命中 多模态评测 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.CL

Multimodal LLMs for OCR, OCR Post-Correction, and Named Entity Recognition in Historical Documents

Gavin Greif, Niclas Griesshaber, Robin Greif

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Hierarchical Adaptive Expert for Multimodal Sentiment Analysis

Jiahao Qin, Feng Liu, Lu Zong

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments 11 pages, 3 figures

Grounded Chain-of-Thought for Multimodal Large Language Models

Qiong Wu, Xiangcong Yang, Yiyi Zhou, Chenxin Fang, Baiyang Song, Xiaoshuai Sun, Rongrong Ji

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.MM

MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs

Yusu Qian, Hanrong Ye, Jean-Philippe Fauconnier, Peter Grasch, Yinfei Yang, Zhe Gan

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments Accepted at ICLR 2025

Towards Harmless Multimodal Assistants with Blind Preference Optimization

Yongqi Li, Lu Yang, Jian Wang, Runyang You, Wenjie Li, Liqiang Nie

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

A Multimodal Benchmark Dataset and Model for Crop Disease Diagnosis

Xiang Liu, Zhaoxiang Liu, Huan Hu, Zezhou Chen, Kohou Wang, Kai Wang, Shiguo Lian

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV 2024 (14 pages, 8 figures)

MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs

Xuannan Liu, Zekun Li, Peipei Li, Huaibo Huang, Shuhan Xia, Xing Cui, Linzhi Huang, Weihong Deng, Zhaofeng He

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ICLR 2025, Project page: https://liuxuannan.github.io/MMFakeBench.github.io/

A Simple Aerial Detection Baseline of Multimodal Language Models

Qingyun Li, Yushi Chen, Xinya Shu, Dong Chen, Xin He, Yi Yu, Xue Yang

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 4 pages, 1 table, 4 figures

Toward Robust Early Detection of Alzheimer's Disease via an Integrated Multimodal Learning Approach

Yifei Chen, Shenghao Zhu, Zhaojie Fang, Chang Liu, Binfeng Zou, Yuhe Wang, Shuo Chang, Fan Jia, Feiwei Qin, Jin Fan, Yong Peng, Changmiao Wang

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 2 figures

Journal ref ICASSP 2025

ChemDFM-X: Towards Large Multimodal Model for Chemistry

Zihan Zhao, Bo Chen, Jingpiao Li, Lu Chen, Liyang Wen, Pengyu Wang, Zichen Zhu, Danyang Zhang, Ziping Wan, Yansi Li, Zhongyang Dai, Xin Chen, Kai Yu

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.MM

Comments 19 pages, 7 figures, 11 tables