多模态大模型

ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models

Yahan Tu, Rui Hu, Jitao Sang

机构 * Beijing Jiaotong University（北京交通大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

SurgVisAgent: Multimodal Agentic Model for Versatile Surgical Visual Enhancement

Zeyu Lei, Hongyuan Yu, Jinlin Wu, Zhen Chen

机构 * University of Chinese Academy of Sciences Multimedia Department, Xiaomi Inc Chinese Academy of Sciences, Institute of Automation Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness

Zixin Chen, Hongzhan Lin, Kaixin Li, Ziyang Luo, Zhen Ye, Guang Chen, Zhiyong Huang, Jing Ma

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2025

OmniEval: A Benchmark for Evaluating Omni-modal Models with Visual, Auditory, and Textual Inputs

Yiman Zhang, Ziheng Luo, Qiangyu Yan, Wei He, Borui Jiang, Xinghao Chen, Kai Han

机构 * Huawei Noah’s Ark Lab（华为诺亚实验室） ； University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 多模态评测 ：omni-modal(title)；audio-visual(abstract)；分类 cs.CV、cs.AI

M3SD: Multi-modal, Multi-scenario and Multi-language Speaker Diarization Dataset

Shilong Wu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.MM、eess.AS

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Shouwei Ruan, Jialing Tao, YueFeng Chen, Hui Xue, Xingxing Wei

机构 * Institution1（机构1） ； Institution2（机构2） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments ICCV2025

SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model

Zhenglin Huang, Jinwei Hu, Xiangtai Li, Yiwei He, Xingyu Zhao, Bei Peng, Baoyuan Wu, Xiaowei Huang, Guangliang Cheng

机构 * University of Liverpool, UK（利物浦大学） ； Nanyang Technological University, SG（南洋理工大学） ； WMG, University of Warwick（沃里克大学工程学院） ； The Chinese University of Hong Kong, Shenzhen, Guangdong, China（香港中文大学（深圳）） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments This version revises and corrects the metric calculations in the tables

MIRAGE: A Multi-modal Benchmark for Spatial Perception, Reasoning, and Intelligence

Chonghan Liu, Haoran Wang, Felix Henry, Pu Miao, Yajie Zhang, Yu Zhao, Peiran Wu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Robustness Evaluation of OCR-based Visual Document Understanding under Multi-Modal Adversarial Attacks

Dong Nguyen Tien, Dung D. Le

机构 * VinUniversity Hanoi（河内Vin大学） 纠错

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 1 figure, under review at EMNLP 2025

On Path to Multimodal Historical Reasoning: HistBench and HistAgent

Jiahao Qiu, Fulian Xiao, Yimin Wang, Yuchen Mao, Yijia Chen, Xinzhe Juan, Shu Zhang, Siran Wang, Xuan Qi, Tongcheng Zhang, Zixin Yao, Jiacheng Guo, Yifu Lu, Charles Argon, Jundi Cui, Daixin Chen, Junran Zhou, Shuyao Zhou, Zhanpeng Zhou, Ling Yang, Shilong Liu, Hongru Wang, Kaixuan Huang, Xun Jiang, Yuming Cao, Yue Chen, Yunfei Chen, Zhengyi Chen, Ruowei Dai, Mengqiu Deng, Jiye Fu, Yunting Gu, Zijie Guan, Zirui Huang, Xiaoyan Ji, Yumeng Jiang, Delong Kong, Haolong Li, Jiaqi Li, Ruipeng Li, Tianze Li, Zhuoran Li, Haixia Lian, Mengyue Lin, Xudong Liu, Jiayi Lu, Jinghan Lu, Wanyu Luo, Ziyue Luo, Zihao Pu, Zhi Qiao, Ruihuan Ren, Liang Wan, Ruixiang Wang, Tianhui Wang, Yang Wang, Zeyu Wang, Zihua Wang, Yujia Wu, Zhaoyi Wu, Hao Xin, Weiao Xing, Ruojun Xiong, Weijie Xu, Yao Shu, Yao Xiao, Xiaorui Yang, Yuchen Yang, Nan Yi, Jiadong Yu, Yangyuxuan Yu, Huiting Zeng, Danni Zhang, Yunjie Zhang, Zhaoyu Zhang, Zhiheng Zhang, Xiaofeng Zheng, Peirong Zhou, Linyan Zhong, Xiaoyin Zong, Ying Zhao, Zhenxin Chen, Lin Ding, Xiaoyu Gao, Bingbing Gong, Yichao Li, Yang Liao, Guang Ma, Tianyuan Ma, Xinrui Sun, Tianyi Wang, Han Xia, Ruobing Xian, Gen Ye, Tengfei Yu, Wentao Zhang, Yuxi Wang, Xi Gao, Mengdi Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 7 figures

Dual Thinking and Logical Processing -- Are Multi-modal Large Language Models Closing the Gap with Human Vision ?

Kailas Dayanandan, Nikhil Kumar, Anand Sinha, Brejesh Lall

机构 * Indian Institute of Technology Delhi（印度理工学院德里） 纠错

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference

Kunxi Li, Zhonghua Jiang, Zhouzhou Shen, Zhaode Wang, Chengfei Lv, Shengyu Zhang, Fan Wu, Fei Wu

机构 * Zhejiang University（浙江大学） ； Southeast University（东南大学） ； Alibaba（阿里巴巴） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs

Ai Jian, Weijie Qiu, Xiaokun Wang, Peiyu Wang, Yunzhuo Hao, Jiangbo Pei, Yichen Wei, Yi Peng, Xuchen Song

机构 * Skywork AI ； Kunlun Inc. 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 36 pages

PunchBench: Benchmarking MLLMs in Multimodal Punchline Comprehension

Kun Ouyang, Yuanxin Liu, Shicheng Li, Yi Liu, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

机构 * State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学） ； WeChat AI, Tencent Inc., China（微信AI，腾讯公司，中国） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments This is the camera-ready version for ACL 2025

MS4UI: A Dataset for Multi-modal Summarization of User Interface Instructional Videos

Yuan Zang, Hao Tan, Seunghyun Yoon, Franck Dernoncourt, Jiuxiang Gu, Kushal Kafle, Chen Sun, Trung Bui

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

AD^2-Bench: A Hierarchical CoT Benchmark for MLLM in Autonomous Driving under Adverse Conditions

Zhaoyang Wei, Chenhui Qiang, Bowen Jiang, Xumeng Han, Xuehui Yu, Zhenjun Han

机构 * University of Chinese Academy of Sciences(UCAS)（中国科学院大学） 纠错

专题命中 多模态评测 ：MLLM(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

ImageChain: Advancing Sequential Image-to-Text Reasoning in Multimodal Large Language Models

Danae Sánchez Villegas, Ingo Ziegler, Desmond Elliott

机构 * University of Copenhagen（哥本哈根大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Code, dataset, and checkpoints are publicly available at https://github.com/danaesavi/ImageChain; v2: added human annotation study to validate SimRate

Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models

Zahra Babaiee, Peyman M. Kiasari, Daniela Rus, Radu Grosu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Interpretable Multimodal Framework for Human-Centered Street Assessment: Integrating Visual-Language Models for Perceptual Urban Diagnostics

HaoTian Lan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 24 pages, 10 figures

OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning

Ling Fu, Zhebin Kuang, Jiajun Song, Mingxin Huang, Biao Yang, Yuzhe Li, Linghao Zhu, Qidi Luo, Xinyu Wang, Hao Lu, Zhang Li, Guozhi Tang, Bin Shan, Chunhui Lin, Qi Liu, Binghong Wu, Hao Feng, Hao Liu, Can Huang, Jingqun Tang, Wei Chen, Lianwen Jin, Yuliang Liu, Xiang Bai

机构 * Huazhong University of Science and Technology（华中科技大学） ； South China University of Technology（华南理工大学） ； University of Adelaide（阿德莱德大学） ； ByteDance（字节跳动） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Seeing What Tastes Good: Revisiting Multimodal Distributional Semantics in the Billion Parameter Era

Dan Oneata, Desmond Elliott, Stella Frank

机构 * politehnica Bucharest（布加勒斯特理工大学） ； Pioneer Center for AI（先锋人工智能中心） ； Department of Computer Science, University of Copenhagen（哥本哈根大学计算机科学系） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments ACL Findings 2025

The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights

Yufang Liu, Yao Du, Tao Ji, Jianing Wang, Yang Liu, Yuanbin Wu, Aimin Zhou, Mengdi Zhang, Xunliang Cai

机构 * School of Computer Science and Technology, East China Normal University（东华师范大学计算机科学与技术学院） ； Meituan Inc.（美团公司） ； Fudan University（复旦大学） ； Pazhou Laboratory（Pazhou实验室） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models

Huanqia Cai, Yijun Yang, Winston Hu

机构 * Tencent Hunyuan Team（腾讯文言团队） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Research on Driving Scenario Technology Based on Multimodal Large Lauguage Model Optimization

Wang Mengjie, Zhu Huiping, Li Jian, Shi Wenxiu, Zhang Song

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

MUDI: A Multimodal Biomedical Dataset for Understanding Pharmacodynamic Drug-Drug Interactions

Tung-Lam Ngo, Ba-Hoang Tran, Duy-Cat Can, Trung-Hieu Do, Oliver Y. Chén, Hoang-Quynh Le

机构 * VNU University of Engineering and Technology (VNU-UET)（越南工程技术大学） ； Lausanne University Hospital (CHUV)（洛桑大学医院） ； University of Lausanne (UNIL)（洛桑大学） ； Hanoi Medical University（河内医学院） ； National Geriatric Hospital, Hanoi（河内国立老年医院） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs

Xiaoyuan Liu, Wenxuan Wang, Youliang Yuan, Jen-tse Huang, Qiuzhi Liu, Pinjia He, Zhaopeng Tu

机构 * School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院） ； Tencent（腾讯） ； Renmin University of China（中国人民大学） ； Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by ACL 2025 main

Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings

Shujian Yang, Shiyao Cui, Chuanrui Hu, Haicheng Wang, Tianwei Zhang, Minlie Huang, Jialiang Lu, Han Qiu

机构 * Shanghai Jiao Tong University（上海交通大学） ； Tsinghua University（清华大学） ； Qihoo 360（奇安信） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2025 (Findings). Camera-ready version

Large Language Model-Driven Distributed Integrated Multimodal Sensing and Semantic Communications

Yubo Peng, Luping Xiang, Bingxin Zhang, Kun Yang

机构 * State Key Laboratory of Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学） ； School of Intelligent Software and Engineering, Nanjing University (Suzhou Campus)（智能软件与工程学院，南京大学（苏州校区）） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models

Kening Zheng, Junkai Chen, Yibo Yan, Xin Zou, Xuming Hu

机构 * Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） ； Guangxi Zhuang Autonomous Region Big Data Research Institute（广西壮族自治区大数据研究院） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by Findings of ACL 2025

Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Caifeng Shan, Ran He, Xing Sun

机构 * State Key Laboratory for Novel Software Technology（新型软件技术国家重点实验室） ； School of Intelligence Science and Technology（智能科学与技术学院） ； State Key Laboratory of Cognitive Intelligence（认知智能国家重点实验室） ； XMU（厦门大学） ； HKU（香港大学） ； PKU（北京大学） ； CUHK（香港中文大学） ； ECNU（华东师范大学） ； CASIA（中国科学院自动化研究所） 纠错

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Project Page: https://video-mme.github.io