多模态大模型

Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search

Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, Dacheng Tao

专题命中 多模态评测 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Technical report

Plots Unlock Time-Series Understanding in Multimodal Models

Mayank Daswani, Mathias M. J. Bellaiche, Marc Wilson, Desislav Ivanov, Mikhail Papkov, Eva Schnider, Jing Tang, Kay Lamerigts, Gabriela Botea, Michael A. Sanchez, Yojan Patel, Shruthi Prabhakara, Shravya Shetty, Umesh Telang

专题命中 多模态评测 ：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

Comments 57 pages

A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models

Jie Liu, Wenxuan Wang, Yihang Su, Jingyuan Huan, Wenting Chen, Yudi Zhang, Cheng-Yi Li, Kao-Jung Chang, Xiaohan Xin, Linlin Shen, Michael R. Lyu

专题命中 多模态评测 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments 20 pages, 15 figures

FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistant

Zhengchao Huang, Bin Xia, Zicheng Lin, Zhun Mou, Wenming Yang, Jiaya Jia

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 23 pages, 21 figures; project page: https://ffaa-vl.github.io

Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective

Meiqi Chen, Yixin Cao, Yan Zhang, Chaochao Lu

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

MME-Finance: A Multimodal Finance Benchmark for Expert-level Understanding and Reasoning

Ziliang Gan, Yu Lu, Dong Zhang, Haohan Li, Che Liu, Jian Liu, Ji Liu, Haipang Wu, Chaoyou Fu, Zenglin Xu, Rongjunchen Zhang, Yong Dai

专题命中 多模态评测 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Project Page: https://hithink-research.github.io/MME-Finance/

Joint Explicit and Implicit Cross-Modal Interaction Network for Anterior Chamber Inflammation Diagnosis

Qian Shao, Ye Dai, Haochao Ying, Kan Xu, Jinhong Wang, Wei Chi, Jian Wu

专题命中 多模态评测 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.MM

Comments IEEE MedAI 2024

Teach Multimodal LLMs to Comprehend Electrocardiographic Images

Ruoqi Liu, Yuelin Bai, Xiang Yue, Ping Zhang

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learning

Haiwen Diao, Ying Zhang, Shang Gao, Jiawen Zhu, Long Chen, Huchuan Lu

专题命中 多模态评测 ：cross-modal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.MM

Comments 12 pages, 9 figures, Accepted by TIP2024

Understanding the Role of LLMs in Multimodal Evaluation Benchmarks

Botian Jiang, Lei Li, Xiaonan Li, Zhaowei Li, Xiachong Feng, Lingpeng Kong, Qi Liu, Xipeng Qiu

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Automatically Generating Visual Hallucination Test Cases for Multimodal Large Language Models

Zhongye Liu, Hongbin Liu, Yuepeng Hu, Zedian Shao, Neil Zhenqiang Gong

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

LIME: Less Is More for MLLM Evaluation

King Zhu, Qianbo Zang, Shian Jia, Siwei Wu, Feiteng Fang, Yizhi Li, Shawn Gavin, Tuney Zheng, Jiawei Guo, Bo Li, Haoning Wu, Xingwei Qu, Jian Yang, Zachary Liu, Xiang Yue, J. H. Liu, Chenghua Lin, Min Yang, Shiwen Ni, Wenhao Huang, Ge Zhang

专题命中 多模态评测 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessment

Kai Liu, Ziqing Zhang, Wenbo Li, Renjing Pei, Fenglong Song, Xiaohong Liu, Linghe Kong, Yulun Zhang

专题命中 多模态评测 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 5 figures. The code and models will be available at https://github.com/Kai-Liu001/Dog-IQA

WorldGPT: Empowering LLM as Multimodal World Model

Zhiqi Ge, Hongzhe Huang, Mingze Zhou, Juncheng Li, Guoming Wang, Siliang Tang, Yueting Zhuang

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.AI、cs.MM

Comments update v2

A Survey on Multimodal Benchmarks: In the Era of Large AI Models

Lin Li, Guikun Chen, Hanrong Shi, Jun Xiao, Long Chen

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.AI、cs.MM

Comments Ongoing project

Unveiling Ontological Commitment in Multi-Modal Foundation Models

Mert Keser, Gesina Schwalbe, Niki Amini-Naieni, Matthias Rottmann, Alois Knoll

专题命中 多模态评测 ：multi-modal(title)；multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

Comments Qualitative Reasoning Workshop 2024 (QR2024) colocated with ECAI2024, camera-ready submission; first two authors contributed equally; 10 pages, 4 figures, 3 tables

MIP-GAF: A MLLM-annotated Benchmark for Most Important Person Localization and Group Context Understanding

Surbhi Madan, Shreya Ghosh, Lownish Rai Sookha, M. A. Ganaie, Ramanathan Subramanian, Abhinav Dhall, Tom Gedeon

专题命中 多模态评测 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted for publication at WACV 2025

Multimodal Multi-turn Conversation Stance Detection: A Challenge Dataset and Effective Model

Fuqiang Niu, Zebang Cheng, Xianghua Fu, Xiaojiang Peng, Genan Dai, Yin Chen, Hu Huang, Bowen Zhang

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.MM

Comments ACM MM2024

DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentation

Anna C. Doris, Daniele Grandi, Ryan Tomich, Md Ferdous Alam, Mohammadmehdi Ataei, Hyunmin Cheong, Faez Ahmed

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

MMBench: Is Your Multi-modal Model an All-around Player?

Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, Dahua Lin

专题命中 多模态评测 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted in ECCV2024 as Oral Presentation

ColorSwap: A Color and Word Order Dataset for Multimodal Evaluation

Jirayu Burapacheep, Ishan Gaur, Agam Bhatia, Tristan Thrush

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments ACL Findings 2024

Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts

Yi Liu, Chengjun Cai, Xiaoli Zhang, Xingliang Yuan, Cong Wang

专题命中 多模态评测 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.AI、cs.MM

Comments To be published in ACM MM 2024

CMMaTH: A Chinese Multi-modal Math Skill Evaluation Benchmark for Foundation Models

Zhong-Zhi Li, Ming-Liang Zhang, Fei Yin, Zhi-Long Ji, Jin-Feng Bai, Zhen-Ru Pan, Fan-Hu Zeng, Jian Xu, Jia-Xin Zhang, Cheng-Lin Liu

专题命中 多模态评测 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CL、cs.AI

LaVy: Vietnamese Multimodal Large Language Model

Chi Tran, Huong Le Thanh

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments 5 pages

CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs

Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, Alexis Chevalier, Sanjeev Arora, Danqi Chen

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments 121 pages, 90 figures

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

Tianle Gu, Zeyang Zhou, Kexin Huang, Dandan Liang, Yixu Wang, Haiquan Zhao, Yuanqi Yao, Xingge Qiao, Keqing Wang, Yujiu Yang, Yan Teng, Yu Qiao, Yingchun Wang

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CL、cs.AI

Visual Hallucinations of Multi-modal Large Language Models

Wen Huang, Hongbin Liu, Minxin Guo, Neil Zhenqiang Gong

专题命中 多模态评测 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments To appear in ACL Findings, 2024

Enhancing Fake News Detection in Social Media via Label Propagation on Cross-modal Tweet Graph

Wanqing Zhao, Yuta Nakashima, Haiyuan Chen, Noboru Babaguchi

专题命中 多模态评测 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CL、cs.MM

Comments 9 pages

M3GIA: A Cognition Inspired Multilingual and Multimodal General Intelligence Ability Benchmark

Wei Song, Yadong Li, Jianhua Xu, Guowei Wu, Lingfeng Ming, Kexin Yi, Weihua Luo, Houyi Li, Yi Du, Fangda Guo, Kaicheng Yu

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Multimodal Table Understanding

Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, Weiping Wang

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments 23 pages, 16 figures, ACL 2024 main conference, camera-ready version