多模态大模型

SemiHVision: Enhancing Medical Multimodal Models with a Semi-Human Annotated Dataset and Fine-Tuned Instruction Generation

Junda Wang, Yujan Ting, Eric Z. Chen, Hieu Tran, Hong Yu, Weijing Huang, Terrence Chen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Difficult Task Yes but Simple Task No: Unveiling the Laziness in Multimodal LLMs

Sihang Zhao, Youliang Yuan, Xiaoying Tang, Pinjia He

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 Findings

Introducing MeMo: A Multimodal Dataset for Memory Modelling in Multiparty Conversations

Maria Tsfasman, Bernd Dudzik, Kristian Fenech, Andras Lorincz, Catholijn M. Jonker, Catharine Oertel

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

MiRAGeNews: Multimodal Realistic AI-Generated News Detection

Runsheng Huang, Liam Dugan, Yue Yang, Chris Callison-Burch

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2024 Findings

GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs

Navid Rajabi, Jana Kosecka

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to NeurIPS 2024 Workshop on Compositional Learning

Needle In A Multimodal Haystack

Weiyun Wang, Shuibo Zhang, Yiming Ren, Yuchen Duan, Tiantong Li, Shuo Liu, Mengkang Hu, Zhe Chen, Kaipeng Zhang, Lewei Lu, Xizhou Zhu, Ping Luo, Yu Qiao, Jifeng Dai, Wenqi Shao, Wenhai Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024 Track Datasets and Benchmarks

DTVLT: A Multi-modal Diverse Text Benchmark for Visual Language Tracking Based on LLM

Xuchen Li, Shiyu Hu, Xiaokun Feng, Dailing Zhang, Meiqi Wu, Jing Zhang, Kaiqi Huang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Preprint, Under Review

R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?

Chunyi Li, Jianbo Zhang, Zicheng Zhang, Haoning Wu, Yuan Tian, Wei Sun, Guo Lu, Xiaohong Liu, Xiongkuo Min, Weisi Lin, Guangtao Zhai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Exploring Attention Mechanisms in Integration of Multi-Modal Information for Sign Language Recognition and Translation

Zaber Ibn Abdul Hakim, Rasman Mubtasim Swargo, Muhammad Abdullah Adnan

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

UlcerGPT: A Multimodal Approach Leveraging Large Language and Vision Models for Diabetic Foot Ulcer Image Transcription

Reza Basiri, Ali Abedi, Chau Nguyen, Milos R. Popovic, Shehroz S. Khan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 13 pages, 3 figures, ICPR 2024 Conference (PRHA workshop)

BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data

Xuwu Wang, Qiwen Cui, Yunzhe Tao, Yiran Wang, Ziwei Chai, Xiaotian Han, Boyi Liu, Jianbo Yuan, Jing Su, Guoyin Wang, Tingkai Liu, Liyu Chen, Tianyi Liu, Tao Sun, Yufeng Zhang, Sirui Zheng, Quanzeng You, Yang Yang, Hongxia Yang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Insight: A Multi-Modal Diagnostic Pipeline using LLMs for Ocular Surface Disease Diagnosis

Chun-Hsiao Yeh, Jiayun Wang, Andrew D. Graham, Andrea J. Liu, Bo Tan, Yubei Chen, Yi Ma, Meng C. Lin

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to MICCAI 2024. Project Webpage: https://danielchyeh.github.io/MDPipe/

MM-Conv: A Multi-modal Conversational Dataset for Virtual Humans

Anna Deichler, Jim O'Regan, Jonas Beskow

专题命中 多模态评测 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.CL

MMMT-IF: A Challenging Multimodal Multi-Turn Instruction Following Benchmark

Elliot L. Epstein, Kaisheng Yao, Jing Li, Xinyi Bai, Hamid Palangi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 24 pages, 16 figures

MaViLS, a Benchmark Dataset for Video-to-Slide Alignment, Assessing Baseline Accuracy with a Multimodal Alignment Algorithm Leveraging Speech, OCR, and Visual Features

Katharina Anderer, Andreas Reich, Matthias Wölfel

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref Proceedings of Interspeech 2024

Model-in-the-Loop (MILO): Accelerating Multimodal AI Data Annotation with LLMs

Yifan Wang, David Stevens, Pranay Shah, Wenwen Jiang, Miao Liu, Xu Chen, Robert Kuo, Na Li, Boying Gong, Daniel Lee, Jiabo Hu, Ning Zhang, Bob Kamma

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Multi-modal Auto-regressive Modeling via Visual Words

Tianshuo Peng, Zuchao Li, Lefei Zhang, Hai Zhao, Ping Wang, Bo Du

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments ACM MM 2024

m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks

Zixian Ma, Weikai Huang, Jieyu Zhang, Tanmay Gupta, Ranjay Krishna

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checking

Ting-Chih Chen, Chia-Wei Tang, Chris Thomas

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 7 figures, The 62nd Annual Meeting of the Association for Computational Linguistics

Mutual Information-based Representations Disentanglement for Unaligned Multimodal Language Sequences

Fan Qian, Jiqing Han, Jianchen Li, Yongjun He, Tieran Zheng, Guibin Zheng

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments 31 pages, 8 figures

MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks

Letitia Parcalabescu, Anette Frank

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Published at ACL 2023 Main (Toronto); 10 pages, 14 appendix pages, 11 figures, 3 tables

Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models

Yupan Huang, Zaiqiao Meng, Fangyu Liu, Yixuan Su, Nigel Collier, Yutong Lu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments ICLR 2024 Workshop (Navigating and Addressing Data Problems for Foundation Models)

Visual Language Tracking with Multi-modal Interaction: A Robust Benchmark

Xuchen Li, Shiyu Hu, Xiaokun Feng, Dailing Zhang, Meiqi Wu, Jing Zhang, Kaiqi Huang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Under Review

Nonverbal Immediacy Analysis in Education: A Multimodal Computational Model

Uroš Petković, Jonas Frenkel, Olaf Hellwich, Rebecca Lazarides

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 3 figures. Camera-ready version for the SAB 2024: 17th International Conference on the Simulation of Adaptive Behavior

Towards Patronizing and Condescending Language in Chinese Videos: A Multimodal Dataset and Detector

Hongbo Wang, Junyu Lu, Yan Han, Kai Ma, Liang Yang, Hongfei Lin

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Under review in ICASSP 2025

PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis

Meng Luo, Hao Fei, Bobo Li, Shengqiong Wu, Qian Liu, Soujanya Poria, Erik Cambria, Mong-Li Lee, Wynne Hsu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACM MM 2024 (Oral)

FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models

Yixuan Li, Xuelin Liu, Xiaoyang Wang, Bu Sung Lee, Shiqi Wang, Anderson Rocha, Weisi Lin

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms

Yiqiao Jin, Minje Choi, Gaurav Verma, Jindong Wang, Srijan Kumar

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments In Proceedings of ACL 2024

Knowledge-Aware Reasoning over Multimodal Semi-structured Tables

Suyash Vardhan Mathur, Jainit Sushil Bafna, Kunal Kartik, Harshita Khandelwal, Manish Shrivastava, Vivek Gupta, Mohit Bansal, Dan Roth

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Exploring Multi-modal Neural Scene Representations With Applications on Thermal Imaging

Mert Özer, Maximilian Weiherer, Martin Hundhausen, Bernhard Egger

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ECCVW'24