多模态大模型

Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMs

Jonathan Roberts, Timo Lüddecke, Rehan Sheikh, Kai Han, Samuel Albanie

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments V3: Fixed typo in Fig.1; V2: Minor formatting changes and added missing subfigure captions

CXR-LLAVA: a multimodal large language model for interpreting chest X-ray images

Seowoo Lee, Jiwon Youn, Hyungjin Kim, Mansu Kim, Soon Ho Yoon

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

An EcoSage Assistant: Towards Building A Multimodal Plant Care Dialogue Assistant

Mohit Tomar, Abhisek Tiwari, Tulika Saha, Prince Jha, Sriparna Saha

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CL、cs.AI

Task Selection and Assignment for Multi-modal Multi-task Dialogue Act Classification with Non-stationary Multi-armed Bandits

Xiangheng He, Junjie Chen, Björn W. Schuller

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICASSP 2024

IdentiFace : A VGG Based Multimodal Facial Biometric System

Mahmoud Rabea, Hanya Ahmed, Sohaila Mahmoud, Nourhan Sayed

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 22 figures and 9 images

Multi-modal Machine Learning for Vehicle Rating Predictions Using Image, Text, and Parametric Data

Hanqi Su, Binyang Song, Faez Ahmed

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments The paper submitted to IDETC/CIE2023, the International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, has been accepted

MedSumm: A Multimodal Approach to Summarizing Code-Mixed Hindi-English Clinical Queries

Akash Ghosh, Arkadeep Acharya, Prince Jha, Aniket Gaudgaul, Rajdeep Majumdar, Sriparna Saha, Aman Chadha, Raghav Jain, Setu Sinha, Shivani Agarwal

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments ECIR 2024

DocMSU: A Comprehensive Benchmark for Document-level Multimodal Sarcasm Understanding

Hang Du, Guoshun Nan, Sicheng Zhang, Binzhu Xie, Junrui Xu, Hehe Fan, Qimei Cui, Xiaofeng Tao, Xudong Jiang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models

Bingbing Wen, Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Bill Howe, Lijuan Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

EDA: Evolving and Distinct Anchors for Multimodal Motion Prediction

Longzhong Lin, Xuewu Lin, Tianwei Lin, Lichao Huang, Rong Xiong, Yue Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI2024)

A Foundational Multimodal Vision Language AI Assistant for Human Pathology

Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Kenji Ikamura, Georg Gerber, Ivy Liang, Long Phi Le, Tong Ding, Anil V Parwani, Faisal Mahmood

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

VRPTEST: Evaluating Visual Referring Prompting in Large Multimodal Models

Zongjie Li, Chaozheng Wang, Chaowei Liu, Pingchuan Ma, Daoyuan Wu, Shuai Wang, Cuiyun Gao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 13 pages

MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning

Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, Lijuan Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Project page at https://mm-narrator.github.io/

Intelligent Knee Sleeves: A Real-time Multimodal Dataset for 3D Lower Body Motion Estimation Using Smart Textile

Wenwen Zhang, Arvin Tashakori, Zenan Jiang, Amir Servati, Harishkumar Narayana, Saeid Soltanian, Rou Yi Yeap, Meng Han Ma, Lauren Toy, Peyman Servati

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by Thirty-seventh Conference on Neural Information Processing Systems (Neurips) D&B Track

INSPECT: A Multimodal Dataset for Pulmonary Embolism Diagnosis and Prognosis

Shih-Cheng Huang, Zepeng Huo, Ethan Steinberg, Chia-Chun Chiang, Matthew P. Lungren, Curtis P. Langlotz, Serena Yeung, Nigam H. Shah, Jason A. Fries

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos

Jielin Qiu, Jiacheng Zhu, William Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, Bo Li, Lijuan Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Project website: https://mmsum-dataset.github.io/

M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models

Wenxuan Zhang, Sharifah Mahani Aljunied, Chang Gao, Yew Ken Chia, Lidong Bing

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments NeurIPS 2023 (Datasets and Benchmarks)

More Robots are Coming: Large Multimodal Models (ChatGPT) can Solve Visually Diverse Images of Parsons Problems

Irene Hou, Owen Man, Sophie Mettille, Sebastian Gutierrez, Kenneth Angelikas, Stephen MacNeil

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning

Enna Sachdeva, Nakul Agarwal, Suhas Chundi, Sean Roelofs, Jiachen Li, Mykel Kochenderfer, Chiho Choi, Behzad Dariush

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Dynamic Task and Weight Prioritization Curriculum Learning for Multimodal Imagery

Huseyin Fuat Alsan, Taner Arsan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

A Novel Site-Agnostic Multimodal Deep Learning Model to Identify Pro-Eating Disorder Content on Social Media

Jonathan Feldman

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Perception Test: A Diagnostic Benchmark for Multimodal Video Models

Viorica Pătrăucean, Lucas Smaira, Ankush Gupta, Adrià Recasens Continente, Larisa Markeeva, Dylan Banarse, Skanda Koppula, Joseph Heyward, Mateusz Malinowski, Yi Yang, Carl Doersch, Tatiana Matejovicova, Yury Sulsky, Antoine Miech, Alex Frechette, Hanna Klimczak, Raphael Koster, Junlin Zhang, Stephanie Winkler, Yusuf Aytar, Simon Osindero, Dima Damen, Andrew Zisserman, João Carreira

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks

ViCLEVR: A Visual Reasoning Dataset and Hybrid Multimodal Fusion Model for Visual Question Answering in Vietnamese

Khiem Vinh Tran, Hao Phu Phan, Kiet Van Nguyen, Ngan Luu Thuy Nguyen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments A pre-print version and submitted to journal

M2DF: Multi-grained Multi-curriculum Denoising Framework for Multimodal Aspect-based Sentiment Analysis

Fei Zhao, Chunhui Li, Zhen Wu, Yawen Ouyang, Jianbing Zhang, Xinyu Dai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted by EMNLP 2023

BaitBuster-Bangla: A Comprehensive Dataset for Clickbait Detection in Bangla with Multi-Feature and Multi-Modal Analysis

Abdullah Al Imran, Md Sakib Hossain Shovon, M. F. Mridha

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

EXMODD: An EXplanatory Multimodal Open-Domain Dialogue dataset

Hang Yin, Pinren Lu, Ziang Li, Bin Sun, Kan Li

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Evaluating Explainable AI on a Multi-Modal Medical Imaging Task: Can Existing Algorithms Fulfill Clinical Requirements?

Weina Jin, Xiaoxiao Li, Ghassan Hamarneh

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments AAAI 2022

MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Tasks

Xiaocui Yang, Wenfang Wu, Shi Feng, Ming Wang, Daling Wang, Yang Li, Qi Sun, Yifei Zhang, Xiaoming Fu, Soujanya Poria

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Underview

XVO: Generalized Visual Odometry via Cross-Modal Self-Training

Lei Lai, Zhongkai Shangguan, Jimuyang Zhang, Eshed Ohn-Bar

专题命中 多模态评测 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023, Paris https://genxvo.github.io/

Application of frozen large-scale models to multimodal task-oriented dialogue

Tatsuki Kawamoto, Takuma Suzuki, Ko Miyama, Takumi Meguro, Tomohiro Takagi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI