多模态大模型

Target-Dependent Multimodal Sentiment Analysis Via Employing Visual-to Emotional-Caption Translation Network using Visual-Caption Pairs

Ananya Pandey, Dinesh Kumar Vishwakarma

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Characterizing Multimodal Long-form Summarization: A Case Study on Financial Reports

Tianyu Cao, Natraj Raman, Danial Dervovic, Chenhao Tan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

PathInsight: Instruction Tuning of Multimodal Datasets and Models for Intelligence Assisted Diagnosis in Histopathology

Xiaomin Wu, Rui Xu, Pengchen Wei, Wenkang Qin, Peixiang Huang, Ziheng Li, Lin Luo

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 2 figures

User-in-the-loop Evaluation of Multimodal LLMs for Activity Assistance

Mrinal Verghese, Brian Chen, Hamid Eghbalzadeh, Tushar Nagarajan, Ruta Desai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 4 figures

An analysis of HOI: using a training-free method with multimodal visual foundation models when only the test set is available, without the training set

Chaoyi Ai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs

Peng Ding, Jingyu Wu, Jun Kuang, Dan Ma, Xuezhi Cao, Xunliang Cai, Shi Chen, Jiajun Chen, Shujian Huang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments Acccepted by ACM MM 2024, 14 pages, 11 figures, 9 tables

D-Rax: Domain-specific Radiologic assistant leveraging multi-modal data and eXpert model predictions

Hareem Nisar, Syed Muhammad Anwar, Zhifan Jiang, Abhijeet Parida, Ramon Sanchez-Jacob, Vishwesh Nath, Holger R. Roth, Marius George Linguraru

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments accepted to the MICCAI 2024 Second International Workshop on Foundation Models for General Medical AI

ERIT Lightweight Multimodal Dataset for Elderly Emotion Recognition and Multimodal Fusion Evaluation

Rita Frieske, Bertram E. Shi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

GigaPevt: Multimodal Medical Assistant

Pavel Blinov, Konstantin Egorov, Ivan Sviridov, Nikolay Ivanov, Stepan Botman, Evgeniy Tagin, Stepan Kudin, Galina Zubkova, Andrey Savchenko

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments IJCAI 2024, 4 pages, 2 figures, 2 tables

Journal ref Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI) Demo Track, 2024, pp. 8614-8618

Multi-modal Imaging Genomics Transformer: Attentive Integration of Imaging with Genomic Biomarkers for Schizophrenia Classification

Nagur Shareef Shaik, Teja Krishna Cherukuri, Vince D. Calhoun, Dong Hye Ye

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted for presentation at the AI for Imaging Genomic Learning (AIIG) Workshop, MICCAI 2024

ConTextual: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Models

Rohan Wadhawan, Hritik Bansal, Kai-Wei Chang, Nanyun Peng

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref PMLR 235:49733-49787, 2024

Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic

Fakhraddin Alwajih, Gagan Bhatia, Muhammad Abdul-Mageed

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

3MVRD: Multimodal Multi-task Multi-teacher Visually-Rich Form Document Understanding

Yihao Ding, Lorenzo Vaiani, Caren Han, Jean Lee, Paolo Garza, Josiah Poon, Luca Cagliero

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at Findings of ACL 2024

I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognition

Yannis Vasilakis, Rachel Bittner, Johan Pauwels

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to ISMIR 2024

How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts

Yusu Qian, Haotian Zhang, Yinfei Yang, Zhe Gan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Bosch Street Dataset: A Multi-Modal Dataset with Imaging Radar for Automated Driving

Karim Armanious, Maurice Quach, Michael Ulrich, Timo Winterling, Johannes Friesen, Sascha Braun, Daniel Jenet, Yuri Feldman, Eitan Kosman, Philipp Rapp, Volker Fischer, Marc Sons, Lukas Kohns, Daniel Eckstein, Daniela Egbert, Simone Letsch, Corinna Voege, Felix Huttner, Alexander Bartler, Robert Maiwald, Yancong Lin, Ulf Rüegg, Claudius Gläser, Bastian Bischoff, Jascha Freess, Karsten Haug, Kathrin Klee, Holger Caesar

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Visualization Literacy of Multimodal Large Language Models: A Comparative Study

Zhimin Li, Haichao Miao, Valerio Pascucci, Shusen Liu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents

Shrestha Mohanty, Negar Arabzadeh, Andrea Tupini, Yuxuan Sun, Alexey Skrynnik, Artem Zholus, Marc-Alexandre Côté, Julia Kiseleva

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

RoBus: A Multimodal Dataset for Controllable Road Networks and Building Layouts Generation

Tao Li, Ruihang Li, Huangnan Zheng, Shanding Ye, Shijian Li, Zhijie Pan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

TransMA: an explainable multi-modal deep learning model for predicting properties of ionizable lipid nanoparticles in mRNA delivery

Kun Wu, Zixu Wang, Xiulong Yang, Yangyang Chen, Zhenqi Han, Jialu Zhang, Lizhuang Liu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 9 figures

Stark: Social Long-Term Multi-Modal Conversation with Persona Commonsense Knowledge

Young-Jun Lee, Dokyong Lee, Junyoung Youn, Kyeongjin Oh, Byungsoo Ko, Jonghwan Hyeon, Ho-Jin Choi

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Project website: https://stark-dataset.github.io

Emotion and Intent Joint Understanding in Multimodal Conversation: A Benchmarking Dataset

Rui Liu, Haolin Zuo, Zheng Lian, Xiaofen Xing, Björn W. Schuller, Haizhou Li

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 8 figures, 12 tables, NeurIPS 2024 Dataset and Benchmark Track

MARS: Multimodal Active Robotic Sensing for Articulated Characterization

Hongliang Zeng, Ping Zhang, Chengjiong Wu, Jiahua Wang, Tingyu Ye, Fang Li

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts

Shubhankar Singh, Purvi Chaurasia, Yerram Varun, Pranshu Pandya, Vatsal Gupta, Vivek Gupta, Dan Roth

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted in ACL 2024 (Findings), 21 pages, 7 figures, 9 Tables

MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations

Hanlei Zhang, Xin Wang, Hua Xu, Qianrui Zhou, Kai Gao, Jianhua Su, jinyue Zhao, Wenrui Li, Yanting Chen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted by ICLR 2024, Long Paper; The abstract is slightly modified due to the length limitation

EHR-Based Mobile and Web Platform for Chronic Disease Risk Prediction Using Large Language Multimodal Models

Chun-Chieh Liao, Wei-Ting Kuo, I-Hsuan Hu, Yen-Chen Shih, Jun-En Ding, Feng Liu, Fang-Ming Hung

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Unveiling the Impact of Multi-Modal Interactions on User Engagement: A Comprehensive Evaluation in AI-driven Conversations

Lichao Zhang, Jia Yu, Shuai Zhang, Long Li, Yangyang Zhong, Guanbao Liang, Yuming Yan, Qing Ma, Fangsheng Weng, Fayu Pan, Jing Li, Renjun Xu, Zhenzhong Lan

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking

Siyuan Wang, Zhuohan Long, Zhihao Fan, Zhongyu Wei

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Plug-and-Play Grounding of Reasoning in Multimodal Large Language Models

Jiaxing Chen, Yuxuan Liu, Dehu Li, Xiang An, Weimo Deng, Ziyong Feng, Yongle Zhao, Yin Xie

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 15 pages, 8 figures

Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding

Tuo Zhang, Tiantian Feng, Yibin Ni, Mengqin Cao, Ruying Liu, Katharine Butler, Yanjun Weng, Mi Zhang, Shrikanth S. Narayanan, Salman Avestimehr

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI