多模态大模型 - arXivDaily 专题

2501.07819 2025-01-15 cs.CV 83%

3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding

Haomiao Xiong, Yunzhi Zhuge, Jiawen Zhu, Lu Zhang, Huchuan Lu

专题命中多模态评测：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted to IEEE Transactions on Multimedia (TMM)

2501.05444 2025-01-10 cs.CV 83%

Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark

Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, Yu Cheng

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2412.14056 2024-12-19 cs.CV cs.AI cs.CL cs.LG cs.MM 83%

A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future

Shilin Sun, Wenbin An, Feng Tian, Fang Nan, Qidong Liu, Jun Liu, Nazaraf Shah, Ping Chen

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments This work has been submitted to the IEEE for possible publication

2412.10594 2024-12-17 cs.CV cs.LG 83%

Towards Unified Benchmark and Models for Multi-Modal Perceptual Metrics

Sara Ghazanfari, Siddharth Garg, Nicolas Flammarion, Prashanth Krishnamurthy, Farshad Khorrami, Francesco Croce

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2410.21108 2024-12-11 cs.CV 83%

LiGAR: LiDAR-Guided Hierarchical Transformer for Multi-Modal Group Activity Recognition

Naga Venkata Sai Raviteja Chappa, Khoa Luu

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted at WACV 2025; 14 pages, 4 figures, 10 tables

2412.00955 2024-12-05 cs.CV 83%

WAFFLE: Multimodal Floorplan Understanding in the Wild

Keren Ganon, Morris Alper, Rachel Mikulinsky, Hadar Averbuch-Elor

专题命中多模态评测：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

Comments Accepted to WACV 2025. Project page: https://tau-vailab.github.io/WAFFLE/

2412.01249 2024-12-03 cs.CL 83%

Data Uncertainty-Aware Learning for Multimodal Aspect-based Sentiment Analysis

Hao Yang, Zhenyu Zhang, Yanyan Zhao, Bing Qin

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

2407.13596 2024-12-02 cs.CV 83%

EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing

Wei Zhang, Miaoxin Cai, Tong Zhang, Jun Li, Yin Zhuang, Xuerui Mao

专题命中多模态评测：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

2411.06735 2024-11-22 cs.AI 83%

Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data

Kai Kim, Howard Tsai, Rajat Sen, Abhimanyu Das, Zihao Zhou, Abhishek Tanpure, Mathew Luo, Rose Yu

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.AI

Comments 21 pages, 4 tables, 2 figures

2411.11360 2024-11-19 cs.CV 83%

CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset

Zhiming Wang, Mingze Wang, Sheng Xu, Yanjing Li, Baochang Zhang

专题命中多模态评测：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV

2410.24046 2024-11-01 eess.IV cs.CV cs.LG 83%

Deep Learning with HM-VGG: AI Strategies for Multi-modal Image Analysis

Junliang Du, Yiru Cang, Tong Zhou, Jiacheng Hu, Weijie He

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2405.05956 2024-10-29 cs.RO cs.CV 83%

Probing Multimodal LLMs as World Models for Driving

Shiva Sreeram, Tsun-Hsuan Wang, Alaa Maalouf, Guy Rosman, Sertac Karaman, Daniela Rus

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments https://github.com/sreeramsa/DriveSim https://www.youtube.com/watch?v=Fs8jgngOJzU

2209.14272 2024-10-28 cs.LG cs.CL cs.CV cs.MM cs.SD eess.AS 83%

Towards Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results

Lukas Christ, Shahin Amiriparian, Alexander Kathan, Niklas Müller, Andreas König, Björn W. Schuller

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments This work has been submitted to the IEEE for possible publication (Major Revision)

2410.13859 2024-10-18 cs.CV 83%

$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

Yaxin Luo, Gen Luo, Jiayi Ji, Yiyi Zhou, Xiaoshuai Sun, Zhiqiang Shen, Rongrong Ji

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2410.09962 2024-10-16 cs.CV 83%

LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models

Han Qiu, Jiaxing Huang, Peng Gao, Qin Qi, Xiaoqin Zhang, Ling Shao, Shijian Lu

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2410.11538 2024-10-16 cs.CV 83%

MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmark

Bin Shan, Xiang Fei, Wei Shi, An-Lan Wang, Guozhi Tang, Lei Liao, Jingqun Tang, Xiang Bai, Can Huang

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments 12 pages, 5 figures, project page: https://github.com/xfey/MCTBench?tab=readme-ov-file

2401.06153 2024-10-11 cs.NE cs.AI 83%

Multimodal Optimization with k-Cluster Big Bang-Big Crunch Algorithm and Postprocessing Methods for Identification and Quantification of Optima

Kemal Erdem Yenin, Reha Oguz Sayin, Kuzey Arar, Kadir Kaan Atalay, Fabio Stroppa

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.AI

Comments 18 pages

2409.20559 2024-10-01 cs.LG cs.CV 83%

Supervised Multi-Modal Fission Learning

Lingchao Mao, Qi wang, Yi Su, Fleming Lure, Jing Li

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2409.15125 2024-09-24 cs.CV 83%

Detect, Describe, Discriminate: Moving Beyond VQA for MLLM Evaluation

Manu Gaur, Darshan Singh S, Makarand Tapaswi

专题命中多模态评测：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments ECCV 2024 Workshop EVAL-FoMo; Project Page: https://katha-ai.github.io/projects/detect-describe-discriminate/

2409.13440 2024-09-23 eess.SP cs.AI cs.CR cs.LG 83%

Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG Representative Learning

Xiaowen Fu, Bingxin Wang, Xinzhou Guo, Guoqing Liu, Yang Xiang

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

2408.12682 2024-08-26 cs.LG cs.AI cs.CL cs.CV cs.MM 83%

MultiMed: Massively Multimodal and Multitask Medical Understanding

Shentong Mo, Paul Pu Liang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2408.12480 2024-08-26 cs.LG cs.CL 83%

Vintern-1B: An Efficient Multimodal Large Language Model for Vietnamese

Khang T. Doan, Bao G. Huynh, Dung T. Hoang, Thuc D. Pham, Nhat H. Pham, Quan T. M. Nguyen, Bang Q. Vo, Suong N. Hoang

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

2407.10474 2024-07-16 cs.MM 83%

Multi-source Knowledge Enhanced Graph Attention Networks for Multimodal Fact Verification

Han Cao, Lingwei Wei, Wei Zhou, Songlin Hu

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.MM

Comments Accepted by ICME 2024

2407.10125 2024-07-16 cs.CV 83%

When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset

Yi Zhang, Wang Zeng, Sheng Jin, Chen Qian, Ping Luo, Wentao Liu

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Accepted to ECCV'2024

2407.02946 2024-07-04 cs.CV 83%

3D Multimodal Image Registration for Plant Phenotyping

Eric Stumpe, Gernot Bodner, Francesco Flagiello, Matthias Zeppelzauer

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 53 pages, 13 Figures, preprint submitted to Computers and Electronics in Agriculture

2407.00743 2024-07-02 cs.MM cs.AI cs.CL eess.AS 83%

AIMDiT: Modality Augmentation and Interaction via Multimodal Dimension Transformation for Emotion Recognition in Conversations

Sheng Wu, Jiaxing Liu, Longbiao Wang, Dongxiao He, Xiaobao Wang, Jianwu Dang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

2406.16442 2024-07-02 cs.CV 83%

EmoLLM: Multimodal Emotional Understanding Meets Large Language Models

Qu Yang, Mang Ye, Bo Du

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments 9 pages

2406.18146 2024-07-01 cs.CV 83%

A Refer-and-Ground Multimodal Large Language Model for Biomedicine

Xiaoshuang Huang, Haifeng Huang, Lingdong Shen, Yehui Yang, Fangxin Shang, Junwei Liu, Jia Liu

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by MICCAI2024

2402.00357 2024-06-21 cs.CV 83%

Safety of Multimodal Large Language Models on Images and Texts

Xin Liu, Yichen Zhu, Yunshi Lan, Chao Yang, Yu Qiao

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted at IJCAI2024

2406.03793 2024-06-07 cs.LG cs.CV 83%

Low-Rank Similarity Mining for Multimodal Dataset Distillation

Yue Xu, Zhilin Lin, Yusong Qiu, Cewu Lu, Yong-Lu Li

专题命中多模态评测：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted at ICML 2024

1. 多模态评测 9087 篇

3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding

Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark

A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future

Towards Unified Benchmark and Models for Multi-Modal Perceptual Metrics

LiGAR: LiDAR-Guided Hierarchical Transformer for Multi-Modal Group Activity Recognition

WAFFLE: Multimodal Floorplan Understanding in the Wild

Data Uncertainty-Aware Learning for Multimodal Aspect-based Sentiment Analysis

EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing

Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data

CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset

Deep Learning with HM-VGG: AI Strategies for Multi-modal Image Analysis

Probing Multimodal LLMs as World Models for Driving

Towards Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results

$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models

MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmark

Multimodal Optimization with k-Cluster Big Bang-Big Crunch Algorithm and Postprocessing Methods for Identification and Quantification of Optima

Supervised Multi-Modal Fission Learning

Detect, Describe, Discriminate: Moving Beyond VQA for MLLM Evaluation

Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG Representative Learning

MultiMed: Massively Multimodal and Multitask Medical Understanding

Vintern-1B: An Efficient Multimodal Large Language Model for Vietnamese

Multi-source Knowledge Enhanced Graph Attention Networks for Multimodal Fact Verification

When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset

3D Multimodal Image Registration for Plant Phenotyping

AIMDiT: Modality Augmentation and Interaction via Multimodal Dimension Transformation for Emotion Recognition in Conversations

EmoLLM: Multimodal Emotional Understanding Meets Large Language Models

A Refer-and-Ground Multimodal Large Language Model for Biomedicine

Safety of Multimodal Large Language Models on Images and Texts

Low-Rank Similarity Mining for Multimodal Dataset Distillation