多模态大模型 - arXivDaily 专题

2211.02580 2022-11-07 cs.CL cs.AI cs.CV 82%

Evaluating and Improving Factuality in Multimodal Abstractive Summarization

David Wan, Mohit Bansal

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2022 (17 pages)

2202.11645 2022-10-19 stat.CO cs.LG eess.SP math.OC 82%

Cyclical Variational Bayes Monte Carlo for Efficient Multi-Modal Posterior Distributions Evaluation

Felipe Igea, Alice Cicirello

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)

Comments Accepted version in MSSP

Journal ref Mechanical Systems and Signal Processing, Volume 186, 2023, 109868, ISSN 0888-3270

2203.05297 2022-09-21 cs.CV cs.CL cs.GR cs.LG cs.MM 82%

BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis

Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, Bo Zheng

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments 28 pages, 15 figures, Accepted by ECCV2022

2209.06416 2022-09-15 cs.CL cs.AI cs.MM 82%

ImageArg: A Multi-modal Tweet Dataset for Image Persuasiveness Mining

Zhexiong Liu, Meiqi Guo, Yue Dai, Diane Litman

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

Comments In Argument Mining Workshop, held in conjunction with the International Conference on Computational Linguistics (COLING), October 2022

2207.14087 2022-09-14 cs.MM cs.CL cs.CV 82%

CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation

Hao Sun, Hongyi Wang, Jiaqing Liu, Yen-Wei Chen, Lanfen Lin

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Accepted by ACM MM 2022

2208.11066 2022-08-24 cs.NE 82%

Enhanced Opposition Differential Evolution Algorithm for Multimodal Optimization

Shatendra Singh, Aruna Tiwari

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)

2205.13445 2022-05-27 cs.CV cs.AI cs.CL cs.IT cs.LG math.IT 82%

Mutual Information Divergence: A Unified Metric for Multimodal Generative Models

Jin-Hwa Kim, Yunji Kim, Jiyoung Lee, Kang Min Yoo, Sang-Woo Lee

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2205.12617 2022-05-26 cs.CL cs.AI cs.CV 82%

DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation

Jingnong Qu, Liunian Harold Li, Jieyu Zhao, Sunipa Dev, Kai-Wei Chang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2204.11588 2022-04-26 cs.IR cs.AI cs.CL cs.CV cs.LG 82%

Ad Creative Discontinuation Prediction with Multi-Modal Multi-Task Neural Survival Networks

Shunsuke Kitada, Hitoshi Iyatomi, Yoshifumi Seki

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 23 pages, 5 figures. Accepted by Appl. Sci. on March 29th, 2022

Journal ref Appl. Sci. 2022, 12(7), 3594

2109.09792 2022-03-01 eess.SY cs.RO cs.SY 82%

Stochastic MPC with Multi-modal Predictions for Traffic Intersections

Siddharth H. Nair, Vijay Govindarajan, Theresa Lin, Chris Meissen, H. Eric Tseng, Francesco Borrelli

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)

Comments Extended version of ITSC 2022 submission

2202.03242 2022-02-08 cs.LG stat.ML 82%

Unsupervised physics-informed disentanglement of multimodal data for high-throughput scientific discovery

Nathaniel Trask, Carianne Martinez, Kookjin Lee, Brad Boyce

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)

2109.12307 2021-09-28 cs.CV cs.AI cs.MM 82%

Multi-Modal Multi-Instance Learning for Retinal Disease Recognition

Xirong Li, Yang Zhou, Jie Wang, Hailan Lin, Jianchun Zhao, Dayong Ding, Weihong Yu, Youxin Chen

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by ACM Multimedia 2021 (Main Track)

2109.11526 2021-09-24 cs.CV cs.CL cs.CY cs.LG cs.MM 82%

MARMOT: A Deep Learning Framework for Constructing Multimodal Representations for Vision-and-Language Tasks

Patrick Y. Wu, Walter R. Mebane

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments 57 pages, 16 figures. Forthcoming in Computational Communication Research

2109.02993 2021-09-08 cs.CV cs.MM cs.SD eess.AS eess.IV 82%

Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors

Hasam Khalid, Minha Kim, Shahroz Tariq, Simon S. Woo

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.MM、eess.AS

Comments 2 Figures, 2 Tables, Accepted for publication at the 1st Workshop on Synthetic Multimedia - Audiovisual Deepfake Generation and Detection (ADGD '21) at ACM MM 2021

2106.09889 2021-06-21 cs.CL cs.CV cs.MM 82%

GEM: A General Evaluation Benchmark for Multimodal Tasks

Lin Su, Nan Duan, Edward Cui, Lei Ji, Chenfei Wu, Huaishao Luo, Yongfei Liu, Ming Zhong, Taroon Bharti, Arun Sacheti

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Accepted by Findings of ACL 2021

2007.00229 2021-02-05 cs.CL cs.AI cs.CV 82%

Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation

Wanrong Zhu, Xin Eric Wang, Tsu-Jui Fu, An Yan, Pradyumna Narayana, Kazoo Sone, Sugato Basu, William Yang Wang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EACL 2021

2012.03678 2020-12-08 cs.CV cs.AI cs.CL cs.LG 82%

Generating Natural Questions from Images for Multimodal Assistants

Alkesh Patel, Akanksha Bindal, Hadas Kotek, Christopher Klein, Jason Williams

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 4 pages, 1 reference page, 5 figures, 4 tables

2011.11286 2020-11-24 cs.MM cs.AI cs.CV 82%

MEG: Multi-Evidence GNN for Multimodal Semantic Forensics

Ekraam Sabir, Ayush Jaiswal, Wael AbdAlmageed, Prem Natarajan

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.AI、cs.MM

Comments To be published at ICPR 2020

2010.06671 2020-10-15 cs.CL cs.AI cs.CV 82%

A Multi-Modal Method for Satire Detection using Textual and Visual Cues

Lily Li, Or Levi, Pedram Hosseini, David A. Broniatowski

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to the Third Workshop on NLP for Internet Freedom (NLP4IF): Censorship, Disinformation, and Propaganda. Co-located with COLING 2020

1909.06442 2019-11-11 q-bio.QM cs.LG stat.ML 82%

Co-Attentive Cross-Modal Deep Learning for Medical Evidence Synthesis and Decision Making

Devin Taylor, Simeon Spasov, Pietro Liò

专题命中多模态评测：cross-modal(title,abstract)；multimodal(abstract)

Comments 7 pages, 2 figures, Machine Learning for Health (ML4H) at NeurIPS 2019 - Extended Abstract, clarified graph and math notation, typos corrected

1805.04975 2019-04-18 q-bio.NC 82%

Multimodal Cross-registration and Quantification of Metric Distortions in Whole Brain Histology of Marmoset using Diffeomorphic Mappings

Brian C. Lee, Meng Kuan Lin, Yan Fu, Junichi Hata, Michael I. Miller, Partha P. Mitra

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)

1903.07478 2019-03-19 q-bio.QM 82%

Neurovascular coupling: insights from multi-modal dynamic causal modelling of fMRI and MEG

Amirhossein Jafarian, Vladimir Litvak, Hayriye Cagnan, Karl J. Friston, Peter Zeidman

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)

1808.06686 2018-08-22 cs.MM cs.AI cs.CV cs.LG cs.SI 82%

Deep Multimodal Image-Repurposing Detection

Ekraam Sabir, Wael AbdAlmageed, Yue Wu, Prem Natarajan

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.AI、cs.MM

Comments To be published at ACM Multimeda 2018 (orals)

1805.02356 2018-05-08 cs.CL cs.AI cs.IR cs.MA cs.MM 82%

Multimodal Machine Translation with Reinforcement Learning

Xin Qian, Ziyi Zhong, Jieli Zhou

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

1701.08251 2017-04-21 cs.CL cs.AI cs.CV 82%

Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation

Nasrin Mostafazadeh, Chris Brockett, Bill Dolan, Michel Galley, Jianfeng Gao, Georgios P. Spithourakis, Lucy Vanderwende

专题命中多模态评测：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

1704.04517 2017-04-18 cs.CL cs.AI cs.CV 82%

ShapeWorld - A new test methodology for multimodal language understanding

Alexander Kuhnle, Ann Copestake

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

1606.01847 2016-09-27 cs.CV cs.AI cs.CL 82%

Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding

Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, Marcus Rohrbach

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to EMNLP 2016

1608.02289 2016-08-09 cs.CV cs.CL cs.MM 82%

Detecting Sarcasm in Multimodal Social Platforms

Rossano Schifanella, Paloma de Juan, Joel Tetreault, Liangliang Cao

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments 10 pages, 3 figures, final version published in the Proceedings of ACM Multimedia 2016

2604.08140 2026-04-10 cs.CR cs.AI cs.MM cs.NI 82%

Multimodal Reasoning with LLM for Encrypted Traffic Interpretation: A Benchmark

基于LLM的多模态推理用于加密流量解释：一个基准

Longgang Zhang, Xiaowei Fu, Fuxiang Huang, Lei Zhang

机构 * School of Microelectronics and Communication Engineering, Chongqing University（重庆大学微电子与通信工程学院）； School of Data Science, Lingnan University（岭南大学数据科学学院）

专题命中多模态评测：multimodal(title,abstract)；分类 cs.AI、cs.MM

AI总结本文提出BGTD基准和mmTraffic框架，通过结合原始字节与结构化注释，实现可解释的加密流量解释，生成高保真的人可读报告，同时保持高分类准确率。

Comments Project page \url{https://github.com/lgzhangzlg/Multimodal-Reasoning-with-LLM-for-Encrypted-Traffic-Interpretation-A-Benchmark}

2509.23499 2026-03-12 cs.CV cs.CL cs.LG 82%

Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional

多模态数据光谱：多模态数据集是多维的

Divyam Madaan, Varshan Muhunthan, Kyunghyun Cho, Sumit Chopra

机构 * New York University（纽约大学）； GenentechCIFAR（基因泰克CIFAR）； New York University Grossman School of Medicine（纽约大学格罗斯曼医学院）

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV、cs.CL；multimodal(comments)

AI总结本文通过多模态大语言模型分析23个视觉问答基准，揭示了不同模态在目标任务中的依赖性差异，发现部分基准因设计缺陷放大了图像依赖性。

Comments Accepted to ICLR 2026. Code available at https://github.com/divyam3897/multimodal-spectrum

1. 多模态评测 9111 篇

Evaluating and Improving Factuality in Multimodal Abstractive Summarization

Cyclical Variational Bayes Monte Carlo for Efficient Multi-Modal Posterior Distributions Evaluation

BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis

ImageArg: A Multi-modal Tweet Dataset for Image Persuasiveness Mining

CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation

Enhanced Opposition Differential Evolution Algorithm for Multimodal Optimization

Mutual Information Divergence: A Unified Metric for Multimodal Generative Models

DisinfoMeme: A Multimodal Dataset for Detecting Meme Intentionally Spreading Out Disinformation

Ad Creative Discontinuation Prediction with Multi-Modal Multi-Task Neural Survival Networks

Stochastic MPC with Multi-modal Predictions for Traffic Intersections

Unsupervised physics-informed disentanglement of multimodal data for high-throughput scientific discovery

Multi-Modal Multi-Instance Learning for Retinal Disease Recognition

MARMOT: A Deep Learning Framework for Constructing Multimodal Representations for Vision-and-Language Tasks

Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors

GEM: A General Evaluation Benchmark for Multimodal Tasks

Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation

Generating Natural Questions from Images for Multimodal Assistants

MEG: Multi-Evidence GNN for Multimodal Semantic Forensics

A Multi-Modal Method for Satire Detection using Textual and Visual Cues

Co-Attentive Cross-Modal Deep Learning for Medical Evidence Synthesis and Decision Making

Multimodal Cross-registration and Quantification of Metric Distortions in Whole Brain Histology of Marmoset using Diffeomorphic Mappings

Neurovascular coupling: insights from multi-modal dynamic causal modelling of fMRI and MEG

Deep Multimodal Image-Repurposing Detection

Multimodal Machine Translation with Reinforcement Learning

Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation

ShapeWorld - A new test methodology for multimodal language understanding

Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding

Detecting Sarcasm in Multimodal Social Platforms

Multimodal Reasoning with LLM for Encrypted Traffic Interpretation: A Benchmark

Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional