多模态大模型 - arXivDaily 专题

2309.09667 2023-09-19 cs.CV 83%

Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation

Huan Liu, Zichang Tan, Qiang Chen, Yunchao Wei, Yao Zhao, Jingdong Wang

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2309.07297 2023-09-15 cs.CV 83%

Multi-Modal Hybrid Learning and Sequential Training for RGB-T Saliency Detection

Guangyu Ren, Jitesh Joshi, Youngjun Cho

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 8 Pages main text, 3 pages supplementary information, 12 figures

2309.05573 2023-09-12 cs.CV 83%

UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase

Youquan Liu, Runnan Chen, Xin Li, Lingdong Kong, Yuchen Yang, Zhaoyang Xia, Yeqi Bai, Xinge Zhu, Yuexin Ma, Yikang Li, Yu Qiao, Yuenan Hou

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments ICCV 2023; 21 pages; 9 figures; 18 tables; Code at https://github.com/PJLab-ADG/PCSeg

2308.11880 2023-08-24 cs.CV cs.LG 83%

SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targets

Cody Simons, Dripta S. Raychaudhuri, Sk Miraj Ahmed, Suya You, Konstantinos Karydis, Amit K. Roy-Chowdhury

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 12 pages, 5 figures, 9 tables, ICCV 2023

2308.06207 2023-08-14 cs.CL 83%

Thinking Like an Expert:Multimodal Hypergraph-of-Thought (HoT) Reasoning to boost Foundation Modals

Fanglong Yao, Changyuan Tian, Jintao Liu, Zequn Zhang, Qing Liu, Li Jin, Shuchao Li, Xiaoyu Li, Xian Sun

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

2305.05880 2023-08-08 cs.MM 83%

ChinaOpen: A Dataset for Open-world Multimodal Learning

Aozhu Chen, Ziyuan Wang, Chengbo Dong, Kaibin Tian, Ruixiang Zhao, Xun Liang, Zhanhui Kang, Xirong Li

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.MM

Comments Accepted by ACMMM 2023

2307.14878 2023-07-28 cs.CL 83%

MESED: A Multi-modal Entity Set Expansion Dataset with Fine-grained Semantic Classes and Hard Negative Entities

Yangning Li, Tingwei Lu, Yinghui Li, Tianyu Yu, Shulin Huang, Hai-Tao Zheng, Rui Zhang, Jun Yuan

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CL

2305.13659 2023-07-20 cs.CV 83%

Flare-Aware Cross-modal Enhancement Network for Multi-spectral Vehicle Re-identification

Aihua Zheng, Zhiqi Ma, Zi Wang, Chenglong Li

专题命中多模态评测：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2307.04532 2023-07-11 cs.CV cs.AI cs.CL eess.AS 83%

Read, Look or Listen? What's Needed for Solving a Multimodal Dataset

Netta Madvil, Yonatan Bitton, Roy Schwartz

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2306.16413 2023-06-29 cs.LG cs.AI cs.CL cs.CV cs.MM 83%

MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning

Paul Pu Liang, Yiwei Lyu, Xiang Fan, Arav Agarwal, Yun Cheng, Louis-Philippe Morency, Ruslan Salakhutdinov

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments JMLR Open Source Software 2023, Code available at https://github.com/pliang279/MultiBench

2305.18212 2023-05-30 cs.IR cs.AI cs.CL cs.CV cs.LG cs.MM 83%

Multimodal Recommendation Dialog with Subjective Preference: A New Challenge and Benchmark

Yuxing Long, Binyuan Hui, Caixia Yuan1, Fei Huang, Yongbin Li, Xiaojie Wang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ACL 2023

2305.03347 2023-05-08 cs.CV 83%

A Large Cross-Modal Video Retrieval Dataset with Reading Comprehension

Weijia Wu, Yuzhong Zhao, Zhuang Li, Jiahong Li, Hong Zhou, Mike Zheng Shou, Xiang Bai

专题命中多模态评测：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2304.14942 2023-05-01 cs.CV 83%

The Emotions of the Crowd: Learning Image Sentiment from Tweets via Cross-modal Distillation

Alessio Serra, Fabio Carrara, Maurizio Tesconi, Fabrizio Falchi

专题命中多模态评测：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2209.00244 2023-04-25 cs.CV eess.IV 83%

MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment

Zicheng Zhang, Wei Sun, Xiongkuo Min, Quan Zhou, Jun He, Qiyuan Wang, Guangtao Zhai

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2209.04355 2023-02-09 cs.AI 83%

MIntRec: A New Dataset for Multimodal Intent Recognition

Hanlei Zhang, Hua Xu, Xin Wang, Qianrui Zhou, Shaojie Zhao, Jiayan Teng

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

Comments Accepted by ACM MM 2022 (Main Track, Long Paper)

2211.05719 2022-12-22 cs.CL cs.AI cs.CV cs.LG cs.MM 83%

MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation

Jiazhan Feng, Qingfeng Sun, Can Xu, Pu Zhao, Yaming Yang, Chongyang Tao, Dongyan Zhao, Qingwei Lin

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2201.10113 2022-11-01 cs.CL cs.LG 83%

Multimodal data matters: language model pre-training over structured and unstructured electronic health records

Sicen Liu, Xiaolong Wang, Yongshuai Hou, Ge Li, Hui Wang, Hui Xu, Yang Xiang, Buzhou Tang

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

Comments 12 pages, 5 figures accepted for publication in the IEEE Journal of Biomedical and Health Informatics (J-BHI)

2209.09513 2022-10-18 cs.CL cs.AI cs.CV cs.LG cs.MM 83%

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, Ashwin Kalyan

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to NeurIPS 2022. 22 pages, 17 figures, 9 tables. Project: https://scienceqa.github.io

2206.08826 2022-09-27 cs.LG cs.CV eess.IV 83%

Multimodal Attention-based Deep Learning for Alzheimer's Disease Diagnosis

Michal Golovanevsky, Carsten Eickhoff, Ritambhara Singh

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 11 pages, 5 figures

Journal ref Journal of the American Medical Informatics Association, 2022; ocac168

2208.08080 2022-08-18 cs.AI cs.CL cs.CV cs.LG cs.MM 83%

Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides

Dong Won Lee, Chaitanya Ahuja, Paul Pu Liang, Sanika Natu, Louis-Philippe Morency

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 9 pages, 5 figures

2104.10412 2022-08-16 cs.CV 83%

Comprehensive Multi-Modal Interactions for Referring Image Segmentation

Kanishk Jain, Vineet Gandhi

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Findings of ACL 2022

Journal ref 2022.findings-acl.270

2207.10316 2022-07-22 cs.CV 83%

AutoAlignV2: Deformable Feature Aggregation for Dynamic Multi-Modal 3D Object Detection

Zehui Chen, Zhenyu Li, Shiquan Zhang, Liangji Fang, Qinhong Jiang, Feng Zhao

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted to ECCV 2022

2204.03804 2022-06-30 eess.IV cs.CV cs.LG math.OC 83%

A Learnable Variational Model for Joint Multimodal MRI Reconstruction and Synthesis

Wanyu Bian, Qingchao Zhang, Xiaojing Ye, Yunmei Chen

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Provisional Accepted by MICCAI2022

2206.13969 2022-06-29 cs.CL 83%

MACSA: A Multimodal Aspect-Category Sentiment Analysis Dataset with Multimodal Fine-grained Aligned Annotations

Hao Yang, Yanyan Zhao, Jianwei Liu, Yang Wu, Bing Qin

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

2205.08811 2022-05-19 cs.CV 83%

PhoCaL: A Multi-Modal Dataset for Category-Level Object Pose Estimation with Photometrically Challenging Objects

Pengyuan Wang, HyunJun Jung, Yitong Li, Siyuan Shen, Rahul Parthasarathy Srikanth, Lorenzo Garattoni, Sven Meier, Nassir Navab, Benjamin Busam

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments 11 pages

2205.05974 2022-05-13 cs.CL 83%

A Computational Acquisition Model for Multimodal Word Categorization

Uri Berger, Gabriel Stanovsky, Omri Abend, Lea Frermann

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

Comments Accepted to NAACL 2022

2205.01989 2022-05-05 cs.CL cs.AI cs.CV cs.MM cs.SI 83%

MM-Claims: A Dataset for Multimodal Claim Detection in Social Media

Gullal S. Cheema, Sherzod Hakimov, Abdul Sittar, Eric Müller-Budack, Christian Otto, Ralph Ewerth

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to Findings of NAACL 2022

2204.01678 2022-04-05 cs.CV cs.LG 83%

MultiMAE: Multi-modal Multi-task Masked Autoencoders

Roman Bachmann, David Mizrahi, Andrei Atanov, Amir Zamir

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Project page at https://multimae.epfl.ch

2105.07122 2022-03-18 cs.CL 83%

Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues

Qingxiu Dong, Ziwei Qin, Heming Xia, Tian Feng, Shoujie Tong, Haoran Meng, Lin Xu, Weidong Zhan, Sujian Li, Zhongyu Wei, Tianyu Liu, Zuifang Sui

专题命中多模态评测：multimodal(title)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CL

Comments ACL 2022 Main conference (Long Paper)

2105.14517 2022-01-12 cs.AI 83%

GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning

Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P. Xing, Liang Lin

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

Comments Accepted to Findings of ACL 2021

1. 多模态评测 9111 篇

Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation

Multi-Modal Hybrid Learning and Sequential Training for RGB-T Saliency Detection

UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase

SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targets

Thinking Like an Expert:Multimodal Hypergraph-of-Thought (HoT) Reasoning to boost Foundation Modals

ChinaOpen: A Dataset for Open-world Multimodal Learning

MESED: A Multi-modal Entity Set Expansion Dataset with Fine-grained Semantic Classes and Hard Negative Entities

Flare-Aware Cross-modal Enhancement Network for Multi-spectral Vehicle Re-identification

Read, Look or Listen? What's Needed for Solving a Multimodal Dataset

MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning

Multimodal Recommendation Dialog with Subjective Preference: A New Challenge and Benchmark

A Large Cross-Modal Video Retrieval Dataset with Reading Comprehension

The Emotions of the Crowd: Learning Image Sentiment from Tweets via Cross-modal Distillation

MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment

MIntRec: A New Dataset for Multimodal Intent Recognition

MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation

Multimodal data matters: language model pre-training over structured and unstructured electronic health records

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

Multimodal Attention-based Deep Learning for Alzheimer's Disease Diagnosis

Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides

Comprehensive Multi-Modal Interactions for Referring Image Segmentation

AutoAlignV2: Deformable Feature Aggregation for Dynamic Multi-Modal 3D Object Detection

A Learnable Variational Model for Joint Multimodal MRI Reconstruction and Synthesis

MACSA: A Multimodal Aspect-Category Sentiment Analysis Dataset with Multimodal Fine-grained Aligned Annotations

PhoCaL: A Multi-Modal Dataset for Category-Level Object Pose Estimation with Photometrically Challenging Objects

A Computational Acquisition Model for Multimodal Word Categorization

MM-Claims: A Dataset for Multimodal Claim Detection in Social Media

MultiMAE: Multi-modal Multi-task Masked Autoencoders

Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues

GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning