多模态大模型 - arXivDaily 专题

2402.08327 2024-06-06 cs.CL 83%

PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers

Weizhe Lin, Jingbiao Mei, Jinghong Chen, Bill Byrne

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CL

Comments ACL 2024; Project page: https://preflmr.github.io/

2311.17005 2024-05-24 cs.CV 83%

MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, Yu Qiao

专题命中多模态评测：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments CVPR 2024 highlight: updated version with Mistral and better performances for MVBench/NExT-QA/STAR/TVQA/EgoSchema/IntentQA

2404.14949 2024-05-21 cs.CV 83%

Multi-Modal Prompt Learning on Blind Image Quality Assessment

Wensheng Pan, Timin Gao, Yan Zhang, Runze Hu, Xiawu Zheng, Enwei Zhang, Yuting Gao, Yutao Liu, Yunhang Shen, Ke Li, Shengchuan Zhang, Liujuan Cao, Rongrong Ji

专题命中多模态评测：multi-modal(title,abstract)；image-text(abstract)；分类 cs.CV

2405.06483 2024-05-13 cs.CL 83%

LyS at SemEval-2024 Task 3: An Early Prototype for End-to-End Multimodal Emotion Linking as Graph-Based Parsing

Ana Ezquerro, David Vilares

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CL

Comments Accepted at SemEval 2024

2404.16821 2024-05-01 cs.CV 83%

How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites

Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, Ji Ma, Jiaqi Wang, Xiaoyi Dong, Hang Yan, Hewei Guo, Conghui He, Botian Shi, Zhenjiang Jin, Chao Xu, Bin Wang, Xingjian Wei, Wei Li, Wenjian Zhang, Bo Zhang, Pinlong Cai, Licheng Wen, Xiangchao Yan, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, Wenhai Wang

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Technical report

2404.16790 2024-04-26 cs.CV 83%

SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension

Bohao Li, Yuying Ge, Yi Chen, Yixiao Ge, Ruimao Zhang, Ying Shan

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2404.16006 2024-04-25 cs.CV 83%

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

Kaining Ying, Fanqing Meng, Jin Wang, Zhiqian Li, Han Lin, Yue Yang, Hao Zhang, Wenbo Zhang, Yuqi Lin, Shuo Liu, Jiayi Lei, Quanfeng Lu, Runjian Chen, Peng Xu, Renrui Zhang, Haozhe Zhang, Peng Gao, Yali Wang, Yu Qiao, Ping Luo, Kaipeng Zhang, Wenqi Shao

专题命中多模态评测：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

Comments 77 pages, 41 figures

2310.08475 2024-04-19 cs.CL cs.AI cs.CV cs.LG cs.MM 83%

Can We Edit Multimodal Large Language Models?

Siyuan Cheng, Bozhong Tian, Qingbin Liu, Xi Chen, Yongheng Wang, Huajun Chen, Ningyu Zhang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2023. Add the Exact Match/Accuracy results of Reliability and T-Generality

2109.09824 2024-03-28 cs.CV cs.LG 83%

Well Googled is Half Done: Multimodal Forecasting of New Fashion Product Sales with Image-based Google Trends

Geri Skenderi, Christian Joppi, Matteo Denitto, Marco Cristani

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by the Wiley Journal of Forecasting

2403.17830 2024-03-27 cs.CV 83%

Assessment of Multimodal Large Language Models in Alignment with Human Values

Zhelun Shi, Zhipin Wang, Hongxing Fan, Zaibin Zhang, Lijun Li, Yongting Zhang, Zhenfei Yin, Lu Sheng, Yu Qiao, Jing Shao

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2311.02692

2402.18107 2024-03-26 cs.MM 83%

Multimodal Interaction Modeling via Self-Supervised Multi-Task Learning for Review Helpfulness Prediction

HongLin Gong, Mengzhao Jia, Liqiang Jing

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.MM

Comments 10 pages,4 figures, 4 tables

2402.18169 2024-03-01 cs.CL 83%

MIKO: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discovery

Feihong Lu, Weiqi Wang, Yangyifei Luo, Ziqin Zhu, Qingyun Sun, Baixuan Xu, Haochen Shi, Shiqi Gao, Qian Li, Yangqiu Song, Jianxin Li

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

Comments 11 pages, 5 figures

2402.00137 2024-02-02 cs.LG cs.CV 83%

Multimodal Neurodegenerative Disease Subtyping Explained by ChatGPT

Diego Machado Reyes, Hanqing Chao, Juergen Hahn, Li Shen, Pingkun Yan

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2401.12032 2024-01-23 cs.HC cs.AI 83%

MINT: A wrapper to make multi-modal and multi-image AI models interactive

Jan Freyberg, Abhijit Guha Roy, Terry Spitz, Beverly Freeman, Mike Schaekermann, Patricia Strachan, Eva Schnider, Renee Wong, Dale R Webster, Alan Karthikesalingam, Yun Liu, Krishnamurthy Dvijotham, Umesh Telang

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.AI

Comments 15 pages, 7 figures

2401.06777 2024-01-17 eess.IV cs.AI 83%

Multimodal Neuroimaging Attention-Based architecture for Cognitive Decline Prediction

Jamie Vo, Naeha Sharif, Ghulam Mubashar Hassan

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

2312.14135 2023-12-27 cs.CV 83%

V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs

Penghao Wu, Saining Xie

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Project page with code: https://vstar-seal.github.io/

2312.10763 2023-12-19 cs.CV 83%

M3DBench: Let's Instruct Large Models with Multi-modal 3D Prompts

Mingsheng Li, Xin Chen, Chi Zhang, Sijin Chen, Hongyuan Zhu, Fukun Yin, Gang Yu, Tao Chen

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2312.08592 2023-12-15 cs.CV 83%

Dietary Assessment with Multimodal ChatGPT: A Systematic Analysis

Frank P. -W. Lo, Jianing Qiu, Zeyu Wang, Junhong Chen, Bo Xiao, Wu Yuan, Stamatia Giannarou, Gary Frost, Benny Lo

专题命中多模态评测：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

Comments 10 pages

2311.11567 2023-12-06 cs.CV 83%

InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models

Xiaotian Han, Quanzeng You, Yongfei Liu, Wentao Chen, Huangjie Zheng, Khalil Mrini, Xudong Lin, Yiqi Wang, Bohan Zhai, Jianbo Yuan, Heng Wang, Hongxia Yang

专题命中多模态评测：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

2311.01066 2023-11-28 eess.IV cs.CV 83%

Dynamic Multimodal Information Bottleneck for Multimodality Classification

Yingying Fang, Shuang Wu, Sheng Zhang, Chaoyan Huang, Tieyong Zeng, Xiaodan Xing, Simon Walsh, Guang Yang

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments WACV 2024

2311.11777 2023-11-21 cs.CV cs.LG eess.IV 83%

Multimodal deep learning for mapping forest dominant height by fusing GEDI with earth observation data

Man Chen, Wenquan Dong, Hao Yu, Iain Woodhouse, Casey M. Ryan, Haoyu Liu, Selena Georgiou, Edward T. A. Mitchard

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2311.10125 2023-11-20 cs.CV 83%

UnifiedVisionGPT: Streamlining Vision-Oriented AI through Generalized Multimodal Framework

Chris Kelly, Luhui Hu, Cindy Yang, Yu Tian, Deshun Yang, Bang Yang, Zaoshan Huang, Zihao Li, Yuexian Zou

专题命中多模态评测：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

Comments 9 pages, 29 figures

2311.02692 2023-11-07 cs.CV 83%

ChEF: A Comprehensive Evaluation Framework for Standardized Assessment of Multimodal Large Language Models

Zhelun Shi, Zhipin Wang, Hongxing Fan, Zhenfei Yin, Lu Sheng, Yu Qiao, Jing Shao

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 39 pages, 26 figures

2306.06687 2023-11-07 cs.CV 83%

LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark

Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Lu Sheng, Lei Bai, Xiaoshui Huang, Zhiyong Wang, Jing Shao, Wanli Ouyang

专题命中多模态评测：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments NeurIPS2023 camera ready ; 37 pages, 33 figures. Code available at https://github.com/OpenLAMM/LAMM ; Project page: https://openlamm.github.io/

2310.07749 2023-11-07 cs.CV 83%

OpenLEAF: Open-Domain Interleaved Image-Text Generation and Evaluation

Jie An, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Lijuan Wang, Jiebo Luo

专题命中多模态评测：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV

2211.05895 2023-10-24 cs.CV cs.AI cs.CL cs.HC cs.MM 83%

Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense

Zhecan Wang, Haoxuan You, Yicheng He, Wenhao Li, Kai-Wei Chang, Shih-Fu Chang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to EMNLP 2022 Long Paper

Journal ref EMNLP 2022

2302.03861 2023-10-16 eess.IV cs.CV 83%

SwinCross: Cross-modal Swin Transformer for Head-and-Neck Tumor Segmentation in PET/CT Images

Gary Y. Li, Junyu Chen, Se-In Jang, Kuang Gong, Quanzheng Li

专题命中多模态评测：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments 9 pages, 3 figures. Med Phys. 2023

2304.04187 2023-10-13 cs.CL 83%

Similarity-Aware Multimodal Prompt Learning for Fake News Detection

Ye Jiang, Xiaomin Yu, Yimin Wang, Xiaoman Xu, Xingyi Song, Diana Maynard

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

2309.09667 2023-09-19 cs.CV 83%

Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation

Huan Liu, Zichang Tan, Qiang Chen, Yunchao Wei, Yao Zhao, Jingdong Wang

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2309.07297 2023-09-15 cs.CV 83%

Multi-Modal Hybrid Learning and Sequential Training for RGB-T Saliency Detection

Guangyu Ren, Jitesh Joshi, Youngjun Cho

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 8 Pages main text, 3 pages supplementary information, 12 figures

1. 多模态评测 9087 篇

PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers

MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

Multi-Modal Prompt Learning on Blind Image Quality Assessment

LyS at SemEval-2024 Task 3: An Early Prototype for End-to-End Multimodal Emotion Linking as Graph-Based Parsing

How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites

SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

Can We Edit Multimodal Large Language Models?

Well Googled is Half Done: Multimodal Forecasting of New Fashion Product Sales with Image-based Google Trends

Assessment of Multimodal Large Language Models in Alignment with Human Values

Multimodal Interaction Modeling via Self-Supervised Multi-Task Learning for Review Helpfulness Prediction

MIKO: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discovery

Multimodal Neurodegenerative Disease Subtyping Explained by ChatGPT

MINT: A wrapper to make multi-modal and multi-image AI models interactive

Multimodal Neuroimaging Attention-Based architecture for Cognitive Decline Prediction

V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs

M3DBench: Let's Instruct Large Models with Multi-modal 3D Prompts

Dietary Assessment with Multimodal ChatGPT: A Systematic Analysis

InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models

Dynamic Multimodal Information Bottleneck for Multimodality Classification

Multimodal deep learning for mapping forest dominant height by fusing GEDI with earth observation data

UnifiedVisionGPT: Streamlining Vision-Oriented AI through Generalized Multimodal Framework

ChEF: A Comprehensive Evaluation Framework for Standardized Assessment of Multimodal Large Language Models

LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark

OpenLEAF: Open-Domain Interleaved Image-Text Generation and Evaluation

Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense

SwinCross: Cross-modal Swin Transformer for Head-and-Neck Tumor Segmentation in PET/CT Images

Similarity-Aware Multimodal Prompt Learning for Fake News Detection

Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation

Multi-Modal Hybrid Learning and Sequential Training for RGB-T Saliency Detection