多模态大模型

VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, Daniel Fried

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ACL 2024. 24 pages. Project page: https://jykoh.com/vwa

UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark

Zhaokun Zhou, Qiulin Wang, Bin Lin, Yiwei Su, Rui Chen, Xin Tao, Amin Zheng, Li Yuan, Pengfei Wan, Di Zhang

专题命中 多模态评测 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

TextHawk: Exploring Efficient Fine-Grained Perception of Multimodal Large Language Models

Ya-Qi Yu, Minghui Liao, Jihao Wu, Yongxin Liao, Xiaoyu Zheng, Wei Zeng

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction

Boyi Li, Rodolfo Corona, Karttikeya Mangalam, Catherine Chen, Daniel Flaherty, Serge Belongie, Kilian Q. Weinberger, Jitendra Malik, Trevor Darrell, Dan Klein

专题命中 多模态评测 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CL、cs.AI

Comments NAACL Findings 2024

Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs

Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, Zhe Gan

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization

Renjie Pi, Tianyang Han, Wei Xiong, Jipeng Zhang, Runtao Liu, Rui Pan, Tong Zhang

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

DialogCC: An Automated Pipeline for Creating High-Quality Multi-Modal Dialogue Dataset

Young-Jun Lee, Byungsoo Ko, Han-Gyu Kim, Jonghwan Hyeon, Ho-Jin Choi

专题命中 多模态评测 ：multi-modal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments NAACL 2024

M3FAS: An Accurate and Robust MultiModal Mobile Face Anti-Spoofing System

Chenqi Kong, Kexin Zheng, Yibing Liu, Shiqi Wang, Anderson Rocha, Haoliang Li

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning

Congyun Jin, Ming Zhang, Xiaowei Ma, Li Yujiao, Yingbo Wang, Yabo Jia, Yuliang Du, Tao Sun, Haowen Wang, Cong Fan, Jinjie Gu, Chenfei Chi, Xiangguo Lv, Fangzhou Li, Wei Xue, Yiran Huang

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；image-text(abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 13 figures

BIKED++: A Multimodal Dataset of 1.4 Million Bicycle Image and Parametric CAD Designs

Lyle Regenwetter, Yazan Abu Obaideh, Amin Heyrani Nobari, Faez Ahmed

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMs

Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, Filip Ilievski

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments 20 pages, 12 figures, 7 tables

SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark

Zhenwen Liang, Kehan Guo, Gang Liu, Taicheng Guo, Yujun Zhou, Tianyu Yang, Jiajun Jiao, Renjie Pi, Jipeng Zhang, Xiangliang Zhang

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments Work in progress

AiGen-FoodReview: A Multimodal Dataset of Machine-Generated Restaurant Reviews and Images on Social Media

Alessandro Gambetti, Qiwei Han

专题命中 多模态评测 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Model

Anwen Hu, Yaya Shi, Haiyang Xu, Jiabo Ye, Qinghao Ye, Ming Yan, Chenliang Li, Qi Qian, Ji Zhang, Fei Huang

专题命中 多模态评测 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CL、cs.MM

Comments 20 pages, 12 figures

Multi-modal Large Language Model Enhanced Pseudo 3D Perception Framework for Visual Commonsense Reasoning

Jian Zhu, Hanli Wang, Miaojing Shi

专题命中 多模态评测 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.MM

A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity

Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, Pascale Fung

专题命中 多模态评测 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CL、cs.AI

Comments 45 pages, AACL 2023

CCMB: A Large-scale Chinese Cross-modal Benchmark

Chunyu Xie, Heng Cai, Jincheng Li, Fanjing Kong, Xiaoyu Wu, Jianfei Song, Henrique Morimitsu, Lin Yao, Dexin Wang, Xiangzheng Zhang, Dawei Leng, Baochang Zhang, Xiangyang Ji, Yafeng Deng

专题命中 多模态评测 ：cross-modal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Journal ref Proceedings of the 31st ACM International Conference on Multimedia (ACM MM), 2023, Pages 4219-4227

Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs

Yuxin Zuo, Bei Li, Chuanhao Lv, Tong Zheng, Tong Xiao, Jingbo Zhu

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Comments Findings of EMNLP2023

VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias

Stefanos-Iordanis Papadopoulos, Christos Koutlis, Symeon Papadopoulos, Panagiotis C. Petrantonakis

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.MM

CCOM-HuQin: an Annotated Multimodal Chinese Fiddle Performance Dataset

Yu Zhang, Ziya Zhou, Xiaobing Li, Feng Yu, Maosong Sun

专题命中 多模态评测 ：multimodal(title,abstract)；audio-visual(abstract)；分类 cs.MM、eess.AS

Comments 15 pages, 11 figures

Journal ref Transactions of the International Society for Music Information Retrieval, 2023, 6(1), 60-74

Blind Multimodal Quality Assessment of Low-light Images

Miaohui Wang, Zhuowei Xu, Mai Xu, Weisi Lin

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.MM

Comments 15 pages

WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models

Conghui He, Zhenjiang Jin, Chao Xu, Jiantao Qiu, Bin Wang, Wei Li, Hang Yan, Jiaqi Wang, Dahua Lin

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Technical Report

Enhancing Subtask Performance of Multi-modal Large Language Model

Yongqiang Zhao, Zhenyu Li, Feng Zhang, Xinhai Xu, Donghong Liu

专题命中 多模态评测 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Link-Context Learning for Multimodal LLMs

Yan Tai, Weichen Fan, Zhao Zhang, Feng Zhu, Rui Zhao, Ziwei Liu

专题命中 多模态评测 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments 10 pages, 8 figures

Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling

Shengqiong Wu, Hao Fei, Yixin Cao, Lidong Bing, Tat-Seng Chua

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments ACL 2023

UAMD-Net: A Unified Adaptive Multimodal Neural Network for Dense Depth Completion

Guancheng Chen, Junli Lin, Huabiao Qin

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 4 figures

Journal ref IEEE Transactions on Circuits and Systems for Video Technology, 2023

WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning

Krishna Srinivasan, Karthik Raman, Jiecao Chen, Michael Bendersky, Marc Najork

专题命中 多模态评测 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.CL

ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning

Yuxin Deng, Jiayi Ma

专题命中 多模态评测 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Weakly Supervised Annotations for Multi-modal Greeting Cards Dataset

Sidra Hanif, Longin Jan Latecki

专题命中 多模态评测 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted for poster presentation at Pretrain@WACV 2023

Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training

Zhihong Chen, Yuhao Du, Jinpeng Hu, Yang Liu, Guanbin Li, Xiang Wan, Tsung-Hui Chang

专题命中 多模态评测 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Natural Language Processing. 11 pages, 3 figures