多模态大模型

GLaMM: Pixel Grounding Large Multimodal Model

Hanoona Rasheed, Muhammad Maaz, Sahal Shaji Mullappilly, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M. Anwer, Erix Xing, Ming-Hsuan Yang, Fahad S. Khan

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024

Concept Visualization: Explaining the CLIP Multi-modal Embedding Using WordNet

Loris Giulivi, Giacomo Boracchi

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication at IJCNN 2024

Understanding Multimodal Deep Neural Networks: A Concept Selection View

Chenming Shang, Hengyuan Zhang, Hao Wen, Yujiu Yang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

On Speculative Decoding for Multimodal Large Language Models

Mukul Gagrani, Raghavv Goel, Wonseok Jeon, Junyoung Park, Mingu Lee, Christopher Lott

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted as a spotlight paper to ELVM workshop at CVPR 2024

A Review of Multi-Modal Large Language and Vision Models

Kilian Carolan, Laura Fennelly, Alan F. Smeaton

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 33 pages, 1 figure

Open-Vocabulary Federated Learning with Multimodal Prototyping

Huimin Zeng, Zhenrui Yue, Dong Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at NAACL 2024

Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction

Inhwan Bae, Junoh Lee, Hae-Gon Jeon

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at CVPR 2024

m3P: Towards Multimodal Multilingual Translation with Multimodal Prompt

Jian Yang, Hongcheng Guo, Yuwei Yin, Jiaqi Bai, Bing Wang, Jiaheng Liu, Xinnian Liang, Linzheng Cahi, Liqun Yang, Zhoujun Li

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments COLING 2024

Multi-Modal Hallucination Control by Visual Information Grounding

Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, Stefano Soatto

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Journal ref IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024

Leveraging Chat-Based Large Vision Language Models for Multimodal Out-Of-Context Detection

Fatma Shalabi, Hichem Felouat, Huy H. Nguyen, Isao Echizen

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 13 pages, 6 figures , conference

Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation

Razvan-George Pasca, Alexey Gavryushin, Muhammad Hamza, Yen-Ling Kuo, Kaichun Mo, Luc Van Gool, Otmar Hilliges, Xi Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Models

Navid Rajabi, Jana Kosecka

专题命中 图文多模态 ：multi-modal(title)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to DMLR @ ICLR 2024

ITEm: Unsupervised Image-Text Embedding Learning for eCommerce

Baohao Liao, Michael Kozielski, Sanjika Hewavitharana, Jiangbo Yuan, Shahram Khadivi, Tomer Lancewicki

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network

Tiancheng Zhao, Peng Liu, Kyusong Lee

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Published at IET CV

Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models

Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, Yang Liu

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings

Daniel Rose, Vaishnavi Himakunthala, Andy Ouyang, Ryan He, Alex Mei, Yujie Lu, Michael Saxon, Chinmay Sonar, Diba Mirza, William Yang Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Zoom-shot: Fast and Efficient Unsupervised Zero-Shot Transfer of CLIP to Vision Encoders with Multimodal Loss

Jordan Shipard, Arnold Wiliem, Kien Nguyen Thanh, Wei Xiang, Clinton Fookes

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 15 pages

Multimodal Parameter-Efficient Few-Shot Class Incremental Learning

Marco D'Alessandro, Alberto Alonso, Enrique Calabrés, Mikel Galar

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Paris, France, 2023, pp. 3385-3395

Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training

Longtian Qiu, Shan Ning, Xuming He

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Comments AAAI 2024.Open sourced, Code and Model Available

UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts

Chenlu Zhan, Yufei Zhang, Yu Lin, Gaoang Wang, Hongwei Wang

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Structural Information Guided Multimodal Pre-training for Vehicle-centric Perception

Xiao Wang, Wentao Wu, Chenglong Li, Zhicheng Zhao, Zhe Chen, Yukai Shi, Jin Tang

专题命中 图文多模态 ：multimodal(title)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by AAAI-2024

Unlock the Power: Competitive Distillation for Multi-Modal Large Language Models

Xinwei Li, Li Lin, Shuai Wang, Chen Qian

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Multimodal Data Augmentation for Image Captioning using Diffusion Models

Changrong Xiao, Sean Xin Xu, Kunpeng Zhang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Socratis: Are large multimodal models emotionally aware?

Katherine Deng, Arijit Ray, Reuben Tan, Saadia Gabriel, Bryan A. Plummer, Kate Saenko

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023 WECIA

Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts

Deepanway Ghosal, Navonil Majumder, Roy Ka-Wei Lee, Rada Mihalcea, Soujanya Poria

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text

Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, Yejin Choi

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments NeurIPS D&B 2023. Project homepage: https://github.com/allenai/mmc4

Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality

Harman Singh, Pengchuan Zhang, Qifan Wang, Mengjiao Wang, Wenhan Xiong, Jingfei Du, Yu Chen

专题命中 图文多模态 ：image-text(title)；multimodal(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2023 (long paper, main conference)

Exploring Affordance and Situated Meaning in Image Captions: A Multimodal Analysis

Pin-Er Chen, Po-Ya Angela Wang, Hsin-Yu Chou, Yu-Hsiang Tseng, Shu-Kai Hsieh

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 10 pages, 9 figures

Leveraging Image-Text Similarity and Caption Modification for the DataComp Challenge: Filtering Track and BYOD Track

Shuhei Yokoo, Peifei Zhu, Yuchi Ishikawa, Mikihiro Tanaka, Masayoshi Kondo, Hirokatsu Kataoka

专题命中 图文多模态 ：image-text(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at the ICCV 2023 Workshop on Towards the Next Generation of Computer Vision Datasets: DataComp Track

Knowledge Extraction and Distillation from Large-Scale Image-Text Colonoscopy Records Leveraging Large Language and Vision Models

Shuo Wang, Yan Zhu, Xiaoyuan Luo, Zhiwei Yang, Yizhe Zhang, Peiyao Fu, Manning Wang, Zhijian Song, Quanlin Li, Pinghong Zhou, Yike Guo

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI