多模态大模型

IntCoOp: Interpretability-Aware Vision-Language Prompt Tuning

Soumya Suvra Ghosal, Samyadeep Basu, Soheil Feizi, Dinesh Manocha

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMs

Gregor Geigle, Abhay Jain, Radu Timofte, Goran Glavaš

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments ALVR Workshop 2024

PUMGPT: A Large Vision-Language Model for Product Understanding

Wei Xue, Zongyi Guo, Baoliang Cui, Zheng Xing, Xiaoyi Zeng, Xiufei Wang, Shuhui Wu, Weiming Lu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

From Pixels to Prose: A Large Dataset of Dense Image Captions

Vasu Singla, Kaiyu Yue, Sukriti Paul, Reza Shirkavand, Mayuka Jayawardhana, Alireza Ganjdanesh, Heng Huang, Abhinav Bhatele, Gowthami Somepalli, Tom Goldstein

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments pixelprose 16M dataset

Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models

Jiayun Luo, Siddhesh Khandelwal, Leonid Sigal, Boyang Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2024; Earlier version of this paper contained an unintentional error stemming from a bug in the code. This version corrects this error, which had to do with filtering of class names. In consultation with CVPR Program Chairs it was suggested errata be submitted as the updated (fixed) code reinforced original findings (albeit with slightly different final numbers)

MAFA: Managing False Negatives for Vision-Language Pre-training

Jaeseok Byun, Dohoon Kim, Taesup Moon

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024 camera ready version

Open-Vocabulary Segmentation with Unpaired Mask-Text Supervision

Zhaoqing Wang, Xiaobo Xia, Ziye Chen, Xiao He, Yandong Guo, Mingming Gong, Tongliang Liu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 27 pages, 18 figures, 10 tables

HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding

Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, Jiawei Zhou

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICML camera-ready version. Code is released at https://github.com/BillChan226/HALC

CountCLIP -- [Re] Teaching CLIP to Count to Ten

Harshvardhan Mestha, Tejas Agrawal, Karan Bania, Shreyas V, Yash Bhisikar

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Uncovering Bias in Large Vision-Language Models with Counterfactuals

Phillip Howard, Anahita Bhiwandiwalla, Kathleen C. Fraser, Svetlana Kiritchenko

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to the CVPR 2024 Responsible Generative AI (ReGenAI) Workshop

Holistic Visual-Textual Sentiment Analysis with Prior Models

Junyu Chen, Jie An, Hanjia Lyu, Christopher Kanan, Jiebo Luo

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Published in MIPR 2024

Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models

Christian Schlarmann, Naman Deep Singh, Francesco Croce, Matthias Hein

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICML 2024 Oral

TopViewRS: Vision-Language Models as Top-View Spatial Reasoners

Chengzu Li, Caiqi Zhang, Han Zhou, Nigel Collier, Anna Korhonen, Ivan Vulić

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 9 pages, 3 figures, 3 tables (21 pages, 4 figures, 15 tables including references and appendices)

Selectively Answering Visual Questions

Julian Martin Eisenschlos, Hernán Maina, Guido Ivetta, Luciana Benotti

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments To be published in the findings of the 2024 Annual Meeting of the Association for Computational Linguistics

OLIVE: Object Level In-Context Visual Embeddings

Timothy Ossowski, Junjie Hu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments ACL 2024

WAVER: Writing-style Agnostic Text-Video Retrieval via Distilling Vision-Language Models Through Open-Vocabulary Knowledge

Huy Le, Tung Kieu, Anh Nguyen, Ngan Le

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICASSP 2024

InsightSee: Advancing Multi-agent Vision-Language Models for Enhanced Visual Understanding

Huaxiang Zhang, Yaojia Mu, Guo-Niu Zhu, Zhongxue Gan

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Evaluating Vision-Language Models on Bistable Images

Artemis Panagopoulou, Coby Melkin, Chris Callison-Burch

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Track Anything Rapter(TAR)

Tharun V. Puthanveettil, Fnu Obaid ur Rahman

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Zero Shot Context-Based Object Segmentation using SLIP (SAM+CLIP)

Saaketh Koundinya Gundavarapu, Arushi Arora, Shreya Agarwal

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 3 figures

Measuring Vision-Language STEM Skills of Neural Models

Jianhao Shen, Ye Yuan, Srbuhi Mirzoyan, Ming Zhang, Chenguang Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted in ICLR 2024

Revisiting the Robust Generalization of Adversarial Prompt Tuning

Fan Yang, Mingxuan Xia, Sangzhou Xia, Chicheng Ma, Hui Hui

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Promoting AI Equity in Science: Generalized Domain Prompt Learning for Accessible VLM Research

Qinglong Cao, Yuntian Chen, Lu Lu, Hao Sun, Zhenzhong Zeng, Xiaokang Yang, Dongxiao Zhang

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Decoding Emotions in Abstract Art: Cognitive Plausibility of CLIP in Recognizing Color-Emotion Associations

Hanna-Sophia Widhoelzl, Ece Takmaz

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments To appear in the Proceedings of the Annual Meeting of the Cognitive Science Society 2024

Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving

Akshay Gopalkrishnan, Ross Greer, Mohan Trivedi

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 3 figures, Accepted at CVPR 2024 Vision and Language for Autonomous Driving and Robotics Workshop

What matters when building vision-language models?

Hugo Laurençon, Léo Tronchon, Matthieu Cord, Victor Sanh

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Learning Hierarchical Image Segmentation For Recognition and By Recognition

Tsung-Wei Ke, Sangwoo Mo, Stella X. Yu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments ICLR 2024 (spotlight). First two authors contributed equally. Code available at https://github.com/twke18/CAST

Exploring the Distinctiveness and Fidelity of the Descriptions Generated by Large Vision-Language Models

Yuhang Huang, Zihan Wu, Chongyang Gao, Jiawei Peng, Xu Yang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 11 pages, 9 figures, 6 tables. For associated code, see https://anonymous.4open.science/r/Explore_FGVDs-E277

Language Models as Knowledge Bases for Visual Word Sense Disambiguation

Anastasia Kritharoula, Maria Lymperaiou, Giorgos Stamou

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Journal ref KBC-LM workshop@ ISWC 2023

VG4D: Vision-Language Model Goes 4D Video Recognition

Zhichao Deng, Xiangtai Li, Xia Li, Yunhai Tong, Shen Zhao, Mengyuan Liu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments ICRA 2024