多模态大模型

StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data

Yanda Li, Chi Zhang, Gang Yu, Zhibin Wang, Bin Fu, Guosheng Lin, Chunhua Shen, Ling Chen, Yunchao Wei

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://github.com/icoz69/StableLLAVA

An Empirical Study of CLIP for Text-based Person Search

Min Cao, Yang Bai, Ziyin Zeng, Mang Ye, Min Zhang

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by AAAI 2024. Code is available at https://github.com/Flame-Chasers/TBPS-CLIP

When Parameter-efficient Tuning Meets General-purpose Vision-language Models

Yihang Zhai, Haixin Wang, Jianlong Chang, Xinlong Yang, Jinan Sun, Shikun Zhang, Qi Tian

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Transferring CLIP's Knowledge into Zero-Shot Point Cloud Semantic Segmentation

Yuanbin Wang, Shaofei Huang, Yulu Gao, Zhen Wang, Rui Wang, Kehua Sheng, Bo Zhang, Si Liu

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

CoPL: Contextual Prompt Learning for Vision-Language Understanding

Koustava Goswami, Srikrishna Karanam, Prateksha Udhayanan, K J Joseph, Balaji Vasan Srinivasan

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at AAAI 2024

Interpretable Visual Understanding with Cognitive Attention Network

Xuejiao Tang, Wenbin Zhang, Yi Yu, Kea Turner, Tyler Derr, Mengyu Wang, Eirini Ntoutsi

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments ICANN21

Prompt Tuning for Zero-shot Compositional Learning

Lingyu Zhang, Ting Hua, Yilin Shen, Hongxia Jin

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

The Contemporary Art of Image Search: Iterative User Intent Expansion via Vision-Language Model

Yilin Ye, Qian Zhu, Shishi Xiao, Kang Zhang, Wei Zeng

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by The 2024 ACM SIGCHI Conference on Computer-Supported Cooperative Work & Social Computing (CSCW) (Proc. CSCW 2024)

Semantic-Aware Frame-Event Fusion based Pattern Recognition via Large Vision-Language Models

Dong Li, Jiandong Jin, Yuhao Zhang, Yanlin Zhong, Yaoyang Wu, Lan Chen, Xiao Wang, Bin Luo

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments In Peer Review

Vision-Language Instruction Tuning: A Review and Analysis

Chen Li, Yixiao Ge, Dian Li, Ying Shan

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments 34 pages, 6 figures

Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models

Shuo Chen, Jindong Gu, Zhen Han, Yunpu Ma, Philip Torr, Volker Tresp

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at NeurIPS 2023 Datasets and Benchmarks Track; 9 pages, 5 figures

GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks

Xinlu Zhang, Yujie Lu, Weizhi Wang, An Yan, Jun Yan, Lianke Qin, Heng Wang, Xifeng Yan, William Yang Wang, Linda Ruth Petzold

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

ChatGPT-Powered Hierarchical Comparisons for Image Classification

Zhiyuan Ren, Yiyang Su, Xiaoming Liu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Neurips 2023 Poster

Text encoders bottleneck compositionality in contrastive vision-language models

Amita Kamath, Jack Hessel, Kai-Wei Chang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2023

LXMERT Model Compression for Visual Question Answering

Maryam Hashemi, Ghazaleh Mahmoudi, Sara Kodeiri, Hadi Sheikhi, Sauleh Eetemadi

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments To appear in The Fourth Annual West Coast NLP (WeCNLP) Summit

Large Language Models are Visual Reasoning Coordinators

Liangyu Chen, Bo Li, Sheng Shen, Jingkang Yang, Chunyuan Li, Kurt Keutzer, Trevor Darrell, Ziwei Liu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at NeurIPS 2023

CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement

Mohammadreza Salehi, Mehrdad Farajtabar, Maxwell Horton, Fartash Faghri, Hadi Pouransari, Raviteja Vemulapalli, Oncel Tuzel, Ali Farhadi, Mohammad Rastegari, Sachin Mehta

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks

Xinsong Zhang, Yan Zeng, Jipeng Zhang, Hang Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Prompting Scientific Names for Zero-Shot Species Recognition

Shubham Parashar, Zhiqiu Lin, Yanan Li, Shu Kong

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2023

Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, Jingren Zhou

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Code, demo and models are available at https://github.com/QwenLM/Qwen-VL

Probing Intersectional Biases in Vision-Language Models with Counterfactual Examples

Phillip Howard, Avinash Madasu, Tiep Le, Gustavo Lujan Moreno, Vasudev Lal

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

DOMINO: A Dual-System for Multi-step Visual Language Reasoning

Peifang Wang, Olga Golovneva, Armen Aghajanyan, Xiang Ren, Muhao Chen, Asli Celikyilmaz, Maryam Fazel-Zarandi

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models

Uddeshya Upadhyay, Shyamgopal Karthik, Massimiliano Mancini, Zeynep Akata

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023

Sigmoid Loss for Language Image Pre-Training

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, Lucas Beyer

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments ICCV'23 Oral. arXiv v2: fix typo in pseudocode; v3: clarify t vs t' init; v4: add SigLIP Base, Large, Shape-Optimized 400M results. Models released at: https://github.com/google-research/big_vision. Xiaohua and Lucas contributed equally

Survey of Social Bias in Vision-Language Models

Nayeon Lee, Yejin Bang, Holy Lovenia, Samuel Cahyawijaya, Wenliang Dai, Pascale Fung

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

When is a Foundation Model a Foundation Model

Saghir Alfasly, Peyman Nejat, Sobhan Hemati, Jibran Khan, Isaiah Lahr, Areej Alsaafin, Abubakr Shafique, Nneka Comfere, Dennis Murphree, Chady Meroueh, Saba Yasir, Aaron Mangold, Lisa Boardman, Vijay Shah, Joaquin J. Garcia, H. R. Tizhoosh

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Improving CLIP Robustness with Knowledge Distillation and Self-Training

Clement Laroudie, Andrei Bursuc, Mai Lan Ha, Gianni Franchi

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Zero-Shot Audio Captioning via Audibility Guidance

Tal Shaharabany, Ariel Shaulov, Lior Wolf

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Parameter Efficient Audio Captioning With Faithful Guidance Using Audio-text Shared Latent Representation

Arvind Krishna Sridhar, Yinyi Guo, Erik Visser, Rehana Mahfuz

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.MM

Comments 5 pages, 5 tables, 1 figure

DeViL: Decoding Vision features into Language

Meghal Dani, Isabel Rio-Torto, Stephan Alaniz, Zeynep Akata

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted at GCPR 2023 (Oral)