视觉大模型 / VLM

Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Model

Hao Cheng, Erjia Xiao, Jindong Gu, Le Yang, Jinhao Duan, Jize Zhang, Jiahang Cao, Kaidi Xu, Renjing Xu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments This paper is accepted by ECCV 2024

Mixture of Prompt Learning for Vision Language Models

Yu Du, Tong Niu, Rong Zhao

专题命中 VLM训练与架构 ：vision language model(title)；vision-language model(abstract)；分类 cs.CV

SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models

Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, Afshin Dehghan

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments Technical report

ComAlign: Compositional Alignment in Vision-Language Models

Ali Abdollah, Amirmohammad Izadi, Armin Saghafian, Reza Vahidimajd, Mohammad Mozafari, Amirreza Mirzaei, Mohammadmahdi Samiei, Mahdieh Soleymani Baghshah

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Revisiting Prompt Pretraining of Vision-Language Models

Zhenyuan Chen, Lingfeng Yang, Shuo Chen, Zhaowei Chen, Jiajun Liang, Xiang Li

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Few-shot Adaptation of Medical Vision-Language Models

Fereshteh Shakeri, Yunshi Huang, Julio Silva-Rodríguez, Houda Bahig, An Tang, Jose Dolz, Ismail Ben Ayed

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments MICCAI 2024 (Spotlight) - Code is available at https://github.com/FereshteShakeri/few-shot-MedVLMs.git

LLaVA-Chef: A Multi-modal Generative Model for Food Recipes

Fnu Mohbat, Mohammed J. Zaki

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.LG

Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning

Zhengqing Gao, Xiang Ao, Xu-Yao Zhang, Cheng-Lin Liu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments PRCV 2024

u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model

Jinjin Xu, Liwu Xu, Yuzhe Yang, Xiang Li, Fanyi Wang, Yanchun Xie, Yi-Jie Huang, Yaqian Li

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Making Large Vision Language Models to be Good Few-shot Learners

Fan Liu, Wenwen Cai, Jian Huo, Chuanyi Zhang, Delong Chen, Jun Zhou

专题命中 VLM训练与架构 ：vision language model(title,abstract)；分类 cs.CV

The Phantom Menace: Unmasking Privacy Leakages in Vision-Language Models

Simone Caldarella, Massimiliano Mancini, Elisa Ricci, Rahaf Aljundi

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

PromptKD: Unsupervised Prompt Distillation for Vision-Language Models

Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, Jian Yang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR 2024. Project Page: https://zhengli97.github.io/PromptKD. Code: https://github.com/zhengli97/PromptKD

Consistency-guided Prompt Learning for Vision-Language Models

Shuvendu Roy, Ali Etemad

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Published as a conference paper at ICLR 2024

FlexAttention for Efficient High-Resolution Vision-Language Models

Junyan Li, Delin Chen, Tianle Cai, Peihao Chen, Yining Hong, Zhenfang Chen, Yikang Shen, Chuang Gan

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024

LLAVADI: What Matters For Multimodal Large Language Models Distillation

Shilin Xu, Xiangtai Li, Haobo Yuan, Lu Qi, Yunhai Tong, Ming-Hsuan Yang

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic

Fakhraddin Alwajih, Gagan Bhatia, Muhammad Abdul-Mageed

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.AI

Test-Time Low Rank Adaptation via Confidence Maximization for Zero-Shot Generalization of Vision-Language Models

Raza Imam, Hanan Gani, Muhammad Huzaifa, Karthik Nandakumar

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Main paper: 11 pages, Supplementary material: 5 pages

EVLM: An Efficient Vision-Language Model for Visual Understanding

Kaibing Chen, Dong Shen, Hanwen Zhong, Huasong Zhong, Kui Xia, Di Xu, Wei Yuan, Yifei Hu, Bin Wen, Tianke Zhang, Changyi Liu, Dewen Fan, Huihui Xiao, Jiahong Wu, Fan Yang, Size Li, Di Zhang

专题命中 VLM训练与架构 ：vision-language model(title)；LLaVA(abstract)；分类 cs.CV

E5-V: Universal Embeddings with Multimodal Large Language Models

Ting Jiang, Minghui Song, Zihan Zhang, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang, Deqing Wang, Fuzhen Zhuang

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Comments Code and models are available at https://github.com/kongds/E5-V

The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?

Qinyu Zhao, Ming Xu, Kartik Gupta, Akshay Asthana, Liang Zheng, Stephen Gould

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments ECCV 2024. Project page: https://github.com/Qinyu-Allen-Zhao/LVLM-LP

Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models

Jinrui Zhang, Teng Wang, Haigang Zhang, Ping Lu, Feng Zheng

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments To appear at ECCV2024

Conceptual Codebook Learning for Vision-Language Models

Yi Zhang, Ke Yu, Siqi Wu, Zhihai He

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024

MuDPT: Multi-modal Deep-symphysis Prompt Tuning for Large Pre-trained Vision-Language Models

Yongzhu Miao, Shasha Li, Jintao Tang, Ting Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments The paper has been accepted by ICME 2023

Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Model

Duy M. H. Nguyen, An T. Le, Trung Q. Nguyen, Nghiem T. Diep, Tai Nguyen, Duy Duong-Tran, Jan Peters, Li Shen, Mathias Niepert, Daniel Sonntag

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Version 1

Improving Visual Storytelling with Multimodal Large Language Models

Xiaochuan Lin, Xiangyong Chen

专题命中 VLM训练与架构 ：multimodal large language model(title)；vision-language model(abstract)；分类 cs.CV

Comments 10 pages

InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models

Xunguang Wang, Zhenlan Ji, Pingchuan Ma, Zongjie Li, Shuai Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification

Gregor Geigle, Radu Timofte, Goran Glavaš

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Can Better Text Semantics in Prompt Tuning Improve VLM Generalization?

Hari Chandana Kuchibhotla, Sai Srinivas Kancheti, Abbavaram Gowtham Reddy, Vineeth N Balasubramanian

专题命中 VLM训练与架构 ：VLM(title)；vision-language model(abstract)；分类 cs.CV

BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Models

Xuefeng Hu, Ke Zhang, Min Sun, Albert Chen, Cheng-Hao Kuo, Ram Nevatia

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Preprint updated from our earlier manuscript submitted to ICLR 2024 (https://openreview.net/forum?id=KNtcoAM5Gy)

ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models

Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, Benyou Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.AI

Comments 22 pages