视觉大模型 / VLM

Towards Robust Instruction Tuning on Multimodal Large Language Models

Wei Han, Hui Chen, Soujanya Poria

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.AI

Comments 24 pages, 7 figures

Regularized Training with Generated Datasets for Name-Only Transfer of Vision-Language Models

Minho Park, Sunghyun Park, Jooyeol Yun, Jaegul Choo

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Preprint. Under review

Boosting Vision-Language Models with Transduction

Maxime Zanella, Benoît Gérin, Ismail Ben Ayed

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters

Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang, Huchuan Lu, You He

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments This work is accepted by CVPR2024. More modifications may be performed

Low-Rank Few-Shot Adaptation of Vision-Language Models

Maxime Zanella, Ismail Ben Ayed

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models

Linli Yao, Lei Li, Shuhuai Ren, Lean Wang, Yuanxin Liu, Xu Sun, Lu Hou

专题命中 VLM训练与架构 ：multimodal large language model(title)；MLLM(abstract)；分类 cs.CV

Knowledge-grounded Adaptation Strategy for Vision-language Models: Building Unique Case-set for Screening Mammograms for Residents Training

Aisha Urooj Khan, John Garrett, Tyler Bradshaw, Lonie Salkowski, Jiwoong Jason Jeong, Amara Tariq, Imon Banerjee

专题命中 VLM训练与架构 ：vision-language model(title)；VLM(abstract)；分类 cs.CV

Disease-informed Adaptation of Vision-Language Models

Jiajin Zhang, Ge Wang, Mannudeep K. Kalra, Pingkun Yan

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Early Accepted by MICCAI 2024

Adapting Dual-encoder Vision-language Models for Paraphrased Retrieval

Jiacheng Cheng, Hijung Valentina Shin, Nuno Vasconcelos, Bryan Russell, Fabian Caba Heilbron

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation

Xun Wu, Shaohan Huang, Furu Wei

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Iterated Learning Improves Compositionality in Large Vision-Language Models

Chenhao Zheng, Jieyu Zhang, Aniruddha Kembhavi, Ranjay Krishna

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR 2024

Distilling Vision-Language Models on Millions of Videos

Yue Zhao, Long Zhao, Xingyi Zhou, Jialin Wu, Chun-Te Chu, Hui Miao, Florian Schroff, Hartwig Adam, Ting Liu, Boqing Gong, Philipp Krähenbühl, Liangzhe Yuan

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR 2024. Project page: https://zhaoyue-zephyrus.github.io/video-instruction-tuning

Semantics-aware Motion Retargeting with Vision-Language Models

Haodong Zhang, ZhiKe Chen, Haocheng Xu, Lei Hao, Xiaofei Wu, Songcen Xu, Zhensong Zhang, Yue Wang, Rong Xiong

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted in CVPR2024

Anchor-based Robust Finetuning of Vision-Language Models

Jinwei Han, Zhiwen Lin, Zhongyisun Sun, Yingguo Gao, Ke Yan, Shouhong Ding, Yuan Gao, Gui-Song Xia

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments CVPR2024

Elysium: Exploring Object-level Perception in Videos via MLLM

Han Wang, Yanjie Wang, Yongjie Ye, Yuxiang Nie, Can Huang

专题命中 VLM训练与架构 ：MLLM(title,abstract)；分类 cs.CV

Efficient Test-Time Adaptation of Vision-Language Models

Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik, Eric Xing

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2024. The code has been released in \url{https://kdiaaa.github.io/tda/}

SC-Tune: Unleashing Self-Consistent Referential Comprehension in Large Vision Language Models

Tongtian Yue, Jie Cheng, Longteng Guo, Xingyuan Dai, Zijia Zhao, Xingjian He, Gang Xiong, Yisheng Lv, Jing Liu

专题命中 VLM训练与架构 ：vision language model(title,abstract)；分类 cs.CV

Comments Accepted by CVPR2024

Tag2Text: Guiding Vision-Language Model via Image Tagging

Xinyu Huang, Youcai Zhang, Jinyu Ma, Weiwei Tian, Rui Feng, Yuejie Zhang, Yaqian Li, Yandong Guo, Lei Zhang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ICLR 2024

GAgent: An Adaptive Rigid-Soft Gripping Agent with Vision Language Models for Complex Lighting Environments

Zhuowei Li, Miao Zhang, Xiaotian Lin, Meng Yin, Shuai Lu, Xueqian Wang

专题命中 VLM训练与架构 ：vision language model(title)；VLM(abstract)；分类 cs.AI

Vision-Language Models Learn Super Images for Efficient Partially Relevant Video Retrieval

Taichi Nishimura, Shota Nakada, Masayoshi Kondo

专题命中 VLM训练与架构 ：vision-language model(title)；VLM(abstract)；分类 cs.CV

Comments 24 pages

IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding

Lanyun Zhu, Deyi Ji, Tianrun Chen, Peng Xu, Jieping Ye, Jun Liu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Controlling Vision-Language Models for Multi-Task Image Restoration

Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by ICLR 2024. Project page: https://algolzw.github.io/daclip-uir/index.html

LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model

Yichen Zhu, Minjie Zhu, Ning Liu, Zhicai Ou, Xiaofeng Mou, Jian Tang

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；分类 cs.CV

Comments The datasets were incomplete as they did not include all the necessary copyrights

VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models

Jiawei Liang, Siyuan Liang, Man Luo, Aishan Liu, Dongchen Han, Ee-Chien Chang, Xiaochun Cao

专题命中 VLM训练与架构 ：visual language model(title,abstract)；分类 cs.CV

InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, Jifeng Dai

专题命中 VLM训练与架构 ：InternVL(title,abstract)；分类 cs.CV

Comments 25 pages, 5 figures, 28 tables

Concept-Guided Prompt Learning for Generalization in Vision-Language Models

Yi Zhang, Ce Zhang, Ke Yu, Yushun Tang, Zhihai He

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted by AAAI 2024

Application Of Vision-Language Models For Assessing Osteoarthritis Disease Severity

Banafshe Felfeliyan, Yuyue Zhou, Shrimanti Ghosh, Jessica Kupper, Shaobo Liu, Abhilash Hareendranathan, Jacob L. Jaremko

专题命中 VLM训练与架构 ：vision-language model(title)；vision language model(abstract)；分类 cs.CV

Cloud-Device Collaborative Learning for Multimodal Large Language Models

Guanqun Wang, Jiaming Liu, Chenxuan Li, Junpeng Ma, Yuan Zhang, Xinyu Wei, Kevin Zhang, Maurice Chong, Ray Zhang, Yijiang Liu, Shanghang Zhang

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；分类 cs.CV

Black-Box Tuning of Vision-Language Models with Effective Gradient Approximation

Zixian Guo, Yuxiang Wei, Ming Liu, Zhilong Ji, Jinfeng Bai, Yiwen Guo, Wangmeng Zuo

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.CV

When Parameter-efficient Tuning Meets General-purpose Vision-language Models

Yihang Zhai, Haixin Wang, Jianlong Chang, Xinlong Yang, Jinan Sun, Shikun Zhang, Qi Tian

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；分类 cs.AI