视觉大模型 / VLM - arXivDaily 专题

2409.16718 2024-11-20 cs.CV cs.AI cs.CL cs.LG cs.RO 82%

Vision-Language Model Fine-Tuning via Simple Parameter-Efficient Modification

Ming Li, Jike Zhong, Chenxin Li, Liuzhuozheng Li, Nie Lin, Masashi Sugiyama

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments EMNLP 2024 Main Conference

2411.03827 2024-11-07 cs.HC 82%

DesignMinds: Enhancing Video-Based Design Ideation with Vision-Language Model and Context-Injected Large Language Model

Tianhao He, Andrija Stankovic, Evangelos Niforatos, Gerd Kortuem

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)

2410.22736 2024-10-31 cs.CL 82%

Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model

Keito Sasagawa, Koki Maeda, Issa Sugiura, Shuhei Kurita, Naoaki Okazaki, Daisuke Kawahara

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)

Comments 15 pages, 7 figures

2408.04821 2024-10-04 cs.RO 82%

VLM-MPC: Vision Language Foundation Model (VLM)-Guided Model Predictive Controller (MPC) for Autonomous Driving

Keke Long, Haotian Shi, Jiaxi Liu, Xiaopeng Li

专题命中 VLM训练与架构：VLM(title,abstract)；vision language model(abstract)

2405.05885 2024-10-03 cs.RO 82%

VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenes

Ziang Guo, Zakhar Yagudin, Artem Lykov, Mikhail Konenkov, Dzmitry Tsetserukou

专题命中 VLM训练与架构：VLM(title,abstract)；visual language model(abstract)

Comments The paper is accepted by the IEEE conference

2409.09276 2024-09-17 cs.RO 82%

Visuo-Tactile Zero-Shot Object Recognition with Vision-Language Model

Shiori Ueda, Atsushi Hashimoto, Masashi Hamaya, Kazutoshi Tanaka, Hideo Saito

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)

Comments 9 pages, 9 figures, accepted to IROS2024, project page: https://omron-sinicx.github.io/visuo-tactile-recognition/

2409.02958 2024-09-06 cs.CV cs.AI cs.LG 82%

Multi-Modal Adapter for Vision-Language Models

Dominykas Seputis, Serghei Mihailov, Soham Chatterjee, Zehao Xiao

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2408.03735 2024-08-08 cs.CV cs.AI cs.LG 82%

Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation

Jingjing Xie, Yuxin Zhang, Mingbao Lin, Liujuan Cao, Rongrong Ji

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ACMMM2024

2407.10870 2024-07-16 cs.CV cs.AI cs.HC cs.LG 82%

GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM

Keshav Bimbraw, Ye Wang, Jing Liu, Toshiaki Koike-Akino

专题命中 VLM训练与架构：VLM(title)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 8 pages, 9 figures

2406.19578 2024-07-01 cs.CV cs.AI cs.CL cs.LG 82%

PathAlign: A vision-language model for whole slide images in histopathology

Faruk Ahmed, Andrew Sellergren, Lin Yang, Shawn Xu, Boris Babenko, Abbi Ward, Niels Olson, Arash Mohtashamian, Yossi Matias, Greg S. Corrado, Quang Duong, Dale R. Webster, Shravya Shetty, Daniel Golden, Yun Liu, David F. Steiner, Ellery Wulczyn

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 9 main pages and 19 pages of supplemental material; 3 main tables, 3 main figures and 11 supplemental tables, 7 supplemental figures

2406.10502 2024-06-18 cs.LG cs.AI cs.CV 82%

Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data

Jiahan Zhang, Qi Wei, Feng Liu, Lei Feng

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICML2024

2406.05130 2024-06-10 cs.CL 82%

An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models

Xiongtao Zhou, Jie He, Yuhua Ke, Guangyao Zhu, Víctor Gutiérrez-Basulto, Jeff Z. Pan

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)

Comments ACL finding 2024

2402.04050 2024-06-04 cs.LG cs.AI cs.CV 82%

Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models

Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, Tieniu Tan

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICML 2024

2404.11207 2024-04-18 cs.CV cs.AI cs.LG 82%

Exploring the Transferability of Visual Prompting for Multimodal Large Language Models

Yichi Zhang, Yinpeng Dong, Siyuan Zhang, Tianzan Min, Hang Su, Jun Zhu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted in CVPR 2024 as Poster (Highlight)

2404.11046 2024-04-18 cs.AI cs.CV cs.LG 82%

Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model

Hao Yan, Yuhong Guo

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2403.19837 2024-04-12 cs.LG cs.AI cs.CL cs.CV cs.LO 82%

Concept-based Analysis of Neural Networks via Vision-Language Models

Ravi Mangal, Nina Narodytska, Divya Gopinath, Boyue Caroline Hu, Anirban Roy, Susmit Jha, Corina Pasareanu

专题命中 VLM训练与架构：vision-language model(title)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

2401.04394 2024-04-04 cs.MM cs.SD eess.AS 82%

SonicVisionLM: Playing Sound with Vision Language Models

Zhifeng Xie, Shengye Yu, Qile He, Mengtian Li

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；VLM(abstract)

Comments CVPR 2024

2401.09454 2024-01-19 cs.CV cs.AI cs.CL cs.LG 82%

Voila-A: Aligning Vision-Language Models with User's Gaze Attention

Kun Yan, Lei Ji, Zeyu Wang, Yuntao Wang, Nan Duan, Shuai Ma

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

2401.06591 2024-01-15 cs.CL 82%

Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation

Seongyun Lee, Seungone Kim, Sue Hyun Park, Geewook Kim, Minjoon Seo

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)

Comments Work in progress

2312.01564 2023-12-05 cs.LG cs.AI cs.CL cs.CV 82%

APoLLo: Unified Adapter and Prompt Learning for Vision Language Models

Sanjoy Chowdhury, Sayan Nag, Dinesh Manocha

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at EMNLP 2023 (Main track)

2305.03785 2023-11-09 cs.DB 82%

Zelda: Video Analytics using Vision-Language Models

Francisco Romero, Caleb Winston, Johann Hauswald, Matei Zaharia, Christos Kozyrakis

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)

2310.08166 2023-11-01 cs.CL 82%

Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Junyu Lu, Dixiang Zhang, Xiaojun Wu, Xinyu Gao, Ruyi Gan, Jiaxing Zhang, Yan Song, Pingjian Zhang

专题命中 VLM训练与架构：vision-language model(title,abstract)；visual question answering(abstract)

2302.04858 2023-10-24 cs.CV cs.AI cs.CL cs.IR cs.LG 82%

Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning

Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Korthikanti, Weili Nie, De-An Huang, Linxi Fan, Zhiding Yu, Shiyi Lan, Bo Li, Ming-Yu Liu, Yuke Zhu, Mohammad Shoeybi, Bryan Catanzaro, Chaowei Xiao, Anima Anandkumar

专题命中 VLM训练与架构：visual language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Findings of EMNLP 2023

2307.00398 2023-10-02 cs.CV cs.AI cs.LG 82%

ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models

Uddeshya Upadhyay, Shyamgopal Karthik, Massimiliano Mancini, Zeynep Akata

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICCV 2023

2203.05557 2022-10-07 cs.CV cs.AI cs.CL cs.LG 82%

Conditional Prompt Learning for Vision-Language Models

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, Ziwei Liu

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2022. Update: Adds results on the DOSCO (DOmain Shift in COntext) benchmark

2607.23052 2026-07-28 cs.CV cs.CL cs.LG 新提交 82%

Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models

相似性并非逻辑：双编码器视觉语言模型的因式推理

Sultan Alshehri, Zhantao Yang, Han Zhang, Marios Savvides

专题命中 VLM训练与架构：vision-language model(title,abstract)；分类 cs.CV、cs.LG；VLM(comments)

AI总结研究双编码器视觉语言模型组合约束失效问题，提出因式推理，将证据提取与约束执行分离，引入LCSE方法，在FACTOR-Bench上实验，提升了准确率并保持检索性能。

Comments Accepted at ICML 2026. 18 pages, 8 figures. Project page: https://sultanmo.github.io/factored-vlm Code and benchmark: https://github.com/SultanMo/factored-vlm

2502.00425 2025-08-12 cs.CV cs.AI 82%

MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization

JiangYong Yu, Sifan Zhou, Dawei Yang, Shuo Wang, Shuoyu Li, Xing Hu, Chen Xu, Zukang Xu, Changyong Shu, Zhihang Yuan

机构 * Southeast University（东南大学）； Xi'an Jiaotong University（西安交通大学）

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACM MM 2025. First PTQ solution for Multimodal large language models applicable to 5 mainstream MLLMs

2501.03895 2025-03-04 cs.CV cs.AI cs.CL 82%

LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token

Shaolei Zhang, Qingkai Fang, Zhe Yang, Yang Feng

专题命中 VLM训练与架构：LLaVA(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICLR 2025. Code: https://github.com/ictnlp/LLaVA-Mini Model: https://huggingface.co/ICTNLP/llava-mini-llama-3.1-8b

2408.03326 2024-10-29 cs.CV cs.AI cs.CL 82%

LLaVA-OneVision: Easy Visual Task Transfer

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, Chunyuan Li

专题命中 VLM训练与架构：LLaVA(title,abstract)；分类 cs.CV、cs.AI

Comments Project Homepage: https://llava-vl.github.io/blog/2024-08-05-llava-onevision/

2407.07895 2024-07-30 cs.CV cs.CL cs.LG 82%

LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models

Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, Chunyuan Li

专题命中 VLM训练与架构：LLaVA(title,abstract)；分类 cs.CV、cs.LG

Comments Project Page: https://llava-vl.github.io/blog/2024-06-16-llava-next-interleave/

1. VLM训练与架构 5030 篇

Vision-Language Model Fine-Tuning via Simple Parameter-Efficient Modification

DesignMinds: Enhancing Video-Based Design Ideation with Vision-Language Model and Context-Injected Large Language Model

Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model

VLM-MPC: Vision Language Foundation Model (VLM)-Guided Model Predictive Controller (MPC) for Autonomous Driving

VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenes

Visuo-Tactile Zero-Shot Object Recognition with Vision-Language Model

Multi-Modal Adapter for Vision-Language Models

Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation

GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM

PathAlign: A vision-language model for whole slide images in histopathology

Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data

An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models

Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models

Exploring the Transferability of Visual Prompting for Multimodal Large Language Models

Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model

Concept-based Analysis of Neural Networks via Vision-Language Models

SonicVisionLM: Playing Sound with Vision Language Models

Voila-A: Aligning Vision-Language Models with User's Gaze Attention

Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation

APoLLo: Unified Adapter and Prompt Learning for Vision Language Models

Zelda: Video Analytics using Vision-Language Models

Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning

ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models

Conditional Prompt Learning for Vision-Language Models

Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models

MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization

LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token

LLaVA-OneVision: Easy Visual Task Transfer

LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models