视觉大模型 / VLM - arXivDaily 专题

2502.11427 2025-02-18 cs.CL cs.CV 83%

Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models

Zikang Liu, Kun Zhou, Wayne Xin Zhao, Dawei Gao, Yaliang Li, Ji-Rong Wen

专题命中 VLM训练与架构：vision-language model(title,abstract)；visual reasoning(abstract)；分类 cs.CV

Comments under review

2407.08507 2025-02-18 cs.CV 83%

Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement

Zijie Yue, Miaojing Shi, Hanli Wang, Shuai Ding, Qijun Chen, Shanlin Yang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments International Journal of Computer Vision

2501.16065 2025-02-12 cs.CV 83%

CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification

Huazhong Zhao, Lei Qi, Xin Geng

专题命中 VLM训练与架构：vision-language model(title,abstract)；visual language model(abstract)；分类 cs.CV

Comments Accepted by IEEE TIFS

2502.06094 2025-02-11 cs.CV 83%

Fair-MoE: Fairness-Oriented Mixture of Experts in Vision-Language Models

Peiran Wang, Linjie Tong, Jiaxiang Liu, Zuozhu Liu

专题命中 VLM训练与架构：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

2502.06019 2025-02-11 cs.CV 83%

Noise is an Efficient Learner for Zero-Shot Vision-Language Models

Raza Imam, Asif Hanif, Jian Zhang, Khaled Waleed Dawoud, Yova Kementchedjhieva, Mohammad Yaqub

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Our code is available at https://github.com/Razaimam45/TNT

2412.16158 2025-02-11 cs.CV 83%

HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding

Chenxin Tao, Shiqian Su, Xizhou Zhu, Chenyu Zhang, Zhe Chen, Jiawen Liu, Wenhai Wang, Lewei Lu, Gao Huang, Yu Qiao, Jifeng Dai

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2502.02458 2025-02-05 cs.CL cs.CV 83%

SAISA: Towards Multimodal Large Language Models with Both Training and Inference Efficiency

Qianhao Yuan, Yanjiang Liu, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han, Le Sun

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

2501.15040 2025-01-28 cs.CV 83%

Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification

Zhongqi Wang, Jia Dai, Kai Li, Xu Li, Yanmeng Guo, Maosheng Xiang

专题命中 VLM训练与架构：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Preprint version

2501.09039 2025-01-17 cs.CR cs.AI cs.CY 83%

Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models

Abdulkadir Erol, Trilok Padhi, Agnik Saha, Ugur Kursuncu, Mehmet Emin Aktas

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.AI

2408.09698 2025-01-14 cs.IR cs.AI 83%

Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation

Yuyang Ye, Zhi Zheng, Yishan Shen, Tianshu Wang, Hengruo Zhang, Peijun Zhu, Runlong Yu, Kai Zhang, Hui Xiong

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.AI

2403.09792 2025-01-14 cs.CV cs.CL 83%

Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models

Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Ji-Rong Wen

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments ECCV 2024 Oral

2402.16315 2025-01-08 cs.CV cs.CL 83%

Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models

Jeonghwan Kim, Heng Ji

专题命中 VLM训练与架构：vision language model(title)；vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

Comments EMNLP 2024; Main Conference

2410.17283 2025-01-03 cs.AI 83%

Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques

Lijie Tao, Haokui Zhang, Haizhao Jing, Yu Liu, Dawei Yan, Guoting Wei, Xizhe Xue

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；分类 cs.AI

2412.15614 2024-12-23 cs.CR cs.CV 83%

Technical Report for ICML 2024 TiFA Workshop MLLM Attack Challenge: Suffix Injection and Projected Gradient Descent Can Easily Fool An MLLM

Yangyang Guo, Ziwei Xu, Xilie Xu, YongKang Wong, Liqiang Nie, Mohan Kankanhalli

专题命中 VLM训练与架构：MLLM(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments ICML TiFA Challenge Technical Report

2412.11917 2024-12-20 cs.CV 83%

Does VLM Classification Benefit from LLM Description Semantics?

Pingchuan Ma, Lennart Rietdorf, Dmytro Kotovenko, Vincent Tao Hu, Björn Ommer

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

Comments AAAI-25 (extended version), Code: https://github.com/CompVis/DisCLIP

2406.18868 2024-12-19 cs.CV 83%

Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models

Yicheng Xu, Yuxin Chen, Jiahao Nie, Yusong Wang, Huiping Zhuang, Manabu Okumura

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024

2412.09616 2024-12-16 cs.CV 83%

V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

Junqi Ge, Ziyi Chen, Jintao Lin, Jinguo Zhu, Xihui Liu, Jifeng Dai, Xizhou Zhu

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments The code and models will be available at https://github.com/OpenGVLab/V2PE

2412.09530 2024-12-13 cs.CV 83%

Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLM

Han Wang, Yuxiang Nie, Yongjie Ye, Deng GuanYu, Yanjie Wang, Shuai Li, Haiyang Yu, Jinghui Lu, Can Huang

专题命中 VLM训练与架构：VLM(title)；vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

2404.10595 2024-12-09 cs.CV 83%

Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases

Kai Chen, Yanze Li, Wenhua Zhang, Yanxin Liu, Pengxiang Li, Ruiyuan Gao, Lanqing Hong, Meng Tian, Xinhai Zhao, Zhenguo Li, Dit-Yan Yeung, Huchuan Lu, Xu Jia

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accept by WACV 2025. Project Page: https://coda-dataset.github.io/coda-lm/

2412.03324 2024-12-06 cs.CV 83%

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

Wangbo Zhao, Yizeng Han, Jiasheng Tang, Zhikai Li, Yibing Song, Kai Wang, Zhangyang Wang, Yang You

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

2411.11066 2024-11-19 cs.CV 83%

TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models

Tingyu Qu, Mingxiao Li, Tinne Tuytelaars, Marie-Francine Moens

专题命中 VLM训练与架构：LLaVA(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments work in progress

2411.10928 2024-11-19 cs.CL cs.AI 83%

Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning

Wenke Huang, Jian Liang, Zekun Shi, Didi Zhu, Guancheng Wan, He Li, Bo Du, Dacheng Tao, Mang Ye

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.AI

2411.10803 2024-11-19 cs.CV 83%

Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model

Ting Liu, Liangtao Shi, Richang Hong, Yue Hu, Quanjun Yin, Linfeng Zhang

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments 8 pages, 4figures

2402.18068 2024-11-19 cs.CV 83%

SynArtifact: Classifying and Alleviating Artifacts in Synthetic Images via Vision-Language Model

Bin Cao, Jianhao Yuan, Yexin Liu, Jian Li, Shuyang Sun, Jing Liu, Bo Zhao

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2411.09301 2024-11-15 cs.CV 83%

LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation

Zhenshi Li, Dilxat Muhtar, Feng Gu, Xueliang Zhang, Pengfeng Xiao, Guangjun He, Xiaoxiang Zhu

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2411.04942 2024-11-08 cs.CV 83%

A Reinforcement Learning-Based Automatic Video Editing Method Using Pre-trained Vision-Language Model

Panwen Hu, Nan Xiao, Feifei Li, Yongquan Chen, Rui Huang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2410.16261 2024-11-08 cs.CV 83%

Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance

Zhangwei Gao, Zhe Chen, Erfei Cui, Yiming Ren, Weiyun Wang, Jinguo Zhu, Hao Tian, Shenglong Ye, Junjun He, Xizhou Zhu, Lewei Lu, Tong Lu, Yu Qiao, Jifeng Dai, Wenhai Wang

专题命中 VLM训练与架构：InternVL(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Technical report

2410.23089 2024-10-31 cs.CV 83%

PIP-MM: Pre-Integrating Prompt Information into Visual Encoding via Existing MLLM Structures

Tianxiang Wu, Minxin Nie, Ziqiang Cao

专题命中 VLM训练与架构：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

2410.22313 2024-10-30 cs.CV cs.RO 83%

Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving

Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, Xinggang Wang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Project Page: https://github.com/hustvl/Senna

2410.17827 2024-10-24 cs.AI 83%

RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification

Marco Mistretta, Andrew D. Bagdanov

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments Accepted for publication at Medical Imaging meets NeurIPS (NeurIPS23)

1. VLM训练与架构 5030 篇

Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models

Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement

CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification

Fair-MoE: Fairness-Oriented Mixture of Experts in Vision-Language Models

Noise is an Efficient Learner for Zero-Shot Vision-Language Models

HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding

SAISA: Towards Multimodal Large Language Models with Both Training and Inference Efficiency

Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification

Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models

Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation

Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models

Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models

Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques

Technical Report for ICML 2024 TiFA Workshop MLLM Attack Challenge: Suffix Injection and Projected Gradient Descent Can Easily Fool An MLLM

Does VLM Classification Benefit from LLM Description Semantics?

Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models

V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLM

Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models

Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning

Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model

SynArtifact: Classifying and Alleviating Artifacts in Synthetic Images via Vision-Language Model

LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation

A Reinforcement Learning-Based Automatic Video Editing Method Using Pre-trained Vision-Language Model

Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance

PIP-MM: Pre-Integrating Prompt Information into Visual Encoding via Existing MLLM Structures

Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving

RE-tune: Incremental Fine Tuning of Biomedical Vision-Language Models for Multi-label Chest X-ray Classification