视觉大模型 / VLM - arXivDaily 专题

2408.15881 2024-10-24 cs.CV 83%

LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation

Fangxun Shu, Yue Liao, Le Zhuo, Chenning Xu, Lei Zhang, Guanghao Zhang, Haonan Shi, Long Chen, Tao Zhong, Wanggui He, Siming Fu, Haoyuan Li, Bolin Li, Zhelun Yu, Si Liu, Hongsheng Li, Hao Jiang

专题命中 VLM训练与架构：LLaVA(title,abstract)；MLLM(abstract)；分类 cs.CV

2410.13861 2024-10-22 cs.CV 83%

PUMA: Empowering Unified MLLM with Multi-granular Visual Generation

Rongyao Fang, Chengqi Duan, Kun Wang, Hao Li, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, Hongsheng Li, Xihui Liu

专题命中 VLM训练与架构：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Project page: https://rongyaofang.github.io/puma/

2403.13756 2024-10-16 cs.CV 83%

Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model

Diwei Wang, Kun Yuan, Candice Muller, Frédéric Blanc, Nicolas Padoy, Hyewon Seo

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments MICCAI 2024

2410.07133 2024-10-11 cs.CV 83%

EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models

Rui Zhao, Hangjie Yuan, Yujie Wei, Shiwei Zhang, Yuchao Gu, Lingmin Ran, Xiang Wang, Zhangjie Wu, Junhao Zhang, Yingya Zhang, Mike Zheng Shou

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2410.06456 2024-10-10 cs.CV 83%

From Generalist to Specialist: Adapting Vision Language Models via Task-Specific Visual Instruction Tuning

Yang Bai, Yang Zhou, Jun Zhou, Rick Siow Mong Goh, Daniel Shu Wei Ting, Yong Liu

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

2403.16167 2024-10-07 cs.CV cs.CL 83%

ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models

Minchan Kim, Minyeong Kim, Junik Bae, Suhwan Choi, Sungkyung Kim, Buru Chang

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments ECCV 2024

2311.10122 2024-10-02 cs.CV 83%

Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, Li Yuan

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.CV

2311.01477 2024-09-30 cs.CV 83%

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models

Liqiang Jing, Ruosen Li, Yunmo Chen, Xinya Du

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted by Findings of EMNLP 2024

2409.17805 2024-09-27 cs.CV 83%

Cascade Prompt Learning for Vision-Language Model Adaptation

Ge Wu, Xin Zhang, Zheng Li, Zhaowei Chen, Jiajun Liang, Jian Yang, Xiang Li

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments ECCV2024

2409.16723 2024-09-27 cs.CV 83%

EAGLE: Towards Efficient Arbitrary Referring Visual Prompts Comprehension for Multimodal Large Language Models

Jiacheng Zhang, Yang Jiao, Shaoxiang Chen, Jingjing Chen, Yu-Gang Jiang

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2409.12597 2024-09-20 cs.CV 83%

LARE: Latent Augmentation using Regional Embedding with Vision-Language Model

Kosuke Sakurai, Tatsuya Ishii, Ryotaro Shimizu, Linxin Song, Masayuki Goto

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments 10 pages, 4 figures

2408.16930 2024-09-02 cs.CV 83%

VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognition

Zaiwei Zhang, Gregory P. Meyer, Zhichao Lu, Ashish Shrivastava, Avinash Ravichandran, Eric M. Wolff

专题命中 VLM训练与架构：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

2408.15556 2024-08-29 cs.CV 83%

Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Dacheng Tao

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2408.09530 2024-08-20 cs.AI 83%

PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding

Dawei Dai, Yuanhui Zhang, Long Xu, Qianlan Yang, Xiaojing Shen, Shuyin Xia, Guoyin Wang

专题命中 VLM训练与架构：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.AI

Comments 8 pages, 4 figs

2408.01800 2024-08-06 cs.CV 83%

MiniCPM-V: A GPT-4V Level MLLM on Your Phone

Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, Maosong Sun

专题命中 VLM训练与架构：MLLM(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments preprint

2405.11537 2024-08-06 cs.RO cs.AI cs.ET 83%

VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications

Mikhail Konenkov, Artem Lykov, Daria Trinitatova, Dzmitry Tsetserukou

专题命中 VLM训练与架构：visual language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments Updated version

2407.09043 2024-07-24 cs.AI 83%

Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models

Namkyeong Lee, Siddhartha Laghuvarapu, Chanyoung Park, Jimeng Sun

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments CIKM 2024 / ACL 2024 Workshop on Languages and Molecule

2407.12709 2024-07-18 cs.CV 83%

MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models

Leyang Shen, Gongwei Chen, Rui Shao, Weili Guan, Liqiang Nie

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Github: https://github.com/JiuTian-VL/MoME

2403.11150 2024-07-11 cs.CV 83%

Training A Small Emotional Vision Language Model for Visual Art Comprehension

Jing Zhang, Liang Zheng, Meng Wang, Dan Guo

专题命中 VLM训练与架构：vision language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

Comments 16 pages

2406.17770 2024-06-28 cs.CV 83%

MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning

Xiangyu Zhao, Xiangtai Li, Haodong Duan, Haian Huang, Yining Li, Kai Chen, Hua Yang

专题命中 VLM训练与架构：LLaVA(title,abstract)；MLLM(abstract)；分类 cs.CV

2406.17659 2024-06-26 cs.AI cs.RO 83%

DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning

Xiaohan Zhang, Zainab Altaweel, Yohei Hayamizu, Yan Ding, Saeid Amiri, Hao Yang, Andy Kaminski, Chad Esselink, Shiqi Zhang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

2406.03907 2024-06-07 cs.CV 83%

Exploring the Zero-Shot Capabilities of Vision-Language Models for Improving Gaze Following

Anshul Gupta, Pierre Vuillecard, Arya Farkhondeh, Jean-Marc Odobez

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted at the GAZE Workshop at CVPR 2024

2311.17091 2024-06-04 cs.CV 83%

Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models

Zhihe Lu, Jiawang Bai, Xin Li, Zeyu Xiao, Xinchao Wang

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted on ICML 2024

2405.17247 2024-05-28 cs.LG 83%

An Introduction to Vision-Language Modeling

Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Mañas, Zhiqiu Lin, Anas Mahmoud, Bargav Jayaraman, Mark Ibrahim, Melissa Hall, Yunyang Xiong, Jonathan Lebensold, Candace Ross, Srihari Jayakumar, Chuan Guo, Diane Bouchacourt, Haider Al-Tahan, Karthik Padthe, Vasu Sharma, Hu Xu, Xiaoqing Ellen Tan, Megan Richards, Samuel Lavoie, Pietro Astolfi, Reyhane Askari Hemmat, Jun Chen, Kushal Tirumala, Rim Assouel, Mazda Moayeri, Arjang Talattof, Kamalika Chaudhuri, Zechun Liu, Xilun Chen, Quentin Garrido, Karen Ullrich, Aishwarya Agrawal, Kate Saenko, Asli Celikyilmaz, Vikas Chandra

专题命中 VLM训练与架构：vision-language model(title,abstract)；VLM(abstract)；分类 cs.LG

2405.11850 2024-05-21 cs.CV 83%

Rethinking Overlooked Aspects in Vision-Language Models

Yuan Liu, Le Tian, Xiao Zhou, Jie Zhou

专题命中 VLM训练与架构：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV

2404.18746 2024-04-30 cs.MM cs.CV 83%

Enhancing Interactive Image Retrieval With Query Rewriting Using Large Language Models and Vision Language Models

Hongyi Zhu, Jia-Hong Huang, Stevan Rudinac, Evangelos Kanoulas

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

2404.16994 2024-04-30 cs.CV 83%

PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning

Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, Jiashi Feng

专题命中 VLM训练与架构：LLaVA(title,abstract)；VLM(abstract)；分类 cs.CV

2310.02992 2024-04-29 cs.CV cs.CL 83%

Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, Furu Wei

专题命中 VLM训练与架构：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Code: https://aka.ms/Kosmos-G Project Page: https://xichenpan.github.io/kosmosg

2404.16033 2024-04-25 cs.CV cs.CL 83%

Cantor: Inspiring Multimodal Chain-of-Thought of MLLM

Timin Gao, Peixian Chen, Mengdan Zhang, Chaoyou Fu, Yunhang Shen, Yan Zhang, Shengchuan Zhang, Xiawu Zheng, Xing Sun, Liujuan Cao, Rongrong Ji

专题命中 VLM训练与架构：MLLM(title)；visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments The project page is available at https://ggg0919.github.io/cantor/

2404.07824 2024-04-12 cs.CV cs.CL 83%

Heron-Bench: A Benchmark for Evaluating Vision Language Models in Japanese

Yuichi Inoue, Kento Sasaki, Yuma Ochi, Kazuki Fujii, Kotaro Tanahashi, Yu Yamaguchi

专题命中 VLM训练与架构：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV

1. VLM训练与架构 5030 篇

LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation

PUMA: Empowering Unified MLLM with Multi-granular Visual Generation

Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model

EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models

From Generalist to Specialist: Adapting Vision Language Models via Task-Specific Visual Instruction Tuning

ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models

Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models

Cascade Prompt Learning for Vision-Language Model Adaptation

EAGLE: Towards Efficient Arbitrary Referring Visual Prompts Comprehension for Multimodal Large Language Models

LARE: Latent Augmentation using Regional Embedding with Vision-Language Model

VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognition

Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding

MiniCPM-V: A GPT-4V Level MLLM on Your Phone

VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications

Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models

MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models

Training A Small Emotional Vision Language Model for Visual Art Comprehension

MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning

DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning

Exploring the Zero-Shot Capabilities of Vision-Language Models for Improving Gaze Following

Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models

An Introduction to Vision-Language Modeling

Rethinking Overlooked Aspects in Vision-Language Models

Enhancing Interactive Image Retrieval With Query Rewriting Using Large Language Models and Vision Language Models

PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning

Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Cantor: Inspiring Multimodal Chain-of-Thought of MLLM

Heron-Bench: A Benchmark for Evaluating Vision Language Models in Japanese