视觉大模型 / VLM - arXivDaily 专题

2304.02560 2024-04-01 cs.CV 57%

VicTR: Video-conditioned Text Representations for Activity Recognition

Kumara Kahatapitiya, Anurag Arnab, Arsha Nagrani, Michael S. Ryoo

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments To appear at CVPR 2024

2403.19811 2024-04-01 cs.CV 57%

X-MIC: Cross-Modal Instance Conditioning for Egocentric Action Generalization

Anna Kukleva, Fadime Sener, Edoardo Remelli, Bugra Tekin, Eric Sauser, Bernt Schiele, Shugao Ma

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments CVPR 2024

2403.10882 2024-03-22 cs.CL cs.AI 57%

Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean

ChangSu Choi, Yongbin Jeong, Seoyoon Park, InHo Won, HyeonSeok Lim, SangMin Kim, Yejee Kang, Chanhyuk Yoon, Jaewan Park, Yiseul Lee, HyeJin Lee, Younggyun Hahm, Hansaem Kim, KyungTae Lim

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.AI

2312.04302 2024-03-22 cs.CV cs.CL 57%

Prompt Highlighter: Interactive Control for Multi-Modal LLMs

Yuechen Zhang, Shengju Qian, Bohao Peng, Shu Liu, Jiaya Jia

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments CVPR 2024; Project Page: https://julianjuaner.github.io/projects/PromptHighlighter

2403.10887 2024-03-19 cs.CV 57%

LuoJiaHOG: A Hierarchy Oriented Geo-aware Image Caption Dataset for Remote Sensing Image-Text Retrival

Yuanxin Zhao, Mi Zhang, Bingnan Yang, Zhan Zhang, Jiaju Kang, Jianya Gong

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2311.18231 2024-03-14 cs.CV 57%

TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model

Hantao Yao, Rui Zhang, Changsheng Xu

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

Comments accepted by CVPR24

2403.05525 2024-03-12 cs.AI 57%

DeepSeek-VL: Towards Real-World Vision-Language Understanding

Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, Chong Ruan

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.AI

Comments https://github.com/deepseek-ai/DeepSeek-VL

2310.02110 2024-03-12 cs.CV 57%

Sieve: Multimodal Dataset Pruning Using Image Captioning Models

Anas Mahmoud, Mostafa Elhoushi, Amro Abbas, Yu Yang, Newsha Ardalani, Hugh Leather, Ari Morcos

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted in CVPR 2024

2403.03719 2024-03-07 cs.CV 57%

Multimodal Transformer for Comics Text-Cloze

Emanuele Vivoli, Joan Lafuente Baeza, Ernest Valveny Llobet, Dimosthenis Karatzas

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

2403.02370 2024-03-06 cs.CL cs.AI 57%

adaptMLLM: Fine-Tuning Multilingual Language Models on Low-Resource Languages with Integrated LLM Playgrounds

Séamus Lankford, Haithem Afli, Andy Way

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.AI

Journal ref Information 2023, 14(12), 638

2403.01431 2024-03-05 cs.CV 57%

Image2Sentence based Asymmetrical Zero-shot Composed Image Retrieval

Yongchao Du, Min Wang, Wengang Zhou, Shuping Hui, Houqiang Li

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments ICLR 2024 spotlight

2402.16769 2024-02-27 cs.CV 57%

Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval

Haowei Liu, Yaya Shi, Haiyang Xu, Chunfeng Yuan, Qinghao Ye, Chenliang Li, Ming Yan, Ji Zhang, Fei Huang, Bing Li, Weiming Hu

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

Comments Accepted to LREC-COLING 2024

2402.14289 2024-02-23 cs.LG cs.CL 57%

TinyLLaVA: A Framework of Small-scale Large Multimodal Models

Baichuan Zhou, Ying Hu, Xi Weng, Junlong Jia, Jie Luo, Xien Liu, Ji Wu, Lei Huang

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.LG

Comments Our model weights and codes will be made public at https://github.com/DLCV-BUAA/TinyLLaVABench

2301.04558 2024-02-23 cs.CV cs.CL 57%

Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing

Shruthi Bannur, Stephanie Hyland, Qianchu Liu, Fernando Pérez-García, Maximilian Ilse, Daniel C. Castro, Benedikt Boecking, Harshita Sharma, Kenza Bouzid, Anja Thieme, Anton Schwaighofer, Maria Wetscherek, Matthew P. Lungren, Aditya Nori, Javier Alvarez-Valle, Ozan Oktay

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

Comments To appear in CVPR 2023

2310.03291 2024-02-22 cs.CV 57%

Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction

Yiren Jian, Tingkai Liu, Yunzhe Tao, Chunhui Zhang, Soroush Vosoughi, Hongxia Yang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2402.06198 2024-02-14 cs.CV 57%

GS-CLIP: Gaussian Splatting for Contrastive Language-Image-3D Pretraining from Real-World Data

Haoyuan Li, Yanpeng Zhou, Yihan Zeng, Hang Xu, Xiaodan Liang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments The content of the technical report needs to be updated and retracted to avoid other impacts

2402.03783 2024-02-07 cs.CV 57%

Exploring Low-Resource Medical Image Classification with Weakly Supervised Prompt Learning

Fudan Zheng, Jindong Cao, Weijiang Yu, Zhiguang Chen, Nong Xiao, Yutong Lu

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by Pattern Recognition

2401.09712 2024-01-19 cs.CV 57%

SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model

Yang Zhan, Zhitong Xiong, Yuan Yuan

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

2401.08968 2024-01-18 cs.CV 57%

COCO is "ALL'' You Need for Visual Instruction Fine-tuning

Xiaotian Han, Yiqi Wang, Bohan Zhai, Quanzeng You, Hongxia Yang

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

2401.03201 2024-01-17 cs.CV cs.MM 57%

3DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding

Zeju Li, Chao Zhang, Xiaoyan Wang, Ruilong Ren, Yifan Xu, Ruifei Ma, Xiangde Liu

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

Comments 9 pages, 5 figures

2311.01459 2024-01-12 cs.CV 57%

Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization

Jameel Hassan, Hanan Gani, Noor Hussein, Muhammad Uzair Khattak, Muzammal Naseer, Fahad Shahbaz Khan, Salman Khan

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2023

2401.00849 2024-01-02 cs.CV 57%

COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training

Alex Jinpeng Wang, Linjie Li, Kevin Qinghong Lin, Jianfeng Wang, Kevin Lin, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 16 pages; Website: http://fingerrec.github.io/cosmo

2401.00268 2024-01-02 cs.CV 57%

COMMA: Co-Articulated Multi-Modal Learning

Lianyu Hu, Liqing Gao, Zekang Liu, Chi-Man Pun, Wei Feng

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted to AAAI2024. Code is available at https://github.com/hulianyuyy/COMMA

2310.00426 2024-01-01 cs.CV 57%

PixArt-$α$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, Zhenguo Li

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Project Page: https://pixart-alpha.github.io

2307.04087 2023-12-29 cs.CV 57%

SVIT: Scaling up Visual Instruction Tuning

Bo Zhao, Boya Wu, Muyang He, Tiejun Huang

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

2308.12960 2023-12-27 cs.CV 57%

Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignment

Sheng Zhang, Muzammal Naseer, Guangyi Chen, Zhiqiang Shen, Salman Khan, Kun Zhang, Fahad Khan

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV

Comments AAAI'24

2312.08636 2023-12-15 cs.CV 57%

MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learning

Yi Xin, Junlong Du, Qiang Wang, Ke Yan, Shouhong Ding

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by AAAI2024

2312.06401 2023-12-12 cs.CV 57%

Compound Text-Guided Prompt Tuning via Image-Adaptive Cues

Hao Tan, Jun Li, Yizhuang Zhou, Jun Wan, Zhen Lei, Xiangyu Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 13 pages, 7 figures, accepted to AAAI 2024

2312.04838 2023-12-11 cs.CV 57%

Learning Generalizable Perceptual Representations for Data-Efficient No-Reference Image Quality Assessment

Suhas Srinath, Shankhanil Mitra, Shika Rao, Rajiv Soundararajan

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted to IEEE/CVF WACV 2024

2312.03046 2023-12-11 cs.CV 57%

Diversified in-domain synthesis with efficient fine-tuning for few-shot classification

Victor G. Turrisi da Costa, Nicola Dall'Asen, Yiming Wang, Nicu Sebe, Elisa Ricci

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV

Comments 14 pages, 6 figures, 8 tables

1. VLM训练与架构 5058 篇

VicTR: Video-conditioned Text Representations for Activity Recognition

X-MIC: Cross-Modal Instance Conditioning for Egocentric Action Generalization

Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean

Prompt Highlighter: Interactive Control for Multi-Modal LLMs

LuoJiaHOG: A Hierarchy Oriented Geo-aware Image Caption Dataset for Remote Sensing Image-Text Retrival

TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model

DeepSeek-VL: Towards Real-World Vision-Language Understanding

Sieve: Multimodal Dataset Pruning Using Image Captioning Models

Multimodal Transformer for Comics Text-Cloze

adaptMLLM: Fine-Tuning Multilingual Language Models on Low-Resource Languages with Integrated LLM Playgrounds

Image2Sentence based Asymmetrical Zero-shot Composed Image Retrieval

Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval

TinyLLaVA: A Framework of Small-scale Large Multimodal Models

Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing

Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction

GS-CLIP: Gaussian Splatting for Contrastive Language-Image-3D Pretraining from Real-World Data

Exploring Low-Resource Medical Image Classification with Weakly Supervised Prompt Learning

SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model

COCO is "ALL'' You Need for Visual Instruction Fine-tuning

3DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding

Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization

COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training

COMMA: Co-Articulated Multi-Modal Learning

PixArt-$α$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

SVIT: Scaling up Visual Instruction Tuning

Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignment

MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learning

Compound Text-Guided Prompt Tuning via Image-Adaptive Cues

Learning Generalizable Perceptual Representations for Data-Efficient No-Reference Image Quality Assessment

Diversified in-domain synthesis with efficient fine-tuning for few-shot classification