视觉大模型 / VLM - arXivDaily 专题

2404.10757 2025-03-18 astro-ph.IM astro-ph.SR cs.CL cs.LG 70%

Deep Learning and LLM-based Methods Applied to Stellar Lightcurve Classification

Yu-Yang Li, Yu Bai, Cunshi Wang, Mengwei Qu, Ziteng Lu, Roberto Soria, Jifeng Liu

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.LG

Comments 35 pages, 20 figures

Journal ref Intell Comput. 2025;4:0110

2503.10731 2025-03-17 cs.CV 70%

Leveraging Vision-Language Embeddings for Zero-Shot Learning in Histopathology Images

Md Mamunur Rahaman, Ewan K. A. Millar, Erik Meijering

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2408.12928 2025-03-17 cs.CV 70%

ParGo: Bridging Vision-Language with Partial and Global Views

An-Lan Wang, Bin Shan, Wei Shi, Kun-Yu Lin, Xiang Fei, Guozhi Tang, Lei Liao, Can Huang, Jingqun Tang, Wei-Shi Zheng

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by AAAI 2025

2503.10322 2025-03-14 cs.CV 70%

Towards Fast, Memory-based and Data-Efficient Vision-Language Policy

Haoxuan Li, Sixu Yan, Yuhan Li, Xinggang Wang

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments 11 pages, 7 figures, 6 tables

2503.09871 2025-03-14 cs.CV 70%

LuciBot: Automated Robot Policy Learning from Generated Videos

Xiaowen Qiu, Yian Wang, Jiting Cai, Zhehuan Chen, Chunru Lin, Tsun-Hsuan Wang, Chuang Gan

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2503.07906 2025-03-12 cs.CV 70%

Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning

Qinghao Ye, Xianhan Zeng, Fu Li, Chunyuan Li, Haoqi Fan

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted by ICLR 2025

2411.15922 2025-03-12 eess.IV cs.CV 70%

PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation

Chia-Ming Lee, Ching-Heng Cheng, Yu-Fan Lin, Yi-Ching Cheng, Wo-Ting Liao, Fu-En Yang, Yu-Chiang Frank Wang, Chih-Chung Hsu

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Project page: https://chingheng0808.github.io/prompthsiP/static.html

2409.12319 2025-03-10 cs.CV cs.MM cs.SD eess.AS 70%

Large Language Models are Strong Audio-Visual Speech Recognition Learners

Umberto Cappellazzo, Minsu Kim, Honglie Chen, Pingchuan Ma, Stavros Petridis, Daniele Falavigna, Alessio Brutti, Maja Pantic

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted for publication at ICASSP 2025. The code and checkpoints are available here: https://github.com/umbertocappellazzo/Llama-AVSR

2503.04417 2025-03-07 cs.AI cs.MA 70%

From Idea to CAD: A Language Model-Driven Multi-Agent System for Collaborative Design

Felix Ocker, Stefan Menzel, Ahmed Sadik, Thiago Rios

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.AI

Comments 11 pages, 3 figures

2502.20035 2025-02-28 cs.CV 70%

AsymLoRA: Harmonizing Data Conflicts and Commonalities in MLLMs

Xuyang Wei, Chunlin Tian, Li Li

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2406.05127 2025-02-27 cs.CV 70%

Towards Semantic Equivalence of Tokenization in Multimodal LLM

Shengqiong Wu, Hao Fei, Xiangtai Li, Jiayi Ji, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments ICLR-2025. The project page: https://chocowu.github.io/SeTok-web/

2403.20271 2025-02-25 cs.CV 70%

Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want

Weifeng Lin, Xinyu Wei, Ruichuan An, Peng Gao, Bocheng Zou, Yulin Luo, Siyuan Huang, Shanghang Zhang, Hongsheng Li

专题命中 VLM训练与架构：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments 30 pages, 8 figures, 15 tables

2502.09057 2025-02-14 cs.CV 70%

Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model

Shiryu Ueno, Yoshikazu Hayashi, Shunsuke Nakatsuka, Yusei Yamada, Hiroaki Aizawa, Kunihito Kato

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments VISAPP 2025

2502.08391 2025-02-13 cs.CV 70%

ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification

Jiangbo Shi, Chen Li, Tieliang Gong, Yefeng Zheng, Huazhu Fu

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.CV

Comments CVPR 2024 (Updated version with corrections for typos and errors.)

2502.07183 2025-02-13 cs.RO cs.CV 70%

Space-Aware Instruction Tuning: Dataset and Benchmark for Guide Dog Robots Assisting the Visually Impaired

ByungOk Han, Woo-han Yun, Beom-Su Seo, Jaehong Kim

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments ICRA 2025

2403.19622 2025-02-04 cs.RO cs.CV 70%

RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents

Zeren Chen, Zhelun Shi, Xiaoya Lu, Lehan He, Sucheng Qian, Zhenfei Yin, Wanli Ouyang, Jing Shao, Yu Qiao, Cewu Lu, Lu Sheng

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments 18 pages, 11 figures, 7 tables. Accepted by NeurIPS 2024 Workshop

2408.12109 2025-02-03 cs.CV cs.CL 70%

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data

Chenglong Wang, Yang Gan, Yifu Huo, Yongyu Mu, Murun Yang, Qiaozhi He, Tong Xiao, Chunliang Zhang, Tongran Liu, Quan Du, Di Yang, Jingbo Zhu

专题命中 VLM训练与架构：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

Comments Accepted by AAAI 2025

2501.16629 2025-01-29 cs.CL cs.CV 70%

CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs

Jinlan Fu, Shenzhen Huangfu, Hao Fei, Xiaoyu Shen, Bryan Hooi, Xipeng Qiu, See-Kiong Ng

专题命中 VLM训练与架构：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Accepted by ICLR 2025

2404.14047 2025-01-14 cs.LG 70%

An empirical study of LLaMA3 quantization: from LLMs to MLLMs

Wei Huang, Xingyu Zheng, Xudong Ma, Haotong Qin, Chengtao Lv, Hong Chen, Jie Luo, Xiaojuan Qi, Xianglong Liu, Michele Magno

专题命中 VLM训练与架构：LLaVA(abstract)；MLLM(abstract)；分类 cs.LG

2408.16228 2025-01-09 cs.RO cs.LG 70%

Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation

Vivek Myers, Bill Chunyuan Zheng, Oier Mees, Sergey Levine, Kuan Fang

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.LG

Comments 27 pages, 14 figures

Journal ref Conference on Robot Learning, 2024

2403.13352 2025-01-03 cs.CV 70%

AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation

Jingkun An, Yinghao Zhu, Zongjian Li, Enshen Zhou, Haoran Feng, Xijie Huang, Bohua Chen, Yemin Shi, Chengwei Pan

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted by AAAI-2025

2412.16211 2024-12-24 cs.CV cs.CL cs.GR 70%

Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation

Yiping Wang, Xuehai He, Kuan Wang, Luyao Ma, Jianwei Yang, Shuohang Wang, Simon Shaolei Du, Yelong Shen

专题命中 VLM训练与架构：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

Comments benchmark paper, project page: https://ypwang61.github.io/project/StoryEval

2406.04295 2024-12-17 cs.CV 70%

Everything to the Synthetic: Diffusion-driven Test-time Adaptation via Synthetic-Domain Alignment

Jiayi Guo, Junhao Zhao, Chaoqun Du, Yulin Wang, Chunjiang Ge, Zanlin Ni, Shiji Song, Humphrey Shi, Gao Huang

专题命中 VLM训练与架构：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments GitHub: https://github.com/SHI-Labs/Diffusion-Driven-Test-Time-Adaptation-via-Synthetic-Domain-Alignment

2412.10372 2024-12-16 cs.CV 70%

UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities

Muhammad Uzair Khattak, Shahina Kunhimon, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Code, models and demo available at https://github.com/mbzuai-oryx/UniMed-CLIP

2412.09618 2024-12-13 cs.CV 70%

EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM

Zhuofan Zong, Dongzhi Jiang, Bingqi Ma, Guanglu Song, Hao Shao, Dazhong Shen, Yu Liu, Hongsheng Li

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments Tech report

2412.09604 2024-12-13 cs.CV 70%

SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding

Hao Li, Changyao Tian, Jie Shao, Xizhou Zhu, Zhaokai Wang, Jinguo Zhu, Wenhan Dou, Xiaogang Wang, Hongsheng Li, Lewei Lu, Jifeng Dai

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2412.07112 2024-12-11 cs.CV cs.CL 70%

Maya: An Instruction Finetuned Multilingual Multimodal Model

Nahid Alam, Karthik Reddy Kanjula, Surya Guthikonda, Timothy Chung, Bala Krishna S Vegesna, Abhipsha Das, Anthony Susevski, Ryan Sze-Yin Chan, S M Iftekhar Uddin, Shayekh Bin Islam, Roshan Santhosh, Snegha A, Drishti Sharma, Chen Liu, Isha Chaturvedi, Genta Indra Winata, Ashvanth. S, Snehanshu Mukherjee, Alham Fikri Aji

专题命中 VLM训练与架构：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

2412.06673 2024-12-10 cs.CV 70%

ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhance

Chunwei Wang, Guansong Lu, Junwei Yang, Runhui Huang, Jianhua Han, Lu Hou, Wei Zhang, Hang Xu

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2412.03555 2024-12-05 cs.CV 70%

PaliGemma 2: A Family of Versatile VLMs for Transfer

Andreas Steiner, André Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Gritsenko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, Siyang Qin, Reeve Ingle, Emanuele Bugliarello, Sahar Kazemzadeh, Thomas Mesnard, Ibrahim Alabdulmohsin, Lucas Beyer, Xiaohua Zhai

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2412.02531 2024-12-04 cs.CV 70%

Multimodal Remote Sensing Scene Classification Using VLMs and Dual-Cross Attention Networks

Jinjin Cai, Kexin Meng, Baijian Yang, Gang Shao

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

1. VLM训练与架构 5048 篇

Deep Learning and LLM-based Methods Applied to Stellar Lightcurve Classification

Leveraging Vision-Language Embeddings for Zero-Shot Learning in Histopathology Images

ParGo: Bridging Vision-Language with Partial and Global Views

Towards Fast, Memory-based and Data-Efficient Vision-Language Policy

LuciBot: Automated Robot Policy Learning from Generated Videos

Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning

PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation

Large Language Models are Strong Audio-Visual Speech Recognition Learners

From Idea to CAD: A Language Model-Driven Multi-Agent System for Collaborative Design

AsymLoRA: Harmonizing Data Conflicts and Commonalities in MLLMs

Towards Semantic Equivalence of Tokenization in Multimodal LLM

Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want

Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model

ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification

Space-Aware Instruction Tuning: Dataset and Benchmark for Guide Dog Robots Assisting the Visually Impaired

RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data

CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs

An empirical study of LLaMA3 quantization: from LLMs to MLLMs

Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation

AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation

Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation

Everything to the Synthetic: Diffusion-driven Test-time Adaptation via Synthetic-Domain Alignment

UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities

EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM

SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding

Maya: An Instruction Finetuned Multilingual Multimodal Model

ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhance

PaliGemma 2: A Family of Versatile VLMs for Transfer

Multimodal Remote Sensing Scene Classification Using VLMs and Dual-Cross Attention Networks