视觉大模型 / VLM

Multimodal large language model for wheat breeding: a new exploration of smart breeding

Guofeng Yang, Yu Li, Yong He, Zhenjiang Zhou, Lingzhen Ye, Hui Fang, Yiqi Luo, Xuping Feng

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)；InternVL(abstract)；分类 cs.AI、cs.LG

MpoxVLM: A Vision-Language Model for Diagnosing Skin Lesions from Mpox Virus Infection

Xu Cao, Wenqian Ye, Kenny Moise, Megan Coffee

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ML4H 2024

RaVL: Discovering and Mitigating Spurious Correlations in Fine-Tuned Vision-Language Models

Maya Varma, Jean-Benoit Delbrouck, Zhihong Chen, Akshay Chaudhari, Curtis Langlotz

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024

Fine-Tuning Vision-Language Model for Automated Engineering Drawing Information Extraction

Muhammad Tayyab Khan, Lequn Chen, Ye Han Ng, Wenhe Feng, Nicholas Yew Jin Tan, Seung Ki Moon

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Paper has been submitted to the 9th International Conference on Innovation in Artificial Intelligence (ICIAI 2025)

VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration

Dezhan Tu, Danylo Vashchilenko, Yuzhe Lu, Panpan Xu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models

Ziyu Liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Conghui He, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Project URL: https://github.com/Liuziyu77/MIA-DPO

MI-VisionShot: Few-shot adaptation of vision-language models for slide-level classification of histopathological images

Pablo Meseguer, Rocío del Amor, Valery Naranjo

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Manuscript accepted for oral presentation at KES-InnovationInMedicine 2024 held on Madeira, Portugal

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers

Yuxin Wen, Qingqing Cao, Qichen Fu, Sachin Mehta, Mahyar Najibi

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations

Neale Ratzlaff, Matthew Lyle Olson, Musashi Hinck, Shao-Yen Tseng, Vasudev Lal, Phillip Howard

专题命中 VLM训练与架构 ：vision-language model(title)；vision language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS workshop on SafeGenAI, 10 pages, 2 figures

VisualRWKV-HD and UHD: Advancing High-Resolution Processing for Visual Language Models

Zihang Li, Haowen Hou

专题命中 VLM训练与架构 ：visual language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models

Yuancheng Xu, Jiarui Yao, Manli Shu, Yanchao Sun, Zichu Wu, Ning Yu, Tom Goldstein, Furong Huang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments Accepted by Thirty-Eighth Annual Conference on Neural Information Processing Systems (Neurips 2024)

Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Models

Juseong Jin, Chang Wook Jeong

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 AIM-FM Workshop

Can Vision-Language Models Replace Human Annotators: A Case Study with CelebA Dataset

Haoming Lu, Feifei Zhong

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024 Workshop (EvalEval 2024)

Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models

Qin Liu, Chao Shang, Ling Liu, Nikolaos Pappas, Jie Ma, Neha Anna John, Srikanth Doss, Lluis Marquez, Miguel Ballesteros, Yassine Benajiba

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.AI、cs.LG

Comments Preprint

Concept-skill Transferability-based Data Selection for Large Vision-Language Models

Jaewoo Lee, Boyang Li, Sung Ju Hwang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.LG

Comments EMNLP 2024

Attention Prompting on Image for Large Vision-Language Models

Runpeng Yu, Weihao Yu, Xinchao Wang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Website, see https://yu-rp.github.io/api-prompting

Blox-Net: Generative Design-for-Robot-Assembly Using VLM Supervision, Physics Simulation, and a Robot with Reset

Andrew Goldberg, Kavish Kondap, Tianshuang Qiu, Zehan Ma, Letian Fu, Justin Kerr, Huang Huang, Kaiyuan Chen, Kuan Fang, Ken Goldberg

专题命中 VLM训练与架构 ：VLM(title,abstract)；vision language model(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 7 Figures

KALIE: Fine-Tuning Vision-Language Models for Open-World Manipulation without Robot Data

Grace Tang, Swetha Rajkumar, Yifei Zhou, Homer Rich Walke, Sergey Levine, Kuan Fang

专题命中 VLM训练与架构 ：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Comments 8 pages, 7 figures

TG-LLaVA: Text Guided LLaVA via Learnable Latent Embeddings

Dawei Yan, Pengcheng Li, Yang Li, Hao Chen, Qingguo Chen, Weihua Luo, Wei Dong, Qingsen Yan, Haokui Zhang, Chunhua Shen

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；vision-language model(abstract)；分类 cs.CV、cs.AI

Do Pre-trained Vision-Language Models Encode Object States?

Kaleb Newman, Shijie Wang, Yuan Zang, David Heffren, Chen Sun

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing

Zilun Zhang, Tiancheng Zhao, Yulong Guo, Jianwei Yin

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments RS5M dataset v5

Journal ref TGRS-2023-06174

An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models

Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, Baobao Chang

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ECCV 2024 (Oral), code is released at https://github.com/pkunlp-icler/FastV,

Building and better understanding vision-language models: insights and future directions

Hugo Laurençon, Andrés Marafioti, Victor Sanh, Léo Tronchon

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learning

Jiajie Li, Garrett Skinner, Gene Yang, Brian R Quaranto, Steven D Schwaitzberg, Peter C W Kim, Jinjun Xiong

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

GP-VLS: A general-purpose vision language model for surgery

Samuel Schmidgall, Joseph Cho, Cyril Zakka, William Hiesinger

专题命中 VLM训练与架构 ：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Reinforcement Learning Friendly Vision-Language Model for Minecraft

Haobin Jiang, Junpeng Yue, Hao Luo, Ziluo Ding, Zongqing Lu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments ECCV 2024

Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-Language Models

Ivan Kapelyukh, Yifei Ren, Ignacio Alzugaray, Edward Johns

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments ICRA 2024. Project webpage with robot videos: https://www.robot-learning.uk/dream2real

Noise-Tolerant Few-Shot Unsupervised Adapter for Vision-Language Models

Eman Ali, Muhammad Haris Khan

专题命中 VLM训练与架构 ：vision-language model(title,abstract)；vision language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted at BMVC 2024

LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models

Ruiyi Zhang, Yufan Zhou, Jian Chen, Jiuxiang Gu, Changyou Chen, Tong Sun

专题命中 VLM训练与架构 ：LLaVA(title,abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Under Review

Vision Language Model-Empowered Contract Theory for AIGC Task Allocation in Teleoperation

Zijun Zhan, Yaxian Dong, Yuqing Hu, Shuai Li, Shaohua Cao, Zhu Han

专题命中 VLM训练与架构 ：vision language model(title,abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 10 figures