视觉大模型 / VLM

GiVE: Guiding Visual Encoder to Perceive Overlooked Information

Junjie Li, Jianghong Ma, Xiaofeng Zhang, Yuhang Li, Jianyang Shi

专题命中 VLM训练与架构 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments This paper was accepted by ICME 2025

MMLNB: Multi-Modal Learning for Neuroblastoma Subtyping Classification Assisted with Textual Description Generation

Huangwei Chen, Yifei Chen, Zhenyu Yan, Mingyang Ding, Chenlei Li, Zhu Zhu, Feiwei Qin

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 25 pages, 7 figures

BREEN: Bridge Data-Efficient Encoder-Free Multimodal Learning with Learnable Queries

Tianle Li, Yongming Rao, Winston Hu, Yu Cheng

专题命中 VLM训练与架构 ：InternVL(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search

Yiming Jia, Jiachen Li, Xiang Yue, Bo Li, Ping Nie, Kai Zou, Wenhu Chen

专题命中 VLM训练与架构 ：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments Technical Report

A Two-Step Concept-Based Approach for Enhanced Interpretability and Trust in Skin Lesion Diagnosis

Cristiano Patrício, Luís F. Teixeira, João C. Neves

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Comments Published in the Computational and Structural Biotechnology Journal

Is Your Video Language Model a Reliable Judge?

Ming Liu, Wensheng Zhang

专题命中 VLM训练与架构 ：VLM(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winner

Kejia Chen, Jiawen Zhang, Jiacong Hu, Jiazhen Yang, Jian Lou, Zunlei Feng, Mingli Song

专题命中 VLM训练与架构 ：visual language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Aligning Large Language Models and Geometric Deep Models for Protein Representation

Dong Shu, Bingbing Duan, Kai Guo, Kaixiong Zhou, Jiliang Tang, Mengnan Du

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Comments 37 pages, 10 figures

PinLanding: Content-First Keyword Landing Page Generation via Multi-Modal AI for Web-Scale Discovery

Faye Zhang, Jasmine Wan, Qianyu Cheng, Jinfeng Rao

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI、cs.LG

Improving Adversarial Transferability in MLLMs via Dynamic Vision-Language Alignment Attack

Chenhe Gu, Jindong Gu, Andong Hua, Yao Qin

专题命中 VLM训练与架构 ：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments arXiv admin note: text overlap with arXiv:2403.09766

Object-centric Binding in Contrastive Language-Image Pretraining

Rim Assouel, Pietro Astolfi, Florian Bordes, Michal Drozdzal, Adriana Romero-Soriano

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

LLaSA: Large Language and Structured Data Assistant

Yao Xu, Shizhu He, Jiabei Chen, Zeng Xiangrong, Bingning Wang, Guang Liu, Jun Zhao, Kang Liu

专题命中 VLM训练与架构 ：vision-language model(abstract)；grounding(abstract)；分类 cs.AI、cs.LG

Comments NAACL 2025 Main

PerPO: Perceptual Preference Optimization via Discriminative Rewarding

Zining Zhu, Liang Zhao, Kangheng Lin, Jinze Yang, En Yu, Chenglong Liu, Haoran Wei, Jianjian Sun, Zheng Ge, Xiangyu Zhang

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Personalized Preference Fine-tuning of Diffusion Models

Meihua Dang, Anikait Singh, Linqi Zhou, Stefano Ermon, Jiaming Song

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

OpenLKA: an open dataset of lane keeping assist from market autonomous vehicles

Yuhang Wang, Abdulaziz Alhuraish, Shengming Yuan, Shuyi Wang, Hao Zhou

专题命中 VLM训练与架构 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Baichuan-Omni Technical Report

Yadong Li, Haoze Sun, Mingan Lin, Tianpeng Li, Guosheng Dong, Tao Zhang, Bowen Ding, Wei Song, Zhenglin Cheng, Yuqi Huo, Song Chen, Xu Li, Da Pan, Shusen Zhang, Xin Wu, Zheng Liang, Jun Liu, Tao Zhang, Keer Lu, Yaqi Zhao, Yanjun Shen, Fan Yang, Kaicheng Yu, Tao Lin, Jianhua Xu, Zenan Zhou, Weipeng Chen

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

SCBench: A Sports Commentary Benchmark for Video LLMs

Kuangzhi Ge, Lingjun Chen, Kevin Zhang, Yulin Luo, Tianyu Shi, Liaoyuan Fan, Xiang Li, Guanqun Wang, Shanghang Zhang

专题命中 VLM训练与架构 ：LLaVA(abstract)；InternVL(abstract)；分类 cs.CV、cs.AI

PAR: Prompt-Aware Token Reduction Method for Efficient Large Multimodal Models

Yingen Liu, Fan Wu, Ruihui Li, Zhuo Tang, Kenli Li

专题命中 VLM训练与架构 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 5 figures,3 tables

BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions

Anas Awadalla, Le Xue, Manli Shu, An Yan, Jun Wang, Senthil Purushwalkam, Sheng Shen, Hannah Lee, Oscar Lo, Jae Sung Park, Etash Guha, Silvio Savarese, Ludwig Schmidt, Yejin Choi, Caiming Xiong, Ran Xu

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

MIRAGE: Multimodal Identification and Recognition of Annotations in Indian General Prescriptions

Tavish Mankash, V. S. Chaithanya Kota, Anish De, Praveen Prakash, Kshitij Jadhav

专题命中 VLM训练与架构 ：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 9 figures, 3 tables, submitted to ISBI 2025

BendVLM: Test-Time Debiasing of Vision-Language Embeddings

Walter Gerych, Haoran Zhang, Kimia Hamidieh, Eileen Pan, Maanas Sharma, Thomas Hartvigsen, Marzyeh Ghassemi

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)

Michael Saxon, Fatima Jahara, Mahsa Khoshnoodi, Yujie Lu, Aditya Sharma, William Yang Wang

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Spotlight

Teach Multimodal LLMs to Comprehend Electrocardiographic Images

Ruoqi Liu, Yuelin Bai, Xiang Yue, Ping Zhang

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

ViT3D Alignment of LLaMA3: 3D Medical Image Report Generation

Siyou Li, Beining Xu, Yihao Luo, Dong Nie, Le Zhang

专题命中 VLM训练与架构 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Democratizing MLLMs in Healthcare: TinyLLaVA-Med for Efficient Healthcare Diagnostics in Resource-Constrained Settings

Aya El Mir, Lukelo Thadei Luoga, Boyuan Chen, Muhammad Abdullah Hanif, Muhammad Shafique

专题命中 VLM训练与架构 ：LLaVA(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Decoding Style: Efficient Fine-Tuning of LLMs for Image-Guided Outfit Recommendation with Preference

Najmeh Forouzandehmehr, Nima Farrokhsiar, Ramin Giahi, Evren Korpeoglu, Kannan Achan

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Comments CIKM 2024

Multi-modal Relation Distillation for Unified 3D Representation Learning

Huiqun Wang, Yiping Bao, Panwang Pan, Zeming Li, Xiao Liu, Ruijie Yang, Di Huang

专题命中 VLM训练与架构 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV2024

CROME: Cross-Modal Adapters for Efficient Multimodal LLM

Sayna Ebrahimi, Sercan O. Arik, Tejas Nama, Tomas Pfister

专题命中 VLM训练与架构 ：visual question answering(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

D-Rax: Domain-specific Radiologic assistant leveraging multi-modal data and eXpert model predictions

Hareem Nisar, Syed Muhammad Anwar, Zhifan Jiang, Abhijeet Parida, Ramon Sanchez-Jacob, Vishwesh Nath, Holger R. Roth, Marius George Linguraru

专题命中 VLM训练与架构 ：vision language model(abstract)；LLaVA(abstract)；分类 cs.AI、cs.LG

Comments accepted to the MICCAI 2024 Second International Workshop on Foundation Models for General Medical AI

UMBRAE: Unified Multimodal Brain Decoding

Weihao Xia, Raoul de Charette, Cengiz Öztireli, Jing-Hao Xue

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments ECCV 2024. Project: https://weihaox.github.io/UMBRAE