多模态大模型 - arXivDaily 专题

2503.10665 2025-03-17 cs.CV cs.AI cs.CL cs.LG 67%

Small Vision-Language Models: A Survey on Compact Architectures and Techniques

Nitesh Patnaik, Navdeep Nayak, Himani Bansal Agrawal, Moinak Chinmoy Khamaru, Gourav Bal, Saishree Smaranika Panda, Rishi Raj, Vishal Meena, Kartheek Vadlamani

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2503.09837 2025-03-17 cs.CV cs.AI cs.CL 67%

On the Limitations of Vision-Language Models in Understanding Image Transforms

Ahmad Mustafa Anis, Hasnain Ali, Saquib Sarfraz

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 8 pages, 15 images

2406.12718 2025-03-17 cs.CV cs.AI cs.CL 67%

Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention

Wenbin An, Feng Tian, Sicong Leng, Jiahao Nie, Haonan Lin, QianYing Wang, Ping Chen, Xiaoqin Zhang, Shijian Lu

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by CVPR 2025

2503.04982 2025-03-10 cs.CV cs.AI cs.CL 67%

LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression

Souvik Kundu, Anahita Bhiwandiwalla, Sungduk Yu, Phillip Howard, Tiep Le, Sharath Nittur Sridhar, David Cobbley, Hao Kang, Vasudev Lal

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments This work has been accepted to NAACL 2025

2503.01619 2025-03-04 cs.CV cs.AI cs.CL cs.LG 67%

Advancing vision-language models in front-end development via data synthesis

Tong Ge, Yashu Liu, Jieping Ye, Tianyi Li, Chao Wang

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

2406.08702 2025-02-11 cs.AI cs.CL cs.CV 67%

VLind-Bench: Measuring Language Priors in Large Vision-Language Models

Kang-il Lee, Minbeom Kim, Seunghyun Yoon, Minsung Kim, Dongryeol Lee, Hyukhun Koh, Kyomin Jung

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NAACL 2025 Findings

2502.01524 2025-02-05 cs.CV cs.AI cs.CL cs.LG 67%

Efficiently Integrate Large Language Models with Visual Perception: A Survey from the Training Paradigm Perspective

Xiaorui Ma, Haoran Xie, S. Joe Qin

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 28 pages, 3 figures

2501.16411 2025-01-30 cs.CV cs.AI cs.CL cs.LG cs.RO 67%

PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding

Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, Yue Wang

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ICLR 2025. Project page: https://physbench.github.io/ Dataset: https://huggingface.co/datasets/USC-GVL/PhysBench

2501.16378 2025-01-29 cs.LG cs.AI cs.CL cs.CV 67%

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update

Qing Li, Jiahui Geng, Zongxiong Chen, Kun Song, Lei Ma, Fakhri Karray

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2404.12652 2025-01-15 cs.CV cs.AI cs.CL cs.LG 67%

Pre-trained Vision-Language Models Learn Discoverable Visual Concepts

Yuan Zang, Tian Yun, Hao Tan, Trung Bui, Chen Sun

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Transactions on Machine Learning Research, 2025

2501.05081 2025-01-10 cs.LG 67%

DriVLM: Domain Adaptation of Vision-Language Models in Autonomous Driving

Xuran Zheng, Chang D. Yoo

专题命中图文多模态：multimodal(abstract)；MLLM(abstract)

2501.01282 2025-01-03 cs.AI cs.CL cs.CV 67%

CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries

Shudong Liu, Yiqiao Jin, Cheng Li, Derek F. Wong, Qingsong Wen, Lichao Sun, Haipeng Chen, Xing Xie, Jindong Wang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Technical report; 26 pages

2410.09575 2025-01-03 cs.CV cs.AI cs.CL cs.LG 67%

Reconstructive Visual Instruction Tuning

Haochen Wang, Anlin Zheng, Yucheng Zhao, Tiancai Wang, Zheng Ge, Xiangyu Zhang, Zhaoxiang Zhang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2306.08685 2024-12-30 cs.CL cs.AI cs.CV 67%

World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models

Ziqiao Ma, Jiayi Pan, Joyce Chai

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ACL 2023 Outstanding Paper

2412.18072 2024-12-25 cs.CV cs.AI cs.CL cs.LG 67%

MMFactory: A Universal Solution Search Engine for Vision-Language Tasks

Wan-Cyuan Fan, Tanzila Rahman, Leonid Sigal

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2411.02902 2024-11-06 cs.CV cs.AI cs.CL cs.CR cs.LG 67%

Membership Inference Attacks against Large Vision-Language Models

Zhan Li, Yongtao Wu, Yihang Chen, Francesco Tonin, Elias Abad Rocamora, Volkan Cevher

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NeurIPS 2024

2411.00855 2024-11-05 cs.LG cs.AI cs.CL cs.CV 67%

Vision-Language Models Can Self-Improve Reasoning via Reflection

Kanzhi Cheng, Yantao Li, Fangzhi Xu, Jianbing Zhang, Hao Zhou, Yang Liu

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2410.13611 2024-10-18 cs.CV cs.AI cs.CL cs.LG 67%

H2OVL-Mississippi Vision Language Models Technical Report

Shaikat Galib, Shanshan Wang, Guanshuo Xu, Pascal Pfeiffer, Ryan Chesler, Mark Landry, Sri Satish Ambati

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

2410.08211 2024-10-11 cs.CV cs.AI cs.CL 67%

LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts

Anh-Quan Cao, Maximilian Jaritz, Matthieu Guillaumin, Raoul de Charette, Loris Bazzani

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2405.19567 2024-10-11 cs.AI cs.CL cs.CV cs.LG 67%

Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding

Shenghuan Sun, Alexander Schubert, Gregory M. Goldgof, Zhiqing Sun, Thomas Hartvigsen, Atul J. Butte, Ahmed Alaa

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Code available at: https://github.com/AlaaLab/Dr-LLaVA

2409.12191 2024-10-04 cs.CV cs.AI cs.CL 67%

Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, Junyang Lin

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Code is available at https://github.com/QwenLM/Qwen2-VL. arXiv admin note: text overlap with arXiv:2408.15262 by other authors

2109.04993 2024-10-02 cs.CV cs.AI cs.CL 67%

LAViTeR: Learning Aligned Visual and Textual Representations Assisted by Image and Caption Generation

Mohammad Abuzar Hashemi, Zhanghexuan Li, Mihir Chauhan, Yan Shen, Abhishek Satbhai, Mir Basheer Ali, Mingchen Gao, Sargur Srihari

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 15 pages, 10 Figures, 5 Tables. Accepted for Oral Presentation at Irish Machine Vision and Image Processing Conference Proceedings (IMVIP), 2024

2409.18046 2024-09-27 cs.CV cs.AI cs.CL cs.LG 67%

IFCap: Image-like Retrieval and Frequency-based Entity Filtering for Zero-shot Captioning

Soeun Lee, Si-Woo Kim, Taewhan Kim, Dong-Jin Kim

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to EMNLP 2024

2408.01959 2024-08-29 cs.CV cs.AI cs.CL cs.CY cs.LG 67%

Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI

Robert Wolfe, Aayushi Dangol, Alexis Hiniker, Bill Howe

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at Artificial Intelligence, Ethics, and Society 2024

2408.00550 2024-08-02 cs.CV cs.AI cs.CL 67%

Mitigating Multilingual Hallucination in Large Vision-Language Models

Xiaoye Qu, Mingyang Song, Wei Wei, Jianfeng Dong, Yu Cheng

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2407.15155 2024-07-23 cs.CV cs.AI cs.MM 67%

Distilling Vision-Language Foundation Models: A Data-Free Approach via Prompt Diversification

Yunyi Xuan, Weijie Chen, Shicai Yang, Di Xie, Luojun Lin, Yueting Zhuang

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by ACMMM 2023

2311.17072 2024-07-18 cs.CV cs.AI cs.LG cs.MM 67%

IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers

Chenglin Yang, Siyuan Qiao, Yuan Cao, Yu Zhang, Tao Zhu, Alan Yuille, Jiahui Yu

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments To appear in ECCV 2024

2406.19578 2024-07-01 cs.CV cs.AI cs.CL cs.LG 67%

PathAlign: A vision-language model for whole slide images in histopathology

Faruk Ahmed, Andrew Sellergren, Lin Yang, Shawn Xu, Boris Babenko, Abbi Ward, Niels Olson, Arash Mohtashamian, Yossi Matias, Greg S. Corrado, Quang Duong, Dale R. Webster, Shravya Shetty, Daniel Golden, Yun Liu, David F. Steiner, Ellery Wulczyn

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 9 main pages and 19 pages of supplemental material; 3 main tables, 3 main figures and 11 supplemental tables, 7 supplemental figures

2405.20204 2024-06-27 cs.CL cs.AI cs.CV cs.IR 67%

Jina CLIP: Your CLIP Model Is Also Your Text Retriever

Andreas Koukounas, Georgios Mastrapas, Michael Günther, Bo Wang, Scott Martens, Isabelle Mohr, Saba Sturua, Mohammad Kalim Akram, Joan Fontanals Martínez, Saahil Ognawala, Susana Guzman, Maximilian Werk, Nan Wang, Han Xiao

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 4 pages, MFM-EAI@ICML2024

2308.07706 2024-06-21 cs.CV cs.AI cs.CL cs.LG 67%

Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models

Kanchan Poudel, Manish Dhakal, Prasiddha Bhandari, Rabin Adhikari, Safal Thapaliya, Bishesh Khanal

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Medical Imaging with Deep Learning (MIDL) 2024 (Oral)

1. 图文多模态 4651 篇

Small Vision-Language Models: A Survey on Compact Architectures and Techniques

On the Limitations of Vision-Language Models in Understanding Image Transforms

Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention

LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression

Advancing vision-language models in front-end development via data synthesis

VLind-Bench: Measuring Language Priors in Large Vision-Language Models

Efficiently Integrate Large Language Models with Visual Perception: A Survey from the Training Paradigm Perspective

PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update

Pre-trained Vision-Language Models Learn Discoverable Visual Concepts

DriVLM: Domain Adaptation of Vision-Language Models in Autonomous Driving

CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries

Reconstructive Visual Instruction Tuning

World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models

MMFactory: A Universal Solution Search Engine for Vision-Language Tasks

Membership Inference Attacks against Large Vision-Language Models

Vision-Language Models Can Self-Improve Reasoning via Reflection

H2OVL-Mississippi Vision Language Models Technical Report

LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts

Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding

Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

LAViTeR: Learning Aligned Visual and Textual Representations Assisted by Image and Caption Generation

IFCap: Image-like Retrieval and Frequency-based Entity Filtering for Zero-shot Captioning

Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI

Mitigating Multilingual Hallucination in Large Vision-Language Models

Distilling Vision-Language Foundation Models: A Data-Free Approach via Prompt Diversification

IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers

PathAlign: A vision-language model for whole slide images in histopathology

Jina CLIP: Your CLIP Model Is Also Your Text Retriever

Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models