多模态大模型 - arXivDaily 专题

2307.05591 2025-02-11 cs.CV cs.CL cs.LG 73%

Linear Alignment of Vision-language Models for Image Captioning

Fabian Paischer, Markus Hofmarcher, Sepp Hochreiter, Thomas Adler

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 9 pages (+ references and appendix)

2502.04470 2025-02-10 cs.CV cs.AI 73%

Color in Visual-Language Models: CLIP deficiencies

Guillem Arias, Ramon Baldrich, Maria Vanrell

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 6 pages, 10 figures, conference, Artificial Intelligence

Journal ref in Color and Imaging Conference, 2024, pp 101 - 106

2305.05208 2025-02-10 cs.CV cs.CL 73%

Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data

Haonan Wang, Minbin Huang, Runhui Huang, Lanqing Hong, Hang Xu, Tianyang Hu, Xiaodan Liang, Zhenguo Li, Hong Cheng, Kenji Kawaguchi

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to NAACL 2025, main conference. 20 pages, 10 figures, 10 tables

2501.08816 2025-01-22 cs.CV cs.AI cs.LG 73%

IDEA: Image Description Enhanced CLIP-Adapter

Zhipeng Ye, Feng Jiang, Qiufeng Wang, Kaizhu Huang, Jiaqi Huang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

2412.11863 2025-01-13 cs.CV cs.CL 73%

GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training

Renqiu Xia, Mingsheng Li, Hancheng Ye, Wenjie Wu, Hongbin Zhou, Jiakang Yuan, Tianshuo Peng, Xinyu Cai, Xiangchao Yan, Bin Wang, Conghui He, Botian Shi, Tao Chen, Junchi Yan, Bo Zhang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Our code is available at https://github.com/Alpha-Innovator/GeoX

2410.08405 2025-01-10 cs.CV cs.AI 73%

AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning

Muhammad Awais, Ali Husain Salem Abdulla Alharthi, Amandeep Kumar, Hisham Cholakkal, Rao Muhammad Anwer

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted at WACV, 2025

2501.02461 2025-01-07 cs.CV cs.AI 73%

FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models

Hui Lin, Chao Zhang, Danfeng Hong, Kexin Dong, Congcong Wen

专题命中图文多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

2412.17601 2024-12-30 cs.CV cs.AI 73%

AFANet: Adaptive Frequency-Aware Network for Weakly-Supervised Few-Shot Semantic Segmentation

Jiaqi Ma, Guo-Sen Xie, Fang Zhao, Zechao Li

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by TMM 2024

2412.04424 2024-12-06 cs.CV cs.AI 73%

Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusion

Jiuhai Chen, Jianwei Yang, Haiping Wu, Dianqi Li, Jianfeng Gao, Tianyi Zhou, Bin Xiao

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

2412.03561 2024-12-05 cs.CV cs.AI 73%

FLAIR: VLM with Fine-grained Language-informed Image Representations

Rui Xiao, Sanghwan Kim, Mariana-Iuliana Georgescu, Zeynep Akata, Stephan Alaniz

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

2405.13800 2024-11-18 cs.CV cs.AI 73%

Dense Connector for MLLMs

Huanjin Yao, Wenhao Wu, Taojiannan Yang, YuXin Song, Mengxi Zhang, Haocheng Feng, Yifan Sun, Zhiheng Li, Wanli Ouyang, Jingdong Wang

专题命中图文多模态：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 27 pages, NeurIPS 2024

Journal ref NeurIPS 2024

2411.07461 2024-11-13 cs.CV cs.AI 73%

BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions

Anas Awadalla, Le Xue, Manli Shu, An Yan, Jun Wang, Senthil Purushwalkam, Sheng Shen, Hannah Lee, Oscar Lo, Jae Sung Park, Etash Guha, Silvio Savarese, Ludwig Schmidt, Yejin Choi, Caiming Xiong, Ran Xu

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

2411.03034 2024-11-06 cs.AI cs.MM 73%

HumanVLM: Foundation for Human-Scene Vision-Language Model

Dawei Dai, Xu Long, Li Yutang, Zhang Yuanhui, Shuyin Xia

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.AI、cs.MM

Comments 34 pages,11 figures

2405.13777 2024-10-25 cs.CV cs.AI 73%

No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models

Angéline Pouget, Lucas Beyer, Emanuele Bugliarello, Xiao Wang, Andreas Peter Steiner, Xiaohua Zhai, Ibrahim Alabdulmohsin

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 17 pages, 5 figures, 4 tables. 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

2403.14774 2024-10-24 cs.CV cs.CL cs.CR cs.LG 73%

Few-Shot Adversarial Prompt Learning on Vision-Language Models

Yiwei Zhou, Xiaobo Xia, Zhiwei Lin, Bo Han, Tongliang Liu

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments NeurIPS 2024

2410.13010 2024-10-18 cs.LG cs.AI cs.CR cs.CV 73%

Hiding-in-Plain-Sight (HiPS) Attack on CLIP for Targetted Object Removal from Images

Arka Daw, Megan Hong-Thanh Chung, Maria Mahbub, Amir Sadovnik

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Published in the 3rd Workshop on New Frontiers in Adversarial Machine Learning at NeurIPS 2024. 10 pages, 7 figures, 3 tables

2409.13980 2024-09-24 cs.CV cs.AI 73%

Enhancing Advanced Visual Reasoning Ability of Large Language Models

Zhiyuan Li, Dongnan Liu, Chaoyi Zhang, Heng Wang, Tengfei Xue, Weidong Cai

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2024 Main

2409.10921 2024-09-18 cs.CV cs.AI 73%

KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graph

Yanbei Jiang, Krista A. Ehinger, Jey Han Lau

专题命中图文多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at IJCAI 2024

2409.06726 2024-09-12 cs.CV cs.AI cs.LG 73%

Feedback-based Modal Mutual Search for Attacking Vision-Language Pre-training Models

Renhua Ding, Xinze Zhang, Xiao Yang, Kun He

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 9 pages

2312.01758 2024-09-04 cs.CV cs.AI 73%

CILF-CIAE: CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation

Yuntao Shou, Wei Ai, Tao Meng, Nan Yin, Keqin Li

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 14 figures, 3 tables

2308.03729 2024-08-13 cs.CV cs.AI 73%

TinyLVLM-eHub: Towards Comprehensive and Efficient Evaluation for Large Vision-Language Models

Wenqi Shao, Meng Lei, Yutao Hu, Peng Gao, Kaipeng Zhang, Fanqing Meng, Peng Xu, Siyuan Huang, Hongsheng Li, Yu Qiao, Ping Luo

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments accepted to IEEE Transactions on Big Data. Project Page: http://lvlm-ehub.opengvlab.com/

2407.19679 2024-07-30 cs.CV cs.AI 73%

Harnessing Large Vision and Language Models in Agriculture: A Review

Hongyan Zhu, Shuai Qin, Min Su, Chengzhi Lin, Anjie Li, Junfeng Gao

专题命中图文多模态：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

2403.17839 2024-07-26 cs.CV cs.AI 73%

ReMamber: Referring Image Segmentation with Mamba Twister

Yuhuan Yang, Chaofan Ma, Jiangchao Yao, Zhun Zhong, Ya Zhang, Yanfeng Wang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments ECCV 2024

2407.08966 2024-07-15 cs.CV cs.AI cs.LG 73%

LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models

Yabin Zhang, Wenjie Zhu, Chenhang He, Lei Zhang

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments ECCV2024; Codes and Supp. are available at: https://github.com/YBZh/LAPT

2406.10902 2024-06-18 cs.CV cs.CL 73%

Light Up the Shadows: Enhance Long-Tailed Entity Grounding with Concept-Guided Vision-Language Models

Yikai Zhang, Qianyu He, Xintao Wang, Siyu Yuan, Jiaqing Liang, Yanghua Xiao

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

2405.05524 2024-05-11 cs.CV cs.MM 73%

Universal Adversarial Perturbations for Vision-Language Pre-trained Models

Peng-Fei Zhang, Zi Huang, Guangdong Bai

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments 9 pages, 5 figures

2404.15946 2024-04-25 cs.CV cs.AI eess.IV 73%

Mammo-CLIP: Leveraging Contrastive Language-Image Pre-training (CLIP) for Enhanced Breast Cancer Diagnosis with Multi-view Mammography

Xuxin Chen, Yuheng Li, Mingzhe Hu, Ella Salari, Xiaoqian Chen, Richard L. J. Qiu, Bin Zheng, Xiaofeng Yang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

2202.09061 2024-04-19 cs.CV cs.CL 73%

VLP: A Survey on Vision-Language Pre-training

Feilong Chen, Duzhen Zhang, Minglun Han, Xiuyi Chen, Jing Shi, Shuang Xu, Bo Xu

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

Comments A Survey on Vision-Language Pre-training

Journal ref Machine Intelligence Research. 20(1), February 2023, 38-56

2305.07490 2024-04-08 cs.CL cs.CV 73%

ArtGPT-4: Towards Artistic-understanding Large Vision-Language Models with Enhanced Adapter

Zhengqing Yuan, Yunhong He, Kun Wang, Yanfang Ye, Lichao Sun

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL

2404.01156 2024-04-02 cs.CV cs.AI 73%

SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

Chull Hwan Song, Taebaek Hwang, Jooyoung Yoon, Shunghyun Choi, Yeong Hyeon Gu

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV、cs.AI

Comments CVPR2024 Accepted

1. 图文多模态 4651 篇

Linear Alignment of Vision-language Models for Image Captioning

Color in Visual-Language Models: CLIP deficiencies

Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data

IDEA: Image Description Enhanced CLIP-Adapter

GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training

AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning

FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models

AFANet: Adaptive Frequency-Aware Network for Weakly-Supervised Few-Shot Semantic Segmentation

Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusion

FLAIR: VLM with Fine-grained Language-informed Image Representations

Dense Connector for MLLMs

BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions

HumanVLM: Foundation for Human-Scene Vision-Language Model

No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models

Few-Shot Adversarial Prompt Learning on Vision-Language Models

Hiding-in-Plain-Sight (HiPS) Attack on CLIP for Targetted Object Removal from Images

Enhancing Advanced Visual Reasoning Ability of Large Language Models

KALE: An Artwork Image Captioning System Augmented with Heterogeneous Graph

Feedback-based Modal Mutual Search for Attacking Vision-Language Pre-training Models

CILF-CIAE: CLIP-driven Image-Language Fusion for Correcting Inverse Age Estimation

TinyLVLM-eHub: Towards Comprehensive and Efficient Evaluation for Large Vision-Language Models

Harnessing Large Vision and Language Models in Agriculture: A Review

ReMamber: Referring Image Segmentation with Mamba Twister

LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models

Light Up the Shadows: Enhance Long-Tailed Entity Grounding with Concept-Guided Vision-Language Models

Universal Adversarial Perturbations for Vision-Language Pre-trained Models

Mammo-CLIP: Leveraging Contrastive Language-Image Pre-training (CLIP) for Enhanced Breast Cancer Diagnosis with Multi-view Mammography

VLP: A Survey on Vision-Language Pre-training

ArtGPT-4: Towards Artistic-understanding Large Vision-Language Models with Enhanced Adapter

SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining