视觉大模型 / VLM - arXivDaily 专题

2403.18525 2024-03-28 cs.CV cs.CL cs.LG 62%

Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP

Reza Abbasi, Mohammad Samiei, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Oral accepted at OODCV 2023(http://www.ood-cv.org)

2403.12910 2024-03-20 cs.RO cs.AI cs.LG 62%

Yell At Your Robot: Improving On-the-Fly from Language Corrections

Lucy Xiaoyang Shi, Zheyuan Hu, Tony Z. Zhao, Archit Sharma, Karl Pertsch, Jianlan Luo, Sergey Levine, Chelsea Finn

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments Project website: https://yay-robot.github.io/

2311.07593 2024-03-18 cs.CL cs.CV cs.LG 62%

Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification

Reza Esfandiarpoor, Stephen H. Bach

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments ICLR 2024

2403.09410 2024-03-15 cs.CV cs.AI 62%

XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization

Yequan Bie, Luyang Luo, Zhixuan Chen, Hao Chen

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

2311.15108 2024-02-13 cs.CV cs.AI 62%

Leveraging Diffusion Perturbations for Measuring Fairness in Computer Vision

Nicholas Lui, Bryan Chia, William Berrios, Candace Ross, Douwe Kiela

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments The Appendix can be found at https://bit.ly/dp-appendix; Added link to code and fixed formatting (Feb 10 2024)

2402.02662 2024-02-07 cs.CV cs.CL cs.LG 62%

Image-Caption Encoding for Improving Zero-Shot Generalization

Eric Yang Yu, Christopher Liao, Sathvik Ravi, Theodoros Tsiligkaridis, Brian Kulis

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2401.02600 2024-01-08 cs.CV cs.AI 62%

Object-oriented backdoor attack against image captioning

Meiling Li, Nan Zhong, Xinpeng Zhang, Zhenxing Qian, Sheng Li

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

2308.06595 2023-12-27 cs.CL cs.AI cs.CV 62%

VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use

Yonatan Bitton, Hritik Bansal, Jack Hessel, Rulin Shao, Wanrong Zhu, Anas Awadalla, Josh Gardner, Rohan Taori, Ludwig Schmidt

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2023, Datasets and Benchmarks. Website: https://visit-bench.github.io/

2303.06854 2023-12-21 cs.CV cs.CL cs.CR cs.LG 62%

Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks

Wenhan Yang, Jingdong Gao, Baharan Mirzasoleiman

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2312.12383 2023-12-20 cs.AI cs.CL cs.CV 62%

Visual AI and Linguistic Intelligence Through Steerability and Composability

David Noever, Samantha Elizabeth Miller Noever

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

2312.08958 2023-12-15 cs.LG cs.AI cs.RO 62%

LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers

Taewook Nam, Juyong Lee, Jesse Zhang, Sung Ju Hwang, Joseph J. Lim, Karl Pertsch

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments 2nd Workshop on Agent Learning in Open-Endedness (ALOE) at NeurIPS 2023

2312.08878 2023-12-15 cs.CV cs.LG stat.AP 62%

Domain Prompt Learning with Quaternion Networks

Qinglong Cao, Zhengqin Xu, Yuntian Chen, Chao Ma, Xiaokang Yang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2308.03793 2023-12-15 cs.CV cs.LG 62%

ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

Xuefeng Hu, Ke Zhang, Lu Xia, Albert Chen, Jiajia Luo, Yuyin Sun, Ken Wang, Nan Qiao, Xiao Zeng, Min Sun, Cheng-Hao Kuo, Ram Nevatia

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted as Oral Paper by 2024 IEEE CVF Winter Conference on Applications of Computer Vision (WACV)

2307.00910 2023-12-13 cs.CV cs.AI 62%

CoPL: Contextual Prompt Learning for Vision-Language Understanding

Koustava Goswami, Srikrishna Karanam, Prateksha Udhayanan, K J Joseph, Balaji Vasan Srinivasan

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at AAAI 2024

2311.13258 2023-11-23 cs.CV cs.CL cs.LG 62%

ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation

Yangyi Chen, Xingyao Wang, Manling Li, Derek Hoiem, Heng Ji

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to EMNLP 2023

2307.10350 2023-10-27 cs.LG cs.CV 62%

Improving Multimodal Datasets with Image Captioning

Thao Nguyen, Samir Yitzhak Gadre, Gabriel Ilharco, Sewoong Oh, Ludwig Schmidt

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted at NeurIPS 2023 Datasets & Benchmarks

2305.14095 2023-10-26 cs.CV cs.LG 62%

S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions

Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jinwoo Shin

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2023

2305.01278 2023-10-25 cs.CV cs.AI cs.CL 62%

VPGTrans: Transfer Visual Prompt Generator across LLMs

Ao Zhang, Hao Fei, Yuan Yao, Wei Ji, Li Li, Zhiyuan Liu, Tat-Seng Chua

专题命中其他VLM ：MLLM(abstract)；分类 cs.CV、cs.AI

Comments Project Website: https://vpgtrans.github.io Code: https://github.com/VPGTrans/VPGTrans NeurIPS 2023

2310.03957 2023-10-09 cs.LG cs.CV 62%

Understanding prompt engineering may not require rethinking generalization

Victor Akinwande, Yiding Jiang, Dylan Sam, J. Zico Kolter

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2307.10504 2023-09-11 cs.CV cs.LG 62%

Identifying Interpretable Subspaces in Image Representations

Neha Kalibhat, Shweta Bhardwaj, Bayan Bruss, Hamed Firooz, Maziar Sanjabi, Soheil Feizi

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Published at ICML 2023 Code: https://github.com/NehaKalibhat/falcon-explain

2306.07282 2023-08-21 cs.CV cs.LG 62%

Waffling around for Performance: Visual Classification with Random Words and Broad Concepts

Karsten Roth, Jae Myung Kim, A. Sophia Koepke, Oriol Vinyals, Cordelia Schmid, Zeynep Akata

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to ICCV 2023. Main paper with 9 pages

2206.02779 2023-07-27 cs.CV cs.GR cs.LG 62%

Blended Latent Diffusion

Omri Avrahami, Ohad Fried, Dani Lischinski

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to SIGGRAPH 2023. Project page: https://omriavrahami.com/blended-latent-diffusion-page/

2307.13681 2023-07-26 cs.GR cs.AI cs.CV 62%

The Visual Language of Fabrics

Valentin Deschaintre, Julia Guerrero-Viu, Diego Gutierrez, Tamy Boubekeur, Belen Masia

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Journal ref ACM Transactions on Graphics 2023

2307.01981 2023-07-06 eess.IV cs.CV cs.LG 62%

A ChatGPT Aided Explainable Framework for Zero-Shot Medical Image Diagnosis

Jiaxiang Liu, Tianxiang Hu, Yan Zhang, Xiaotang Gai, Yang Feng, Zuozhu Liu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Workshop on Interpretable ML in Healthcare at International Conference on Machine Learning (ICML) 2023

2303.13391 2023-06-29 cs.CV cs.LG 62%

Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis

Chantal Pellegrini, Matthias Keicher, Ege Özsoy, Petra Jiraskova, Rickmer Braren, Nassir Navab

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments provisionally accepted for publication at MICCAI 2023, 9 pages, 2 figures, 6 tables

2305.03660 2023-05-08 cs.CL cs.AI cs.IR cs.LG 62%

Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models

Mercy Ranjit, Gopinath Ganapathy, Ranjit Manuel, Tanuja Ganu

专题命中其他VLM ：vision language model(abstract)；分类 cs.AI、cs.LG

2302.03298 2023-04-18 cs.CV cs.AI 62%

Diversity is Definitely Needed: Improving Model-Agnostic Zero-shot Classification via Stable Diffusion

Jordan Shipard, Arnold Wiliem, Kien Nguyen Thanh, Wei Xiang, Clinton Fookes

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments (10 pages, 6 figures, 3 tables, preprint)

2304.00779 2023-04-04 cs.CV cs.AI 62%

Probabilistic Prompt Learning for Dense Prediction

Hyeongjun Kwon, Taeyong Song, Somi Jeong, Jin Kim, Jinhyun Jang, Kwanghoon Sohn

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments accepted to CVPR 2023

2303.06580 2023-03-14 cs.CV cs.CL cs.LG 62%

Towards General Purpose Medical AI: Continual Learning Medical Foundation Model

Huahui Yi, Ziyuan Qin, Qicheng Lao, Wei Xu, Zekun Jiang, Dequan Wang, Shaoting Zhang, Kang Li

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2303.01258 2023-03-03 cs.CL cs.AI cs.LG 62%

Domain-adapted large language models for classifying nuclear medicine reports

Zachary Huemann, Changhee Lee, Junjie Hu, Steve Y. Cho, Tyler Bradshaw

专题命中其他VLM ：vision language model(abstract)；分类 cs.AI、cs.LG

1. 其他VLM 1565 篇

Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP

Yell At Your Robot: Improving On-the-Fly from Language Corrections

Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification

XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization

Leveraging Diffusion Perturbations for Measuring Fairness in Computer Vision

Image-Caption Encoding for Improving Zero-Shot Generalization

Object-oriented backdoor attack against image captioning

VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use

Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks

Visual AI and Linguistic Intelligence Through Steerability and Composability

LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers

Domain Prompt Learning with Quaternion Networks

ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

CoPL: Contextual Prompt Learning for Vision-Language Understanding

ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation

Improving Multimodal Datasets with Image Captioning

S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions

VPGTrans: Transfer Visual Prompt Generator across LLMs

Understanding prompt engineering may not require rethinking generalization

Identifying Interpretable Subspaces in Image Representations

Waffling around for Performance: Visual Classification with Random Words and Broad Concepts

Blended Latent Diffusion

The Visual Language of Fabrics

A ChatGPT Aided Explainable Framework for Zero-Shot Medical Image Diagnosis

Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis

Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models

Diversity is Definitely Needed: Improving Model-Agnostic Zero-shot Classification via Stable Diffusion

Probabilistic Prompt Learning for Dense Prediction

Towards General Purpose Medical AI: Continual Learning Medical Foundation Model

Domain-adapted large language models for classifying nuclear medicine reports