多模态大模型

BDC-Adapter: Brownian Distance Covariance for Better Vision-Language Reasoning

Yi Zhang, Ce Zhang, Zihan Liao, Yushun Tang, Zhihai He

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by BMVC 2023

TouchStone: Evaluating Vision-Language Models by Language Models

Shuai Bai, Shusheng Yang, Jinze Bai, Peng Wang, Xingxuan Zhang, Junyang Lin, Xinggang Wang, Chang Zhou, Jingren Zhou

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments https://github.com/OFA-Sys/TouchStone

CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say No

Hualiang Wang, Yi Li, Huifeng Yao, Xiaomeng Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023

CAT-ViL: Co-Attention Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery

Long Bai, Mobarakol Islam, Hongliang Ren

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments To appear in MICCAI 2023. Code availability: https://github.com/longbai1006/CAT-ViL

PreSTU: Pre-Training for Scene-Text Understanding

Jihyung Kil, Soravit Changpinyo, Xi Chen, Hexiang Hu, Sebastian Goodman, Wei-Lun Chao, Radu Soricut

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ICCV 2023

VT-CLIP: Enhancing Vision-Language Models with Visual-guided Texts

Longtian Qiu, Renrui Zhang, Ziyu Guo, Ziyao Zeng, Zilu Guo, Yafeng Li, Guangnan Zhang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

CLIP-Count: Towards Text-Guided Zero-Shot Object Counting

Ruixiang Jiang, Lingbo Liu, Changwen Chen

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACM MM 2023

Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Models

Zheng Ma, Mianzhi Pan, Wenhan Wu, Kanzhi Cheng, Jianbing Zhang, Shujian Huang, Jiajun Chen

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at ACM Multimedia (ACMMM) 2023

GIST: Generating Image-Specific Text for Fine-grained Object Classification

Kathleen M. Lewis, Emily Mu, Adrian V. Dalca, John Guttag

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments The first two authors contributed equally to this work and are listed in alphabetical order

Text-guided Foundation Model Adaptation for Pathological Image Classification

Yunkun Zhang, Jin Gao, Mu Zhou, Xiaosong Wang, Yu Qiao, Shaoting Zhang, Dequan Wang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to MICCAI2023

Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?

Cheng-En Wu, Yu Tian, Haichao Yu, Heng Wang, Pedro Morgado, Yu Hen Hu, Linjie Yang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICCV2023

ELVIS: Empowering Locality of Vision Language Pre-training with Intra-modal Similarity

Sumin Seo, JaeWoong Shin, Jaewoo Kang, Tae Soo Kim, Thijs Kooi

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Under review

OxfordTVG-HIC: Can Machine Make Humorous Captions from Images?

Runjia Li, Shuyang Sun, Mohamed Elhoseiny, Philip Torr

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ICCV 2023

MotionGPT: Human Motion as a Foreign Language

Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, Tao Chen

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Project Page: https://github.com/OpenMotionLab/MotionGPT

KU-DMIS-MSRA at RadSum23: Pre-trained Vision-Language Model for Radiology Report Summarization

Gangwoo Kim, Hajung Kim, Lei Ji, Seongsu Bae, Chanhwi Kim, Mujeen Sung, Hyunjae Kim, Kun Yan, Eric Chang, Jaewoo Kang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Published at BioNLP workshop @ ACL 2023

Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution

Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim Alabdulmohsin, Avital Oliver, Piotr Padlewski, Alexey Gritsenko, Mario Lučić, Neil Houlsby

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers

Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Chun Yuan, Jiaqi Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments ICML 2023. Website: https://dachuanshi.com/UPop-Project

DesCo: Learning Object Recognition with Rich Language Descriptions

Liunian Harold Li, Zi-Yi Dou, Nanyun Peng, Kai-Wei Chang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relations

Tiancheng Zhao, Tianqi Zhang, Mingwei Zhu, Haozhan Shen, Kyusong Lee, Xiaopeng Lu, Jianwei Yin

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 9 pages, preprint

Improving Zero-Shot Detection of Low Prevalence Chest Pathologies using Domain Pre-trained Language Models

Aakash Mishra, Rajat Mittal, Christy Jestin, Kostas Tingos, Pranav Rajpurkar

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments 3 pages, 1 table, Medical Imaging with Deep Learning, Short Paper

Training Vision-Language Transformers from Captions

Liangke Gui, Yingshan Chang, Qiuyuan Huang, Subhojit Som, Alex Hauptmann, Jianfeng Gao, Yonatan Bisk

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation

Yinda Chen, Che Liu, Wei Huang, Sibo Cheng, Rossella Arcucci, Zhiwei Xiong

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

SciCap+: A Knowledge Augmented Dataset to Study the Challenges of Scientific Figure Captioning

Zhishen Yang, Raj Dabre, Hideki Tanaka, Naoaki Okazaki

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Published in SDU workshop at AAAI23

Identifying Shared Decodable Concepts in the Human Brain Using Image-Language Foundation Models

Cory Efird, Alex Murphy, Joel Zylberberg, Alona Fyshe

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Under review

Masked Vision-Language Transformer in Fashion

Ge-Peng Ji, Mingcheng Zhuge, Dehong Gao, Deng-Ping Fan, Christos Sakaridis, Luc Van Gool

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by Machine Intelligence Research (2023)

Journal ref Machine Intelligence Research. 20, 421-434 (2023)

CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks

Tuomas Oikarinen, Tsui-Wei Weng

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Published in ICLR 2023 Conference (Spotlight). New v5(5 June 2023) - Added crowdsourced user study in Appendix B, not included in ICLR publication

Scalable Performance Analysis for Vision-Language Models

Santiago Castro, Oana Ignat, Rada Mihalcea

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Camera-ready version for *SEM 2023

KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature Adaptation of Vision-Language Models

Zhiwei Jia, Pradyumna Narayana, Arjun R. Akula, Garima Pruthi, Hao Su, Sugato Basu, Varun Jampani

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments ACL 2023

ZegOT: Zero-shot Segmentation Through Optimal Transport of Text Prompts

Kwanyoung Kim, Yujin Oh, Jong Chul Ye

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 18pages, 8 figures

Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?

Tian Yun, Usha Bhalla, Ellie Pavlick, Chen Sun

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Published in Transactions on Machine Learning Research (TMLR) 2023