多模态大模型

CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Natural Language

Aditya Sanghi, Rao Fu, Vivian Liu, Karl Willis, Hooman Shayani, Amir Hosein Khasahmadi, Srinath Sridhar, Daniel Ritchie

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted at Conference on Computer Vision and Pattern Recognition 2023(CVPR2023)

Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

An Yang, Junshu Pan, Junyang Lin, Rui Men, Yichang Zhang, Jingren Zhou, Chang Zhou

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

DiffCap: Exploring Continuous Diffusion on Image Captioning

Yufeng He, Zefan Cai, Xu Gan, Baobao Chang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

LLM2Loss: Leveraging Language Models for Explainable Model Diagnostics

Shervin Ardeshir

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

CREPE: Can Vision-Language Foundation Models Reason Compositionally?

Zixian Ma, Jerry Hong, Mustafa Omer Gul, Mona Gandhi, Irena Gao, Ranjay Krishna

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Updated figures and numbers

Masked Autoencoding Does Not Help Natural Language Supervision at Scale

Floris Weers, Vaishaal Shankar, Angelos Katharopoulos, Yinfei Yang, Tom Gunter

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted at CVPR 2023

Measuring Progress in Fine-grained Vision-and-Language Understanding

Emanuele Bugliarello, Laurent Sartran, Aishwarya Agrawal, Lisa Anne Hendricks, Aida Nematzadeh

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments ACL 2023

A Review of Vision-Language Models and their Performance on the Hateful Memes Challenge

Bryan Zhao, Andrew Zhang, Blake Watson, Gillian Kearney, Isaac Dale

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Less is More: Removing Text-regions Improves CLIP Training Efficiency and Robustness

Liangliang Cao, Bowen Zhang, Chen Chen, Yinfei Yang, Xianzhi Du, Wencong Zhang, Zhiyun Lu, Yantao Zheng

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 8 figures

Learning CLIP Guided Visual-Text Fusion Transformer for Video-based Pedestrian Attribute Recognition

Jun Zhu, Jiandong Jin, Zihan Yang, Xiaohao Wu, Xiao Wang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by CVPR-2023 Workshop@NFVLR (New Frontiers in Visual Language Reasoning: Compositionality, Prompts and Causality)

A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning

Aishwarya Kamath, Peter Anderson, Su Wang, Jing Yu Koh, Alexander Ku, Austin Waters, Yinfei Yang, Jason Baldridge, Zarana Parekh

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments CVPR 2023

Combined Scaling for Zero-shot Transfer Learning

Hieu Pham, Zihang Dai, Golnaz Ghiasi, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh-Thang Luong, Yonghui Wu, Mingxing Tan, Quoc V. Le

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

A Study of Autoregressive Decoders for Multi-Tasking in Computer Vision

Lucas Beyer, Bo Wan, Gagan Madan, Filip Pavetic, Andreas Steiner, Alexander Kolesnikov, André Susano Pinto, Emanuele Bugliarello, Xiao Wang, Qihang Yu, Liang-Chieh Chen, Xiaohua Zhai

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Detecting Backdoors in Pre-trained Encoders

Shiwei Feng, Guanhong Tao, Siyuan Cheng, Guangyu Shen, Xiangzhe Xu, Yingqi Liu, Kaiyuan Zhang, Shiqing Ma, Xiangyu Zhang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted at CVPR 2023. Code is available at https://github.com/GiantSeaweed/DECREE

FlexiViT: One Model for All Patch Sizes

Lucas Beyer, Pavel Izmailov, Alexander Kolesnikov, Mathilde Caron, Simon Kornblith, Xiaohua Zhai, Matthias Minderer, Michael Tschannen, Ibrahim Alabdulmohsin, Filip Pavetic

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Code and pre-trained models available at https://github.com/google-research/big_vision. All authors made significant technical contributions. CVPR 2023

Open-Vocabulary Object Detection using Pseudo Caption Labels

Han-Cheol Cho, Won Young Jhoo, Wooyoung Kang, Byungseok Roh

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Who are you referring to? Coreference resolution in image narrations

Arushi Goel, Basura Fernando, Frank Keller, Hakan Bilen

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments 15 pages

Scaling Vision-Language Models with Sparse Mixture of Experts

Sheng Shen, Zhewei Yao, Chunyuan Li, Trevor Darrell, Kurt Keutzer, Yuxiong He

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Preprint

Learning Transferable Spatiotemporal Representations from Natural Script Knowledge

Ziyun Zeng, Yuying Ge, Xihui Liu, Bin Chen, Ping Luo, Shu-Tao Xia, Yixiao Ge

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2023; camera-ready version

Medical visual question answering using joint self-supervised learning

Yuan Zhou, Jing Mei, Yiqin Yu, Tanveer Syeda-Mahmood

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Retrieval-augmented Image Captioning

Rita Ramos, Desmond Elliott, Bruno Martins

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Journal ref EACL 2023

Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training

Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, Pascale Fung

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted at EACL 2023

Affective Faces for Goal-Driven Dyadic Communication

Scott Geng, Revant Teotia, Purva Tendulkar, Sachit Menon, Carl Vondrick

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classification

Fang Peng, Xiaoshan Yang, Linhui Xiao, Yaowei Wang, Changsheng Xu

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

CiT: Curation in Training for Effective Vision-Language Data

Hu Xu, Saining Xie, Po-Yao Huang, Licheng Yu, Russell Howes, Gargi Ghosh, Luke Zettlemoyer, Christoph Feichtenhofer

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Technical Report

An Empirical Investigation into the Use of Image Captioning for Automated Software Documentation

Kevin Moran, Ali Yachnes, George Purnell, Junayed Mahmud, Michele Tufano, Carlos Bernal-Cárdenas, Denys Poshyvanyk, Zach H'Doubler

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Published in the Proceedings of the 29th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER'22), Honolulu, Hawaii, March 15-18, 2022, pp. 514-525

Generalized Decoding for Pixel, Image, and Language

Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Lu Yuan, Nanyun Peng, Lijuan Wang, Yong Jae Lee, Jianfeng Gao

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments https://x-decoder-vl.github.io

3D-LatentMapper: View Agnostic Single-View Reconstruction of 3D Shapes

Alara Dirik, Pinar Yanardag

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS - WiML workshop 2022

Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection

Hanoona Rasheed, Muhammad Maaz, Muhammad Uzair Khattak, Salman Khan, Fahad Shahbaz Khan

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted at NeurIPS 2022

Delving into Out-of-Distribution Detection with Vision-Language Representations

Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, Yixuan Li

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 36th Conference on Neural Information Processing Systems (NeurIPS 2022)