多模态大模型 - arXivDaily 专题

2310.08166 2023-11-01 cs.CL 70%

Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Junyu Lu, Dixiang Zhang, Xiaojun Wu, Xinyu Gao, Ruyi Gan, Jiaxing Zhang, Yan Song, Pingjian Zhang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CL

2310.18049 2023-10-30 cs.CV 70%

Text Augmented Spatial-aware Zero-shot Referring Image Segmentation

Yucheng Suo, Linchao Zhu, Yi Yang

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Findings of EMNLP2023

2305.15023 2023-10-25 cs.CV 70%

Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models

Gen Luo, Yiyi Zhou, Tianhe Ren, Shengxin Chen, Xiaoshuai Sun, Rongrong Ji

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

2309.14491 2023-09-27 cs.CV 70%

Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Driving

Mahyar Najibi, Jingwei Ji, Yin Zhou, Charles R. Qi, Xinchen Yan, Scott Ettinger, Dragomir Anguelov

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments ICCV 2023

2308.16354 2023-09-01 cs.CV 70%

Catalog Phrase Grounding (CPG): Grounding of Product Textual Attributes in Product Images for e-commerce Vision-Language Applications

Wenyi Wu, Karim Bouyarmane, Ismail Tutar

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments KDD 2022 Workshop on First Content Understanding and Generation for e-Commerce

2308.13853 2023-08-29 cs.CV 70%

Beyond One-to-One: Rethinking the Referring Image Segmentation

Yutao Hu, Qixiong Wang, Wenqi Shao, Enze Xie, Zhenguo Li, Jungong Han, Ping Luo

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments ICCV 2023

2308.08754 2023-08-21 cs.CV 70%

Fine-grained Text and Image Guided Point Cloud Completion with CLIP Model

Wei Song, Jun Zhou, Mingjie Wang, Hongchen Tan, Nannan Li, Xiuping Liu

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2307.12980 2023-07-25 cs.CV 70%

A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models

Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2303.12112 2023-07-21 cs.CV cs.AI cs.CL cs.MM 70%

Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation

Sara Sarto, Manuele Barraco, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2023 (highlight paper)

2306.09085 2023-06-16 cs.CV cs.AI cs.CL cs.LG cs.MM 70%

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

Sihan Chen, Xingjian He, Handong Li, Xiaojie Jin, Jiashi Feng, Jing Liu

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

2306.04715 2023-06-09 cs.CV 70%

UniBoost: Unsupervised Unimodal Pre-training for Boosting Zero-shot Vision-Language Tasks

Yanan Sun, Zihan Zhong, Qi Fan, Chi-Keung Tang, Yu-Wing Tai

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2303.14302 2023-06-06 cs.CV 70%

VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining

Junjie Ke, Keren Ye, Jiahui Yu, Yonghui Wu, Peyman Milanfar, Feng Yang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2023, https://github.com/google-research/google-research/tree/master/vila

2305.19595 2023-06-02 cs.CV 70%

Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models

Sivan Doveh, Assaf Arbelle, Sivan Harary, Roei Herzig, Donghyun Kim, Paola Cascante-bonilla, Amit Alfassy, Rameswar Panda, Raja Giryes, Rogerio Feris, Shimon Ullman, Leonid Karlinsky

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

2305.15483 2023-05-26 cs.CV 70%

Weakly Supervised Vision-and-Language Pre-training with Relative Representations

Chi Chen, Peng Li, Maosong Sun, Yang Liu

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by ACL 2023

2305.12223 2023-05-24 cs.CV 70%

What Makes for Good Visual Tokenizers for Large Language Models?

Guangzhi Wang, Yixiao Ge, Xiaohan Ding, Mohan Kankanhalli, Ying Shan

专题命中图文多模态：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

Comments 15 pages, 3 figures. Project released at: https://github.com/TencentARC/GVT

2304.13583 2023-04-27 eess.IV cs.CV 70%

Multi-Modality Deep Network for Extreme Learned Image Compression

Xuhao Jiang, Weimin Tan, Tian Tan, Bo Yan, Liquan Shen

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments 13 pages, 14 figures, accepted by AAAI 2023

2301.04926 2023-04-07 cs.CV 70%

CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, Wenping Wang

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2023

2302.12248 2023-03-30 cs.CV 70%

Learning Visual Representations via Language-Guided Sampling

Mohamed El Banani, Karan Desai, Justin Johnson

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted to CVPR 2023. v2 is camera-ready version with additional ImageNet evaluations. Project page: https://github.com/mbanani/lgssl

2303.02483 2023-03-07 cs.CV 70%

FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks

Xiao Han, Xiatian Zhu, Licheng Yu, Li Zhang, Yi-Zhe Song, Tao Xiang

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments CVPR 2023

2301.09045 2023-02-21 cs.CV 70%

Champion Solution for the WSDM2023 Toloka VQA Challenge

Shengyi Gao, Zhe Chen, Guo Chen, Wenhai Wang, Tong Lu

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Technical report in WSDM Cup 2023

2105.05636 2023-01-06 cs.CV 70%

VL-NMS: Breaking Proposal Bottlenecks in Two-Stage Visual-Language Matching

Chenchi Zhang, Wenbo Ma, Jun Xiao, Hanwang Zhang, Jian Shao, Yueting Zhuang, Long Chen

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2009.01449

2212.07086 2023-01-05 cs.CV 70%

NLIP: Noise-robust Language-Image Pre-training

Runhui Huang, Yanxin Long, Jianhua Han, Hang Xu, Xiwen Liang, Chunjing Xu, Xiaodan Liang

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments AAAI 2023

2212.08985 2022-12-20 cs.CV 70%

Efficient Image Captioning for Edge Devices

Ning Wang, Jiangrong Xie, Hang Luo, Qinglin Cheng, Jihao Wu, Mingbo Jia, Linlin Li

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments To appear in AAAI 2023

2205.13125 2022-11-21 cs.CV 70%

Prompt-based Learning for Unpaired Image Captioning

Peipei Zhu, Xiao Wang, Lin Zhu, Zhenglong Sun, Weishi Zheng, Yaowei Wang, Changwen Chen

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

2210.17322 2022-11-01 cs.CV 70%

Generative Negative Text Replay for Continual Vision-Language Pretraining

Shipeng Yan, Lanqing Hong, Hang Xu, Jianhua Han, Tinne Tuytelaars, Zhenguo Li, Xuming He

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments ECCV 2022

2210.10276 2022-10-20 cs.CV 70%

CLIP-Driven Fine-grained Text-Image Person Re-identification

Shuanglin Yan, Neng Dong, Liyan Zhang, Jinhui Tang

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

2210.08901 2022-10-18 cs.CV 70%

Contrastive Language-Image Pre-Training with Knowledge Graphs

Xuran Pan, Tianzhu Ye, Dongchen Han, Shiji Song, Gao Huang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by NeurIPS2022

2208.02515 2022-09-20 cs.CV 70%

Fine-Grained Semantically Aligned Vision-Language Pre-Training

Juncheng Li, Xin He, Longhui Wei, Long Qian, Linchao Zhu, Lingxi Xie, Yueting Zhuang, Qi Tian, Siliang Tang

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2022

2207.13162 2022-08-23 cs.CV cs.AI cs.CL cs.MM 70%

Retrieval-Augmented Transformer for Image Captioning

Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CBMI 2022

2208.09374 2022-08-22 cs.CV 70%

VLMAE: Vision-Language Masked Autoencoder

Sunan He, Taian Guo, Tao Dai, Ruizhi Qiao, Chen Wu, Xiujun Shu, Bo Ren

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 12 pages, 7 figures

1. 图文多模态 4651 篇

Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Text Augmented Spatial-aware Zero-shot Referring Image Segmentation

Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models

Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Driving

Catalog Phrase Grounding (CPG): Grounding of Product Textual Attributes in Product Images for e-commerce Vision-Language Applications

Beyond One-to-One: Rethinking the Referring Image Segmentation

Fine-grained Text and Image Guided Point Cloud Completion with CLIP Model

A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models

Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

UniBoost: Unsupervised Unimodal Pre-training for Boosting Zero-shot Vision-Language Tasks

VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining

Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models

Weakly Supervised Vision-and-Language Pre-training with Relative Representations

What Makes for Good Visual Tokenizers for Large Language Models?

Multi-Modality Deep Network for Extreme Learned Image Compression

CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

Learning Visual Representations via Language-Guided Sampling

FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks

Champion Solution for the WSDM2023 Toloka VQA Challenge

VL-NMS: Breaking Proposal Bottlenecks in Two-Stage Visual-Language Matching

NLIP: Noise-robust Language-Image Pre-training

Efficient Image Captioning for Edge Devices

Prompt-based Learning for Unpaired Image Captioning

Generative Negative Text Replay for Continual Vision-Language Pretraining

CLIP-Driven Fine-grained Text-Image Person Re-identification

Contrastive Language-Image Pre-Training with Knowledge Graphs

Fine-Grained Semantically Aligned Vision-Language Pre-Training

Retrieval-Augmented Transformer for Image Captioning

VLMAE: Vision-Language Masked Autoencoder