视觉大模型 / VLM - arXivDaily 专题

2304.06212 2023-04-14 cs.CV 57%

[CLS] Token is All You Need for Zero-Shot Semantic Segmentation

Letian Wu, Wenyao Zhang, Tengping Jiang, Wankou Yang, Xin Jin, Wenjun Zeng

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 8 pages,6 figures

2304.05995 2023-04-13 cs.CV 57%

APPLeNet: Visual Attention Parameterized Prompt Learning for Few-Shot Remote Sensing Image Generalization using CLIP

Mainak Singha, Ankit Jha, Bhupendra Solanki, Shirsha Bose, Biplab Banerjee

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 11 Pages, 6 figures, 8 tables, Accepted in Earth Vision (CVPR 2023)

2301.02228 2023-04-04 eess.IV cs.CL cs.CV 57%

MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology

Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, Weidi Xie

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

2303.14865 2023-03-28 cs.CV 57%

Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokens

Yuxiao Chen, Jianbo Yuan, Yu Tian, Shijie Geng, Xinyu Li, Ding Zhou, Dimitris N. Metaxas, Hongxia Yang

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

Comments Accepted to CVPR 2023

2303.13283 2023-03-24 cs.CV cs.CL 57%

Visual-Language Prompt Tuning with Knowledge-guided Context Optimization

Hantao Yao, Rui Zhang, Changsheng Xu

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

Comments accepted by CVPR23

2303.11866 2023-03-22 cs.CV 57%

Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning

Zaid Khan, Yun Fu

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted to ICLR 2023

2303.06591 2023-03-14 cs.CV 57%

Accommodating Audio Modality in CLIP for Multimodal Processing

Ludan Ruan, Anwen Hu, Yuqing Song, Liang Zhang, Sipeng Zheng, Qin Jin

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by AAAI2023

2301.07236 2023-01-19 cs.CV 57%

Effective End-to-End Vision Language Pretraining with Semantic Visual Loss

Xiaofeng Yang, Fayao Liu, Guosheng Lin

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV

2301.02241 2023-01-06 cs.CV cs.CL 57%

CiT: Curation in Training for Effective Vision-Language Data

Hu Xu, Saining Xie, Po-Yao Huang, Licheng Yu, Russell Howes, Gargi Ghosh, Luke Zettlemoyer, Christoph Feichtenhofer

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Technical Report

2212.01568 2022-12-06 cs.CV 57%

Generalizing Multiple Object Tracking to Unseen Domains by Introducing Natural Language Representation

En Yu, Songtao Liu, Zhuoling Li, Jinrong Yang, Zeming li, Shoudong Han, Wenbing Tao

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

Comments Accepted by AAAI2023

2207.11401 2022-12-05 cs.CL cs.CV cs.MM 57%

Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language Explanations

Qian Yang, Yunxin Li, Baotian Hu, Lin Ma, Yuxing Ding, Min Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 11 pages (including Supplementary Materials); Accepted to ACM MM 2022

Journal ref ACM International Conference on Multimedia. 2022. 3587-3597

2211.08736 2022-11-17 cs.CV cs.MM 57%

AlignVE: Visual Entailment Recognition Based on Alignment Relations

Biwei Cao, Jiuxin Cao, Jie Gui, Jiayun Shen, Bo Liu, Lei He, Yuan Yan Tang, James Tin-Yau Kwok

专题命中 VLM训练与架构：visual question answering(abstract)；分类 cs.CV

Comments This paper is accepted for publication as a REGULAR paper in the IEEE Transactions on Multimedia

2210.13617 2022-10-27 cs.CL cs.AI 57%

Adapters for Enhanced Modeling of Multilingual Knowledge and Text

Yifan Hou, Wenxiang Jiao, Meizhen Liu, Carl Allen, Zhaopeng Tu, Mrinmaya Sachan

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.AI

Comments Our code, models, and data (e.g., integration corpus and extended datasets) are available: https://github.com/yifan-h/Multilingual_Space

2209.07526 2022-10-21 cs.CV 57%

OmniVL:One Foundation Model for Image-Language and Video-Language Tasks

Junke Wang, Dongdong Chen, Zuxuan Wu, Chong Luo, Luowei Zhou, Yucheng Zhao, Yujia Xie, Ce Liu, Yu-Gang Jiang, Lu Yuan

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments To appear at NeurIPs 2022, Camera Ready with Typos fixed

2210.04722 2022-10-11 cs.CV 57%

Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment

Jielin Qiu, Jiacheng Zhu, Mengdi Xu, Franck Dernoncourt, Trung Bui, Zhaowen Wang, Bo Li, Ding Zhao, Hailin Jin

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

2210.04506 2022-10-11 cs.CV 57%

Bridging CLIP and StyleGAN through Latent Alignment for Image Editing

Wanfeng Zheng, Qiang Li, Xiaoyan Guo, Pengfei Wan, Zhongyuan Wang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 20 pages, 23 figures

2210.04287 2022-10-11 cs.CV 57%

Learning to Decompose Visual Features with Latent Textual Prompts

Feng Wang, Manling Li, Xudong Lin, Hairong Lv, Alexander G. Schwing, Heng Ji

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2210.03794 2022-10-11 cs.CV 57%

SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Models

Omiros Pantazis, Gabriel Brostow, Kate Jones, Oisin Mac Aodha

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments BMVC 2022

2206.02338 2022-10-04 cs.CV 57%

OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression

Wanhua Li, Xiaoke Huang, Zheng Zhu, Yansong Tang, Xiu Li, Jie Zhou, Jiwen Lu

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by NeurIPS2022. Code is available at https://github.com/xk-huang/OrdinalCLIP

2010.12773 2022-09-01 cs.CL cs.AI 57%

Structure-Grounded Pretraining for Text-to-SQL

Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, Matthew Richardson

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

Comments Accepted to NAACL 2021. The Spider-Realistic dataset is available at https://doi.org/10.5281/zenodo.5205322

2203.04904 2022-07-18 cs.MM cs.CL cs.CV 57%

Rethinking Task Sampling for Few-shot Vision-Language Transfer Learning

Zhenhailong Wang, Hang Yu, Manling Li, Han Zhao, Heng Ji

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 4 pages, 4 figures, under review

2007.04965 2022-06-07 cs.LG cs.NE stat.ML 57%

A Study on Encodings for Neural Architecture Search

Colin White, Willie Neiswanger, Sam Nolen, Yash Savani

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.LG

Journal ref Advances in Neural Information Processing Systems 2020

2204.07955 2022-04-22 cs.CV cs.CL cs.MM 57%

Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis

Yan Ling, Jianfei Yu, Rui Xia

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by ACL 2022 (long paper)

2203.05175 2022-03-11 cs.CV 57%

MVP: Multimodality-guided Visual Pre-training

Longhui Wei, Lingxi Xie, Wengang Zhou, Houqiang Li, Qi Tian

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 14 pages

2201.04026 2022-01-12 cs.CV cs.CL cs.MM 57%

Uni-EDEN: Universal Encoder-Decoder Network by Multi-Granular Vision-Language Pre-training

Yehao Li, Jiahao Fan, Yingwei Pan, Ting Yao, Weiyao Lin, Tao Mei

专题命中 VLM训练与架构：visual question answering(abstract)；分类 cs.CV

Comments ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)

2110.12567 2021-10-26 cs.LG cs.CL stat.ML 57%

Alignment Attention by Matching Key and Query Distributions

Shujian Zhang, Xinjie Fan, Huangjie Zheng, Korawat Tanwisuth, Mingyuan Zhou

专题命中 VLM训练与架构：visual question answering(abstract)；分类 cs.LG

Comments NeurIPS 2021; Our code is publicly available at https://github.com/szhang42/alignment_attention

2005.00171 2021-01-27 cs.CL cs.AI cs.IR 57%

Cross-lingual Entity Alignment with Incidental Supervision

Muhao Chen, Weijia Shi, Ben Zhou, Dan Roth

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

Comments EACL 2021

1706.03661 2021-01-25 cs.AI cs.RO 57%

DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self

Clément Moulin-Frier, Tobias Fischer, Maxime Petit, Grégoire Pointeau, Jordi-Ysard Puigbo, Ugo Pattacini, Sock Ching Low, Daniel Camilleri, Phuong Nguyen, Matej Hoffmann, Hyung Jin Chang, Martina Zambelli, Anne-Laure Mealier, Andreas Damianou, Giorgio Metta, Tony J. Prescott, Yiannis Demiris, Peter Ford Dominey, Paul F. M. J. Verschure

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

Comments Preprint version; final version available at http://ieeexplore.ieee.org/ IEEE Transactions on Cognitive and Developmental Systems (Accepted) DOI: 10.1109/TCDS.2017.2754143

Journal ref IEEE Transactions on Cognitive and Developmental Systems 10 (4), 1005-1022, 2018

2005.07327 2020-07-31 cs.CV 57%

ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language

Zhe Wang, Zhiyuan Fang, Jun Wang, Yezhou Yang

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.CV

Comments ECCV2020, 18 pages, 6 figures

1908.06066 2019-12-04 cs.CV 57%

Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training

Gen Li, Nan Duan, Yuejian Fang, Ming Gong, Daxin Jiang, Ming Zhou

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

Comments accepted by AAAI-2020. arXiv admin note: text overlap with arXiv:1909.11740 by other authors

1. VLM训练与架构 5058 篇

[CLS] Token is All You Need for Zero-Shot Semantic Segmentation

APPLeNet: Visual Attention Parameterized Prompt Learning for Few-Shot Remote Sensing Image Generalization using CLIP

MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology

Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokens

Visual-Language Prompt Tuning with Knowledge-guided Context Optimization

Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning

Accommodating Audio Modality in CLIP for Multimodal Processing

Effective End-to-End Vision Language Pretraining with Semantic Visual Loss

CiT: Curation in Training for Effective Vision-Language Data

Generalizing Multiple Object Tracking to Unseen Domains by Introducing Natural Language Representation

Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language Explanations

AlignVE: Visual Entailment Recognition Based on Alignment Relations

Adapters for Enhanced Modeling of Multilingual Knowledge and Text

OmniVL:One Foundation Model for Image-Language and Video-Language Tasks

Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment

Bridging CLIP and StyleGAN through Latent Alignment for Image Editing

Learning to Decompose Visual Features with Latent Textual Prompts

SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Models

OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression

Structure-Grounded Pretraining for Text-to-SQL

Rethinking Task Sampling for Few-shot Vision-Language Transfer Learning

A Study on Encodings for Neural Architecture Search

Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis

MVP: Multimodality-guided Visual Pre-training

Uni-EDEN: Universal Encoder-Decoder Network by Multi-Granular Vision-Language Pre-training

Alignment Attention by Matching Key and Query Distributions

Cross-lingual Entity Alignment with Incidental Supervision

DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self

ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language

Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training