视觉大模型 / VLM - arXivDaily 专题

2405.01926 2024-05-06 cs.CV 57%

Auto-Encoding Morph-Tokens for Multimodal LLM

Kaihang Pan, Siliang Tang, Juncheng Li, Zhaoyu Fan, Wei Chow, Shuicheng Yan, Tat-Seng Chua, Yueting Zhuang, Hanwang Zhang

专题命中其他VLM ：MLLM(abstract)；分类 cs.CV

Comments Accepted by ICML 2024

2404.19644 2024-05-01 cs.CV 57%

MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation

Min Zhang, Haoxuan Li, Fei Wu, Kun Kuang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments ICLR 24

2312.16794 2024-04-15 cs.CV 57%

ZONE: Zero-Shot Instruction-Guided Local Editing

Shanglin Li, Bohan Zeng, Yutang Feng, Sicheng Gao, Xuhui Liu, Jiaming Liu, Li Lin, Xu Tang, Yao Hu, Jianzhuang Liu, Baochang Zhang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at CVPR 2024

2404.07177 2024-04-11 cs.LG 57%

Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic

Sachin Goyal, Pratyush Maini, Zachary C. Lipton, Aditi Raghunathan, J. Zico Kolter

专题命中其他VLM ：vision-language model(abstract)；分类 cs.LG

Comments Published at CVPR 2024

2311.07853 2024-04-09 cs.CL cs.LG 57%

Learning Mutually Informed Representations for Characters and Subwords

Yilin Wang, Xinyi Hu, Matthew R. Gormley

专题命中其他VLM ：vision-language model(abstract)；分类 cs.LG

2402.19326 2024-04-08 cs.CV 57%

Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction

Hao Li, Ying Chen, Yifei Chen, Wenxian Yang, Bowen Ding, Yuchen Han, Liansheng Wang, Rongshan Yu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2311.08046 2024-04-08 cs.CV 57%

Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, Li Yuan

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024 (Highlight)

2404.03413 2024-04-05 cs.CV 57%

MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens

Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Essam Sleiman, Deyao Zhu, Jian Ding, Mohamed Elhoseiny

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV

Comments 6 pages,8 figures

2404.03042 2024-04-05 cs.CV 57%

AWOL: Analysis WithOut synthesis using Language

Silvia Zuffi, Michael J. Black

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

2401.02460 2024-04-05 cs.CV 57%

Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions

Oindrila Saha, Grant Van Horn, Subhransu Maji

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

2210.04936 2024-04-04 cs.CV 57%

EarthNets: Empowering AI in Earth Observation

Zhitong Xiong, Fahong Zhang, Yi Wang, Yilei Shi, Xiao Xiang Zhu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments 30 pages

2404.00710 2024-04-02 cs.CV 57%

Unknown Prompt, the only Lacuna: Unveiling CLIP's Potential for Open Domain Generalization

Mainak Singha, Ankit Jha, Shirsha Bose, Ashwin Nair, Moloud Abdar, Biplab Banerjee

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted in CVPR 2024

2403.13600 2024-03-21 cs.CV 57%

VL-Mamba: Exploring State Space Models for Multimodal Learning

Yanyuan Qiao, Zheng Yu, Longteng Guo, Sihan Chen, Zijia Zhao, Mingzhen Sun, Qi Wu, Jing Liu

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV

2403.13343 2024-03-21 cs.CV 57%

TiBiX: Leveraging Temporal Information for Bidirectional X-ray and Report Generation

Santosh Sanjeev, Fadillah Adamsyah Maani, Arsen Abzhanov, Vijay Ram Papineni, Ibrahim Almakky, Bartłomiej W. Papież, Mohammad Yaqub

专题命中其他VLM ：vision language model(abstract)；分类 cs.CV

2403.12037 2024-03-20 cs.CV 57%

MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control

Enshen Zhou, Yiran Qin, Zhenfei Yin, Yuzhou Huang, Ruimao Zhang, Lu Sheng, Yu Qiao, Jing Shao

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV

Comments Project page: https://sites.google.com/view/minedreamer/main

2310.16640 2024-03-19 cs.CV cs.HC 57%

EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Expression Recognition

Niki Maria Foteinopoulou, Ioannis Patras

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at FG'2024

2403.10245 2024-03-18 cs.CV 57%

CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning

Yukun Li, Guansong Pang, Wei Suo, Chenchen Jing, Yuling Xi, Lingqiao Liu, Hao Chen, Guoqiang Liang, Peng Wang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

2403.06946 2024-03-12 cs.CV 57%

Split to Merge: Unifying Separated Modalities for Unsupervised Domain Adaptation

Xinyao Li, Yuke Li, Zhekai Du, Fengling Li, Ke Lu, Jingjing Li

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments CVPR 2024 camera ready

2403.06453 2024-03-12 cs.CV cs.GR 57%

FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

Yuki Tatsukawa, I-Chao Shen, Anran Qi, Yuki Koyama, Takeo Igarashi, Ariel Shamir

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments 11 pages. Eurographics 2024. https://yukistavailable.github.io/fontclip.github.io/

2403.06295 2024-03-12 cs.CV 57%

A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets

Thang Doan, Sima Behpour, Xin Li, Wenbin He, Liang Gou, Liu Ren

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

2403.05124 2024-03-11 cs.CV 57%

CLIP-Gaze: Towards General Gaze Estimation via Visual-Linguistic Model

Pengwei Yin, Guanzhong Zeng, Jingjing Wang, Di Xie

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to AAAI 2024

2403.03715 2024-03-07 cs.CV 57%

MeaCap: Memory-Augmented Zero-shot Image Captioning

Zequn Zeng, Yan Xie, Hao Zhang, Chiyu Chen, Zhengjue Wang, Bo Chen

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR2024

2403.02899 2024-03-06 cs.AI 57%

Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation

Zhekai Du, Xinyao Li, Fengling Li, Ke Lu, Lei Zhu, Jingjing Li

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI

2403.02714 2024-03-06 cs.CV 57%

DomainVerse: A Benchmark Towards Real-World Distribution Shifts For Tuning-Free Adaptive Domain Generalization

Feng Hou, Jin Yuan, Ying Yang, Yang Liu, Yang Zhang, Cheng Zhong, Zhongchao Shi, Jianping Fan, Yong Rui, Zhiqiang He

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Currently in review for ICML 2024

2311.00967 2024-02-21 cs.RO cs.AI cs.CL 57%

Vision-Language Interpreter for Robot Task Planning

Keisuke Shirai, Cristian C. Beltran-Hernandez, Masashi Hamaya, Atsushi Hashimoto, Shohei Tanaka, Kento Kawaharazuka, Kazutoshi Tanaka, Yoshitaka Ushiku, Shinsuke Mori

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI

Comments ICRA 2024

2305.15072 2024-02-20 cs.CV cs.MM 57%

PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathology

Yuxuan Sun, Chenglu Zhu, Sunyi Zheng, Kai Zhang, Lin Sun, Zhongyi Shui, Yunlong Zhang, Honglin Li, Lin Yang

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV

Comments 9 pages, 7 figures

2402.06015 2024-02-16 cs.CL cs.CV 57%

Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing

Yong Cao, Wenyan Li, Jiaang Li, Yifei Yuan, Antonia Karamolegkou, Daniel Hershcovich

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments work in process

2401.02814 2024-02-02 cs.RO cs.CV 57%

Object-Centric Instruction Augmentation for Robotic Manipulation

Junjie Wen, Yichen Zhu, Minjie Zhu, Jinming Li, Zhiyuan Xu, Zhengping Che, Chaomin Shen, Yaxin Peng, Dong Liu, Feifei Feng, Jian Tang

专题命中其他VLM ：MLLM(abstract)；分类 cs.CV

Comments accepted to ICRA2024

2401.17881 2024-02-01 cs.CV 57%

PVLR: Prompt-driven Visual-Linguistic Representation Learning for Multi-Label Image Recognition

Hao Tan, Zichang Tan, Jun Li, Jun Wan, Zhen Lei

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments 15 pages, 8 figures

2401.17217 2024-02-01 cs.HC cs.CV 57%

GazeGPT: Augmenting Human Capabilities using Gaze-contingent Contextual AI for Smart Eyewear

Robert Konrad, Nitish Padmanaban, J. Gabriel Buckmaster, Kevin C. Boyle, Gordon Wetzstein

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV

Comments Project video: https://youtu.be/AuDFHHTK_m8

1. 其他VLM 1566 篇

Auto-Encoding Morph-Tokens for Multimodal LLM

MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation

ZONE: Zero-Shot Instruction-Guided Local Editing

Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic

Learning Mutually Informed Representations for Characters and Subwords

Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction

Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens

AWOL: Analysis WithOut synthesis using Language

Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions

EarthNets: Empowering AI in Earth Observation

Unknown Prompt, the only Lacuna: Unveiling CLIP's Potential for Open Domain Generalization

VL-Mamba: Exploring State Space Models for Multimodal Learning

TiBiX: Leveraging Temporal Information for Bidirectional X-ray and Report Generation

MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control

EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Expression Recognition

CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning

Split to Merge: Unifying Separated Modalities for Unsupervised Domain Adaptation

FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets

CLIP-Gaze: Towards General Gaze Estimation via Visual-Linguistic Model

MeaCap: Memory-Augmented Zero-shot Image Captioning

Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation

DomainVerse: A Benchmark Towards Real-World Distribution Shifts For Tuning-Free Adaptive Domain Generalization

Vision-Language Interpreter for Robot Task Planning

PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathology

Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing

Object-Centric Instruction Augmentation for Robotic Manipulation

PVLR: Prompt-driven Visual-Linguistic Representation Learning for Multi-Label Image Recognition

GazeGPT: Augmenting Human Capabilities using Gaze-contingent Contextual AI for Smart Eyewear