视觉大模型 / VLM - arXivDaily 专题

2410.24018 2024-11-01 cs.LG cs.CV 62%

Bayesian-guided Label Mapping for Visual Reprogramming

Chengyi Cai, Zesheng Ye, Lei Feng, Jianzhong Qi, Feng Liu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2410.22927 2024-10-31 cs.CV cs.LG 62%

An Individual Identity-Driven Framework for Animal Re-Identification

Yihao Wu, Di Zhao, Jingfeng Zhang, Yun Sing Koh

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments 10 pages

2410.22362 2024-10-31 eess.IV cs.AI cs.CV 62%

MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation

Jialin Luo, Yuanzhi Wang, Ziqi Gu, Yide Qiu, Shuaizhen Yao, Fuyun Wang, Chunyan Xu, Wenhua Zhang, Dan Wang, Zhen Cui

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024

2410.13030 2024-10-18 cs.CV cs.CL cs.LG 62%

Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts

Sri Harsha Dumpala, Aman Jaiswal, Chandramouli Sastry, Evangelos Milios, Sageev Oore, Hassan Sajjad

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2410.02027 2024-10-10 cs.CV cs.AI 62%

Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval

Kyle Buettner, Adriana Kovashka

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2024 Main - Short

2403.15837 2024-10-10 cs.CV cs.CL cs.LG 62%

Centered Masking for Language-Image Pre-Training

Mingliang Liang, Martha Larson

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2409.19474 2024-10-08 cs.CV cs.AI 62%

FairPIVARA: Reducing and Assessing Biases in CLIP-Based Multimodal Models

Diego A. B. Moreira, Alef Iury Ferreira, Jhessica Silva, Gabriel Oliveira dos Santos, Luiz Pereira, João Medrado Gondim, Gustavo Bonil, Helena Maia, Nádia da Silva, Simone Tiemi Hashiguti, Jefersson A. dos Santos, Helio Pedrini, Sandra Avila

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 10 figures. Accepted to 35th British Machine Vision Conference (BMVC 2024), Workshop on Privacy, Fairness, Accountability and Transparency in Computer Vision

2410.03303 2024-10-07 cs.LG cs.CV 62%

SELU: Self-Learning Embodied MLLMs in Unknown Environments

Boyu Li, Haobin Jiang, Ziluo Ding, Xinrun Xu, Haoran Li, Dongbin Zhao, Zongqing Lu

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.LG

2409.19610 2024-10-01 cs.LG cs.CL cs.CV 62%

Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method

Bikang Pan, Wei Huang, Ye Shi

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2406.05596 2024-09-20 cs.CV cs.LG 62%

Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classification

Yunhe Gao, Difei Gu, Mu Zhou, Dimitris Metaxas

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments MICCAI 2024 Early Accept

2407.18134 2024-09-13 cs.CV cs.LG 62%

$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

Vlad Sobal, Mark Ibrahim, Randall Balestriero, Vivien Cabannes, Diane Bouchacourt, Pietro Astolfi, Kyunghyun Cho, Yann LeCun

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2409.05405 2024-09-12 cs.CV cs.AI cs.IR cs.MM 62%

A Survey of Multimodal Composite Editing and Retrieval

Suyan Li, Fuxiang Huang, Lei Zhang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 20 pages, 3 figures, and 11 tables

2407.19832 2024-08-22 cs.CV cs.AI cs.CL 62%

ML-Mamba: Efficient Multi-Modal Large Language Model Utilizing Mamba-2

Wenjun Huang, Jiakai Pan, Jiahao Tang, Yanyu Ding, Yifei Xing, Yuhe Wang, Zhengzhuo Wang, Jianguo Hu

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

2408.08652 2024-08-19 cs.LG cs.AI cs.HC 62%

TextCAVs: Debugging vision models using text

Angus Nicolson, Yarin Gal, J. Alison Noble

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments 11 pages, 2 figures. Accepted at iMIMIC Workshop at MICCAI 2024

2408.05337 2024-08-13 cs.CV cs.AI 62%

VACoDe: Visual Augmented Contrastive Decoding

Sihyeon Kim, Boryeong Cho, Sangmin Bae, Sumyeong Ahn, Se-Young Yun

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 7 figures

2402.17510 2024-08-02 cs.CV cs.AI 62%

Demonstrating and Reducing Shortcuts in Vision-Language Representation Learning

Maurits Bleeker, Mariya Hendriksen, Andrew Yates, Maarten de Rijke

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 27 pages, accepted at TMLR

2407.09760 2024-07-16 cs.CV cs.AI 62%

ICCV23 Visual-Dialog Emotion Explanation Challenge: SEU_309 Team Technical Report

Yixiao Yuan, Yingzhe Peng

专题命中其他VLM ：vision language model(abstract)；分类 cs.CV、cs.AI

2209.07341 2024-07-10 cs.LG cs.CR cs.CV 62%

Does CLIP Know My Face?

Dominik Hintersdorf, Lukas Struppek, Manuel Brack, Felix Friedrich, Patrick Schramowski, Kristian Kersting

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Published in the Journal of Artificial Intelligence Research (JAIR)

Journal ref Journal of Artificial Intelligence Research (JAIR), Vol. 80 (2024)

2407.03921 2024-07-08 cs.LG cs.CV 62%

Concept Bottleneck Models Without Predefined Concepts

Simon Schrodi, Julian Schur, Max Argus, Thomas Brox

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2406.18589 2024-06-28 cs.CV cs.LG 62%

Text-Guided Alternative Image Clustering

Andreas Stephan, Lukas Miklautz, Collin Leiber, Pedro Henrique Luz de Araujo, Dominik Répás, Claudia Plant, Benjamin Roth

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2406.17781 2024-06-27 cs.CV cs.AI cs.HC 62%

Large Language Models estimate fine-grained human color-concept associations

Kushin Mukherjee, Timothy T. Rogers, Karen B. Schloss

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

2406.14481 2024-06-21 cs.LG cs.AI cs.NE q-bio.NC 62%

Revealing Vision-Language Integration in the Brain with Multimodal Networks

Vighnesh Subramaniam, Colin Conwell, Christopher Wang, Gabriel Kreiman, Boris Katz, Ignacio Cases, Andrei Barbu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments ICML 2024; 23 pages, 11 figures

2406.12946 2024-06-21 eess.AS cs.AI cs.CL cs.LG 62%

Instruction Data Generation and Unsupervised Adaptation for Speech Language Models

Vahid Noroozi, Zhehuai Chen, Somshubra Majumdar, Steve Huang, Jagadeesh Balam, Boris Ginsburg

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted for Interspeech 2024

2406.10328 2024-06-18 cs.CV cs.CL cs.LG 62%

From Pixels to Prose: A Large Dataset of Dense Image Captions

Vasu Singla, Kaiyu Yue, Sukriti Paul, Reza Shirkavand, Mayuka Jayawardhana, Alireza Ganjdanesh, Heng Huang, Abhinav Bhatele, Gowthami Somepalli, Tom Goldstein

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments pixelprose 16M dataset

2406.10318 2024-06-18 cs.CV cs.AI 62%

Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding

Tuo Zhang, Tiantian Feng, Yibin Ni, Mengqin Cao, Ruying Liu, Katharine Butler, Yanjun Weng, Mi Zhang, Shrikanth S. Narayanan, Salman Avestimehr

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

2404.16829 2024-05-27 cs.CV cs.AI cs.CL 62%

Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials

Ye Fang, Zeyi Sun, Tong Wu, Jiaqi Wang, Ziwei Liu, Gordon Wetzstein, Dahua Lin

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://sunzey.github.io/Make-it-Real/

2403.15593 2024-05-20 cs.CV cs.LG 62%

FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs

Sepehr Dehdashtian, Lan Wang, Vishnu Naresh Boddeti

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments The Twelfth International Conference on Learning Representations (ICLR) 2024

2310.20225 2024-04-30 cs.CV cs.AI 62%

A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction

Yu Hao, Fan Yang, Hao Huang, Shuaihang Yuan, Sundeep Rangan, John-Ross Rizzo, Yao Wang, Yi Fang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

2404.15532 2024-04-25 cs.HC cs.AI cs.CL cs.CV cs.MA 62%

BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis

Shuhang Lin, Wenyue Hua, Lingyao Li, Che-Jui Chang, Lizhou Fan, Jianchao Ji, Hang Hua, Mingyu Jin, Jiebo Luo, Yongfeng Zhang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 26 pages, 14 figures The data and code for this project are accessible at https://github.com/agiresearch/battleagent

2302.10174 2024-04-02 cs.CV cs.LG 62%

Towards Universal Fake Image Detectors that Generalize Across Generative Models

Utkarsh Ojha, Yuheng Li, Yong Jae Lee

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

1. 其他VLM 1565 篇

Bayesian-guided Label Mapping for Visual Reprogramming

An Individual Identity-Driven Framework for Animal Re-Identification

MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation

Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts

Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval

Centered Masking for Language-Image Pre-Training

FairPIVARA: Reducing and Assessing Biases in CLIP-Based Multimodal Models

SELU: Self-Learning Embodied MLLMs in Unknown Environments

Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method

Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classification

$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

A Survey of Multimodal Composite Editing and Retrieval

ML-Mamba: Efficient Multi-Modal Large Language Model Utilizing Mamba-2

TextCAVs: Debugging vision models using text

VACoDe: Visual Augmented Contrastive Decoding

Demonstrating and Reducing Shortcuts in Vision-Language Representation Learning

ICCV23 Visual-Dialog Emotion Explanation Challenge: SEU_309 Team Technical Report

Does CLIP Know My Face?

Concept Bottleneck Models Without Predefined Concepts

Text-Guided Alternative Image Clustering

Large Language Models estimate fine-grained human color-concept associations

Revealing Vision-Language Integration in the Brain with Multimodal Networks

Instruction Data Generation and Unsupervised Adaptation for Speech Language Models

From Pixels to Prose: A Large Dataset of Dense Image Captions

Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding

Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials

FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs

A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction

BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis

Towards Universal Fake Image Detectors that Generalize Across Generative Models