视觉大模型 / VLM - arXivDaily 专题

2409.07129 2024-09-12 cs.CV 57%

MVLLaVA: An Intelligent Agent for Unified and Flexible Novel View Synthesis

Hanyu Jiang, Jian Xue, Xing Lan, Guohong Hu, Ke Lu

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments project page: https://jamesjg.github.io/MVLLaVA_homepage/

2402.02335 2024-09-10 cs.CV cs.IR 57%

Video Editing for Video Retrieval

Bin Zhu, Kevin Flanagan, Adriano Fragomeni, Michael Wray, Dima Damen

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2409.03583 2024-09-06 cs.CV 57%

Text-Guided Mixup Towards Long-Tailed Image Categorization

Richard Franklin, Jiawei Yao, Deyang Zhong, Qi Qian, Juhua Hu

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by BMVC'24, code is available at https://github.com/rsamf/text-guided-mixup

2409.03402 2024-09-06 cs.AI cs.RO 57%

Game On: Towards Language Models as RL Experimenters

Jingwei Zhang, Thomas Lampe, Abbas Abdolmaleki, Jost Tobias Springenberg, Martin Riedmiller

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.AI

2408.07500 2024-09-04 cs.CV 57%

Cross-Platform Video Person ReID: A New Benchmark Dataset and Adaptation Approach

Shizhou Zhang, Wenlong Luo, De Cheng, Qingchun Yang, Lingyan Ran, Yinghui Xing, Yanning Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Published at ECCV 2024

2408.14342 2024-08-30 cs.CV physics.med-ph 57%

Dual-Domain CLIP-Assisted Residual Optimization Perception Model for Metal Artifact Reduction

Xinrui Zhang, Ailong Cai, Shaoyu Wang, Linyuan Wang, Zhizhong Zheng, Lei Li, Bin Yan

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

Comments 14 pages, 18 figures

2408.10280 2024-08-28 cs.LG 57%

NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models

Cheng Lin, Lujun Li, Dezhi Li, Jie Zou, Wei Xue, Yike Guo

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.LG

Comments Work in progress, revisions ongoing

2408.14339 2024-08-27 cs.CV 57%

ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty

Xindi Wu, Dingli Yu, Yangsibo Huang, Olga Russakovsky, Sanjeev Arora

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

Comments 43 pages

2408.13909 2024-08-27 cs.CV cs.CL 57%

LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task

Ali Asgarov, Samir Rustamov

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2402.19474 2024-08-26 cs.CV 57%

The All-Seeing Project V2: Towards General Relation Comprehension of the Open World

Weiyun Wang, Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang, Qingyun Li, Lewei Lu, Xizhou Zhu, Yu Qiao, Jifeng Dai

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments Accepted to ECCV2024 main conference

2408.12141 2024-08-23 cs.CV 57%

TRRG: Towards Truthful Radiology Report Generation With Cross-modal Disease Clue Enhanced Large Language Model

Yuhao Wang, Chao Hao, Yawen Cui, Xinqi Su, Weicheng Xie, Tao Tan, Zitong Yu

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2408.10086 2024-08-20 cs.AI 57%

ARMADA: Attribute-Based Multimodal Data Augmentation

Xiaomeng Jin, Jeonghwan Kim, Yu Zhou, Kuan-Hao Huang, Te-Lin Wu, Nanyun Peng, Heng Ji

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

2403.17844 2024-08-20 cs.LG 57%

Mechanistic Design and Scaling of Hybrid Architectures

Michael Poli, Armin W Thomas, Eric Nguyen, Pragaash Ponnusamy, Björn Deiseroth, Kristian Kersting, Taiji Suzuki, Brian Hie, Stefano Ermon, Christopher Ré, Ce Zhang, Stefano Massaroli

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.LG

2408.09176 2024-08-20 cs.AI cs.CL cs.SC 57%

Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making

Siyu Wu, Alessandro Oltramari, Jonathan Francis, C. Lee Giles, Frank E. Ritter

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

Comments 20 pages, 8 figures, 2 tables

2308.07151 2024-08-20 cs.CV 57%

Diffusion Based Augmentation for Captioning and Retrieval in Cultural Heritage

Dario Cioni, Lorenzo Berlincioni, Federico Becattini, Alberto del Bimbo

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted at ICCV 2023 4th Workshop on e-Heritage

2408.06755 2024-08-14 cs.CV cs.CL 57%

Sumotosima: A Framework and Dataset for Classifying and Summarizing Otoscopic Images

Eram Anwarul Khan, Anas Anwarul Haq Khan

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments Work in Progress

2408.05706 2024-08-13 cs.CV 57%

Decoder Pre-Training with only Text for Scene Text Recognition

Shuai Zhao, Yongkun Du, Zhineng Chen, Yu-Gang Jiang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by ACM MM 2024

2408.04331 2024-08-09 cs.CL cs.CV 57%

Enhancing Journalism with AI: A Study of Contextualized Image Captioning for News Articles using LLMs and LMMs

Aliki Anagnostopoulou, Thiago Gouvea, Daniel Sonntag

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

2408.03511 2024-08-08 cs.CV cs.CL 57%

MoExtend: Tuning New Experts for Modality and Task Extension

Shanshan Zhong, Shanghua Gao, Zhongzhan Huang, Wushao Wen, Marinka Zitnik, Pan Zhou

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments ACL 2024 - SRW

2408.02001 2024-08-06 cs.CV 57%

AdaCBM: An Adaptive Concept Bottleneck Model for Explainable and Accurate Diagnosis

Townim F. Chowdhury, Vu Minh Hieu Phan, Kewen Liao, Minh-Son To, Yutong Xie, Anton van den Hengel, Johan W. Verjans, Zhibin Liao

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted at MICCAI 2024, the 27th International Conference on Medical Image Computing and Computer Assisted Intervention

2403.02875 2024-08-06 cs.CV cs.CL cs.IR 57%

Enhancing Conceptual Understanding in Multimodal Contrastive Learning through Hard Negative Samples

Philipp J. Rösch, Norbert Oswald, Michaela Geierhos, Jindřich Libovický

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2408.01363 2024-08-05 cs.IR cs.CL cs.CV cs.MM 57%

Toward Automatic Relevance Judgment using Vision--Language Models for Image--Text Retrieval Evaluation

Jheng-Hong Yang, Jimmy Lin

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments Accepted by ACM SIGIR 2024 LLM4Eval Workshop: https://llm4eval.github.io/papers

2408.00491 2024-08-02 cs.CL cs.CV cs.MM 57%

GalleryGPT: Analyzing Paintings with Large Multimodal Models

Yi Bin, Wenhao Shi, Yujuan Ding, Zhiqiang Hu, Zheng Wang, Yang Yang, See-Kiong Ng, Heng Tao Shen

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments Accepted as Oral Presentation at ACM Multimedia 2024

2407.20928 2024-07-31 cs.CV 57%

UniProcessor: A Text-induced Unified Low-level Image Processor

Huiyu Duan, Xiongkuo Min, Sijing Wu, Wei Shen, Guangtao Zhai

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2407.16977 2024-07-29 cs.CV cs.MM 57%

Selective Vision-Language Subspace Projection for Few-shot CLIP

Xingyu Zhu, Beier Zhu, Yi Tan, Shuo Wang, Yanbin Hao, Hanwang Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted as an Oral Paper at ACM Multimedia 2024

2308.03135 2024-07-25 cs.CV 57%

EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding

Jiazhou Zhou, Xu Zheng, Yuanhuiyi Lyu, Lin Wang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments ECCV 2024 Accepted. Camera-ready version with supplementary

2404.09624 2024-07-25 cs.CV 57%

AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception

Yipo Huang, Xiangfei Sheng, Zhichao Yang, Quan Yuan, Zhichao Duan, Pengfei Chen, Leida Li, Weisi Lin, Guangming Shi

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

Comments Accepted by ACMMM24

2407.15819 2024-07-23 cs.CV 57%

Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight

Ziyuan Huang, Kaixiang Ji, Biao Gong, Zhiwu Qing, Qinglong Zhang, Kecheng Zheng, Jian Wang, Jingdong Chen, Ming Yang

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

2407.15295 2024-07-23 cs.CV cs.SE 57%

VideoGameBunny: Towards vision assistants for video games

Mohammad Reza Taesiri, Cor-Paul Bezemer

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

2402.01293 2024-07-23 cs.LG cs.CL 57%

Can MLLMs Perform Text-to-Image In-Context Learning?

Yuchen Zeng, Wonjun Kang, Yicong Chen, Hyung Il Koo, Kangwook Lee

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.LG

Comments Accepted at COLM 2024

1. VLM训练与架构 5058 篇

MVLLaVA: An Intelligent Agent for Unified and Flexible Novel View Synthesis

Video Editing for Video Retrieval

Text-Guided Mixup Towards Long-Tailed Image Categorization

Game On: Towards Language Models as RL Experimenters

Cross-Platform Video Person ReID: A New Benchmark Dataset and Adaptation Approach

Dual-Domain CLIP-Assisted Residual Optimization Perception Model for Metal Artifact Reduction

NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models

ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty

LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task

The All-Seeing Project V2: Towards General Relation Comprehension of the Open World

TRRG: Towards Truthful Radiology Report Generation With Cross-modal Disease Clue Enhanced Large Language Model

ARMADA: Attribute-Based Multimodal Data Augmentation

Mechanistic Design and Scaling of Hybrid Architectures

Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making

Diffusion Based Augmentation for Captioning and Retrieval in Cultural Heritage

Sumotosima: A Framework and Dataset for Classifying and Summarizing Otoscopic Images

Decoder Pre-Training with only Text for Scene Text Recognition

Enhancing Journalism with AI: A Study of Contextualized Image Captioning for News Articles using LLMs and LMMs

MoExtend: Tuning New Experts for Modality and Task Extension

AdaCBM: An Adaptive Concept Bottleneck Model for Explainable and Accurate Diagnosis

Enhancing Conceptual Understanding in Multimodal Contrastive Learning through Hard Negative Samples

Toward Automatic Relevance Judgment using Vision--Language Models for Image--Text Retrieval Evaluation

GalleryGPT: Analyzing Paintings with Large Multimodal Models

UniProcessor: A Text-induced Unified Low-level Image Processor

Selective Vision-Language Subspace Projection for Few-shot CLIP

EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding

AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception

Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight

VideoGameBunny: Towards vision assistants for video games

Can MLLMs Perform Text-to-Image In-Context Learning?