视觉大模型 / VLM - arXivDaily 专题

2412.01725 2024-12-03 cs.CV 70%

Attacks on multimodal models

Viacheslav Iablochnikov, Alexander Rogachev

专题命中 VLM训练与架构：VLM(abstract)；LLaVA(abstract)；分类 cs.CV

Comments 19 pages, 13 figures, 3 tables

2412.01343 2024-12-03 cs.CV 70%

MoTrans: Customized Motion Transfer with Text-driven Video Diffusion Models

Xiaomin Li, Xu Jia, Qinghe Wang, Haiwen Diao, Mengmeng Ge, Pengxiang Li, You He, Huchuan Lu

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by ACM MM 2024, code will be released in https://github.com/XiaominLi1997/MoTrans

2407.11522 2024-12-03 cs.CV 70%

FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models

Pengxiang Li, Zhi Gao, Bofei Zhang, Tao Yuan, Yuwei Wu, Mehrtash Harandi, Yunde Jia, Song-Chun Zhu, Qing Li

专题命中 VLM训练与架构：vision language model(abstract)；LLaVA(abstract)；分类 cs.CV

Comments NeurIPS 2024 Dataset & Benchmark Track

2411.17837 2024-11-28 cs.CV 70%

OracleSage: Towards Unified Visual-Linguistic Understanding of Oracle Bone Scripts through Cross-Modal Knowledge Fusion

Hanqi Jiang, Yi Pan, Junhao Chen, Zhengliang Liu, Yifan Zhou, Peng Shu, Yiwei Li, Huaqin Zhao, Stephen Mihm, Lewis C Howe, Tianming Liu

专题命中 VLM训练与架构：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

2411.16828 2024-11-27 cs.CV 70%

CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions

Yanqing Liu, Xianhang Li, Zeyu Wang, Bingchen Zhao, Cihang Xie

专题命中 VLM训练与架构：LLaVA(abstract)；MLLM(abstract)；分类 cs.CV

Comments 12 pages

2411.13909 2024-11-25 cs.CV 70%

Panther: Illuminate the Sight of Multimodal LLMs with Instruction-Guided Visual Prompts

Honglin Li, Yuting Gao, Chenglu Zhu, Jingdong Chen, Ming Yang, Lin Yang

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2411.10161 2024-11-18 cs.CV 70%

SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuning

Zewen Chen, Juan Wang, Wen Wang, Sunhan Xu, Hang Xiong, Yun Zeng, Jian Guo, Shuxun Wang, Chunfeng Yuan, Bing Li, Weiming Hu

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2410.15735 2024-11-15 cs.AI 70%

AutoTrain: No-code training for state-of-the-art models

Abhishek Thakur

专题命中 VLM训练与架构：VLM(abstract)；visual language model(abstract)；分类 cs.AI

2408.09984 2024-11-13 cs.CV 70%

Boosting Open-Domain Continual Learning via Leveraging Intra-domain Category-aware Prototype

Yadong Lu, Shitian Zhao, Boxiang Yun, Dongsheng Jiang, Yin Li, Qingli Li, Yan Wang

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

2310.01412 2024-11-12 cs.CV cs.RO 70%

DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model

Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee. K. Wong, Zhenguo Li, Hengshuang Zhao

专题命中 VLM训练与架构：grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Accepted by RA-L. The project page is available at https://tonyxuqaq.github.io/projects/DriveGPT4/

2411.00988 2024-11-05 cs.CV 70%

Retrieval-enriched zero-shot image classification in low-resource domains

Nicola Dall'Asen, Yiming Wang, Enrico Fini, Elisa Ricci

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted to EMNLP 2024 (Main)

2410.21237 2024-10-29 cs.AI 70%

Hierarchical Knowledge Graph Construction from Images for Scalable E-Commerce

Zhantao Yang, Han Zhang, Fangyi Chen, Anudeepsekhar Bolimera, Marios Savvides

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

2407.00557 2024-10-24 cs.CV 70%

Explaining Chest X-ray Pathology Models using Textual Concepts

Vijay Sadashivaiah, Pingkun Yan, James A. Hendler

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted at NeurIPS'24 workshop on Advancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond (AIM-FM)

2410.11829 2024-10-16 cs.CV 70%

MMFuser: Multimodal Multi-Layer Feature Fuser for Fine-Grained Vision-Language Understanding

Yue Cao, Yangzhou Liu, Zhe Chen, Guangchen Shi, Wenhai Wang, Danhuai Zhao, Tong Lu

专题命中 VLM训练与架构：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments 11 pages, 6 figures, technical report

2410.09286 2024-10-15 cs.RO cs.AI 70%

Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videos

Harsh Mahesheka, Zhixian Xie, Zhaoran Wang, Wanxin Jin

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

2410.07880 2024-10-11 cs.CL cs.LG 70%

Unsupervised Data Validation Methods for Efficient Model Training

Yurii Paniv

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.LG

2406.16562 2024-10-11 cs.CV cs.CL 70%

EVALALIGN: Supervised Fine-Tuning Multimodal LLMs with Human-Aligned Data for Evaluating Text-to-Image Models

Zhiyu Tan, Xiaomeng Yang, Luozheng Qin, Mengping Yang, Cheng Zhang, Hao Li

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments Project page: https://sais-fuxi.github.io/projects/evalalign/

2410.06777 2024-10-10 cs.CV 70%

HERM: Benchmarking and Enhancing Multimodal LLMs for Human-Centric Understanding

Keliang Li, Zaifei Yang, Jiahe Zhao, Hongze Shen, Ruibing Hou, Hong Chang, Shiguang Shan, Xilin Chen

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

2410.02874 2024-10-08 cs.RO cs.AI 70%

Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL

Naoaki Kanazawa, Kento Kawaharazuka, Yoshiki Obinata, Kei Okada, Masayuki Inaba

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

Comments Accepted at Advanced Robotics, website - https://kanazawanaoaki.github.io/cook-from-recipe-pddl/

2406.01020 2024-10-08 cs.CV 70%

ATTIQA: Generalizable Image Quality Feature Extractor using Attribute-aware Pretraining

Daekyu Kwon, Dongyoung Kim, Sehwan Ki, Younghyun Jo, Hyong-Euk Lee, Seon Joo Kim

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.CV

2404.18202 2024-10-01 cs.AI cs.MM 70%

WorldGPT: Empowering LLM as Multimodal World Model

Zhiqi Ge, Hongzhe Huang, Mingze Zhou, Juncheng Li, Guoming Wang, Siliang Tang, Yueting Zhuang

专题命中 VLM训练与架构：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments update v2

2404.16538 2024-10-01 cs.CV 70%

OpenDlign: Open-World Point Cloud Understanding with Depth-Aligned Images

Ye Mao, Junpeng Jing, Krystian Mikolajczyk

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments 17 pages (Accepted by NeurIPS 2024)

2409.16261 2024-09-25 cs.CV 70%

CDChat: A Large Multimodal Model for Remote Sensing Change Description

Mubashir Noman, Noor Ahsan, Muzammal Naseer, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Fahad Shahbaz Khan

专题命中 VLM训练与架构：LLaVA(abstract)；grounding(abstract)；分类 cs.CV

2409.15574 2024-09-25 cs.CV 70%

Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model

Jing Wei Tan, SeungKyu Kim, Eunsu Kim, Sung Hak Lee, Sangjeong Ahn, Won-Ki Jeong

专题命中 VLM训练与架构：vision language model(abstract)；VLM(abstract)；分类 cs.CV

2409.13540 2024-09-23 cs.CV 70%

FullAnno: A Data Engine for Enhancing Image Comprehension of MLLMs

Jing Hao, Yuxiang Zhao, Song Chen, Yanpeng Sun, Qiang Chen, Gang Zhang, Kun Yao, Errui Ding, Jingdong Wang

专题命中 VLM训练与架构：LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments 7 pages, 5 figures, 2 tables

2409.08582 2024-09-16 cs.CV 70%

ChangeChat: An Interactive Model for Remote Sensing Change Analysis via Multimodal Instruction Tuning

Pei Deng, Wenqian Zhou, Hanlin Wu

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments 5 pages, 2 figures

2409.01835 2024-09-10 cs.CV cs.CL 70%

Towards Generative Class Prompt Learning for Fine-grained Visual Recognition

Soumitri Chattopadhyay, Sanket Biswas, Emanuele Vivoli, Josep Lladós

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Accepted in BMVC 2024

2405.09771 2024-09-04 cs.LG 70%

Harmonizing Generalization and Personalization in Federated Prompt Learning

Tianyu Cui, Hongxia Li, Jingya Wang, Ye Shi

专题命中 VLM训练与架构：vision-language model(abstract)；VLM(abstract)；分类 cs.LG

2408.16730 2024-08-30 cs.CV 70%

VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computation

Shiwei Wu, Joya Chen, Kevin Qinghong Lin, Qimeng Wang, Yan Gao, Qianli Xu, Tong Xu, Yao Hu, Enhong Chen, Mike Zheng Shou

专题命中 VLM训练与架构：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

2407.02392 2024-08-29 cs.CV 70%

TokenPacker: Efficient Visual Projector for Multimodal LLM

Wentong Li, Yuqian Yuan, Jian Liu, Dongqi Tang, Song Wang, Jie Qin, Jianke Zhu, Lei Zhang

专题命中 VLM训练与架构：visual reasoning(abstract)；MLLM(abstract)；分类 cs.CV

Comments 16 pages, Codes:https://github.com/CircleRadon/TokenPacker

1. VLM训练与架构 5048 篇

Attacks on multimodal models

MoTrans: Customized Motion Transfer with Text-driven Video Diffusion Models

FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models

OracleSage: Towards Unified Visual-Linguistic Understanding of Oracle Bone Scripts through Cross-Modal Knowledge Fusion

CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions

Panther: Illuminate the Sight of Multimodal LLMs with Instruction-Guided Visual Prompts

SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuning

AutoTrain: No-code training for state-of-the-art models

Boosting Open-Domain Continual Learning via Leveraging Intra-domain Category-aware Prototype

DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model

Retrieval-enriched zero-shot image classification in low-resource domains

Hierarchical Knowledge Graph Construction from Images for Scalable E-Commerce

Explaining Chest X-ray Pathology Models using Textual Concepts

MMFuser: Multimodal Multi-Layer Feature Fuser for Fine-Grained Vision-Language Understanding

Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videos

Unsupervised Data Validation Methods for Efficient Model Training

EVALALIGN: Supervised Fine-Tuning Multimodal LLMs with Human-Aligned Data for Evaluating Text-to-Image Models

HERM: Benchmarking and Enhancing Multimodal LLMs for Human-Centric Understanding

Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL

ATTIQA: Generalizable Image Quality Feature Extractor using Attribute-aware Pretraining

WorldGPT: Empowering LLM as Multimodal World Model

OpenDlign: Open-World Point Cloud Understanding with Depth-Aligned Images

CDChat: A Large Multimodal Model for Remote Sensing Change Description

Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model

FullAnno: A Data Engine for Enhancing Image Comprehension of MLLMs

ChangeChat: An Interactive Model for Remote Sensing Change Analysis via Multimodal Instruction Tuning

Towards Generative Class Prompt Learning for Fine-grained Visual Recognition

Harmonizing Generalization and Personalization in Federated Prompt Learning

VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computation

TokenPacker: Efficient Visual Projector for Multimodal LLM