视觉大模型 / VLM - arXivDaily 专题

2402.05741 2024-10-24 cs.RO cs.AI cs.CV cs.LG 67%

Real-World Robot Applications of Foundation Models: A Review

Kento Kawaharazuka, Tatsuya Matsushima, Andrew Gambardella, Jiaxian Guo, Chris Paxton, Andy Zeng

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2410.08612 2024-10-14 cs.CV cs.AI cs.LG 67%

Synth-SONAR: Sonar Image Synthesis with Enhanced Diversity and Realism via Dual Diffusion Models and GPT Prompting

Purushothaman Natarajan, Kamal Basha, Athira Nambiar

专题命中其他VLM ：visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 12 pages, 5 tables and 9 figures

2410.08172 2024-10-11 cs.RO cs.AI cs.CV cs.LG 67%

On the Evaluation of Generative Robotic Simulations

Feng Chen, Botian Xu, Pu Hua, Peiqi Duan, Yanchao Yang, Yi Ma, Huazhe Xu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project website: https://sites.google.com/view/evaltasks

2404.16193 2024-09-23 cs.CV cs.AI cs.LG cs.MM eess.IV 67%

Improving Multi-label Recognition using Class Co-Occurrence Probabilities

Samyak Rawlekar, Shubhang Bhatnagar, Vishnuvardhan Pogunulu Srinivasulu, Narendra Ahuja

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to ICPR 2024, CVPR workshops 2024

2409.09362 2024-09-17 cs.CL 67%

Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM

Yuanjie Lyu, Tong Xu, Zihan Niu, Bo Peng, Jing Ke, Enhong Chen

专题命中其他VLM ：multimodal large language model(abstract)；MLLM(abstract)

2407.08583 2024-08-06 cs.AI cs.CV cs.LG 67%

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective

Zhen Qin, Daoyuan Chen, Wenhao Zhang, Liuyi Yao, Yilun Huang, Bolin Ding, Yaliang Li, Shuiguang Deng

专题命中其他VLM ：MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Ongoing work. 21 pages. Related materials are continually maintained and available at https://github.com/modelscope/data-juicer/blob/main/docs/awesome_llm_data.md

2408.01417 2024-08-05 cs.CL cs.AI cs.CV cs.LG 67%

Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs

Yilun Hua, Yoav Artzi

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to COLM 2024

2407.03056 2024-07-31 cs.CV cs.AI cs.LG 67%

Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

Marco Mistretta, Alberto Baldrati, Marco Bertini, Andrew D. Bagdanov

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted for publication at ECCV24

2404.15592 2024-07-23 cs.CV cs.AI cs.CL cs.IR cs.LG 67%

ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction

Henry Peng Zou, Vinay Samuel, Yue Zhou, Weizhi Zhang, Liancheng Fang, Zihe Song, Philip S. Yu, Cornelia Caragea

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ACL 2024 (Findings) - Scores: Soundness - 4/4/4, Dataset - 4/4/4, Overall Assessment - 4/3.5/3.5, Meta - 4

2407.14412 2024-07-22 cs.CV cs.AI cs.LG 67%

DEAL: Disentangle and Localize Concept-level Explanations for VLMs

Tang Li, Mengmeng Ma, Xi Peng

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments In Proceedings of the European Conference on Computer Vision (ECCV), 2024

2406.03586 2024-06-11 cs.CV cs.AI cs.LG 67%

CountCLIP -- [Re] Teaching CLIP to Count to Ten

Harshvardhan Mestha, Tejas Agrawal, Karan Bania, Shreyas V, Yash Bhisikar

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2406.02780 2024-06-06 cs.CV cs.AI cs.LG 67%

LADI v2: Multi-label Dataset and Classifiers for Low-Altitude Disaster Imagery

Samuel Scheele, Katherine Picchione, Jeffrey Liu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2406.02554 2024-06-06 eess.AS cs.AI cs.CL cs.CV cs.LG cs.MM 67%

Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition

Shijian Deng, Erin E. Kosloski, Siddhi Patel, Zeke A. Barnett, Yiyang Nan, Alexander Kaplan, Sisira Aarukapalli, William T. Doan, Matthew Wang, Harsh Singh, Pamela R. Rollins, Yapeng Tian

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2404.18532 2024-05-16 cs.CL cs.AI cs.CV cs.LG 67%

MileBench: Benchmarking MLLMs in Long Context

Dingjie Song, Shunian Chen, Guiming Hardy Chen, Fei Yu, Xiang Wan, Benyou Wang

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 31 pages, 13 figures, 14 tables; We add results of GPT-4o in this version

2404.16305 2024-04-29 cs.MM cs.SD eess.AS 67%

Semantically consistent Video-to-Audio Generation using Multimodal Language Large Model

Gehui Chen, Guan'an Wang, Xiaowen Huang, Jitao Sang

专题命中其他VLM ：multimodal large language model(abstract)；MLLM(abstract)

2404.08937 2024-04-16 cs.CV cs.AI cs.LG 67%

ChimpVLM: Ethogram-Enhanced Chimpanzee Behaviour Recognition

Otto Brookes, Majid Mirmehdi, Hjalmar Kuhl, Tilo Burghardt

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2402.08473 2024-02-14 cs.CV cs.AI cs.LG 67%

Intriguing Differences Between Zero-Shot and Systematic Evaluations of Vision-Language Transformer Models

Shaeke Salman, Md Montasir Bin Shams, Xiuwen Liu, Lingjiong Zhu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 30 pages, 30 figures

2308.07931 2024-01-02 cs.CV cs.AI cs.CL cs.LG cs.RO 67%

Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation

William Shen, Ge Yang, Alan Yu, Jansen Wong, Leslie Pack Kaelbling, Phillip Isola

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project website at https://f3rm.csail.mit.edu, Accepted at the 7th Annual Conference on Robot Learning (CoRL), 2023 in Atlanta, US

2312.03818 2023-12-15 cs.CV cs.AI cs.CL cs.LG 67%

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

Zeyi Sun, Ye Fang, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments project page: https://aleafy.github.io/alpha-clip code: https://github.com/SunzeY/AlphaCLIP

2312.04344 2023-12-13 cs.CL cs.AI cs.CV cs.LG 67%

Enhancing Medical Task Performance in GPT-4V: A Comprehensive Study on Prompt Engineering Strategies

Pengcheng Chen, Ziyan Huang, Zhongying Deng, Tianbin Li, Yanzhou Su, Haoyu Wang, Jin Ye, Yu Qiao, Junjun He

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

2310.10625 2023-10-17 cs.CV cs.AI cs.LG cs.RO 67%

Video Language Planning

Yilun Du, Mengjiao Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B. Tenenbaum, Leslie Kaelbling, Andy Zeng, Jonathan Tompson

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments https://video-language-planning.github.io/

2206.00621 2023-06-13 cs.CL cs.AI cs.CV cs.LG 67%

Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training

Yan Zeng, Wangchunshu Zhou, Ao Luo, Ziming Cheng, Xinsong Zhang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ACL 2023

2201.06289 2022-06-10 cs.CV cs.AI cs.LG 67%

The CLEAR Benchmark: Continual LEArning on Real-World Imagery

Zhiqiu Lin, Jia Shi, Deepak Pathak, Deva Ramanan

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project site: https://clear-benchmark.github.io

2608.10932 2026-08-12 cs.CV cs.AI 新提交 62%

Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation

基于几何知识蒸馏的时序锚定组合式相机运动理解

Dazhao Du, Shiyan Du, Jian Liu, Yongjian Yu, Bohai Gu, Tao Han, Hualuo Liu, Eric Liu, Yujia Zhang, Xi Chen, Song Guo

机构 * The Hong Kong University of Science and Technology（香港科技大学）； Tencent（腾讯）

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

AI总结本研究提出CamDistill方法，通过几何知识蒸馏实现时序锚定的组合式相机运动理解，推出含4229个片段的CamChoreo基准，提升了相机运动识别的细粒度与效率。

2608.07663 2026-08-11 cs.CV cs.AI cs.CL 新提交 62%

Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding

保持简洁：用于超长视频理解的多键情景记忆检索

Yeeun Choi, Youngbeom Yoo, Joon-Young Lee, Hyolim Kang, Seon Joo Kim

机构 * Yonsei University（延世大学）； Adobe Research（奥多比研究院）

专题命中其他VLM ：MLLM(abstract_cn)；分类 cs.CV、cs.AI

AI总结针对超长视频理解的两阶段范式需求，提出MERIT框架，通过多键表示与按需时间扩展实现精准检索，在三个长视频基准数据集上取得最优性能。

Comments Accepted to ECCV 2026 (Oral). Project Page: https://choi-yeeun.github.io/MERIT/

2608.05233 2026-08-07 cs.AI cs.CV 新提交 62%

Coherence-Oriented Dream Scene Visualisation

面向连贯性的梦境场景可视化

Azra Açıl, Simon Colton

机构 * School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦玛丽女王大学电子工程与计算机科学学院）

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结该研究提出DSV系统，通过大型语言模型拆分梦境描述为四部分，结合文本到图像模型生成连贯的四面板图像序列，经DreamBank数据集50次可视化评估，用CLIP等模型指标验证了其质量、保真度与连贯性。

Comments short paper accepted at ICCC 2026

2608.02803 2026-08-05 cs.CV cs.AI 新提交 62%

SAGE: Semantic Explainability of Attention-Based Survival Models in Computational Pathology

SAGE：计算病理学中基于注意力的生存模型的语义可解释性

Abdallah Lamane, Abdul Rahman Diab, Ren-Chin Wu, William Lotter

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结该研究提出SAGE框架，可从ABMIL模型提取全局语义解释，在7个TCGA癌症队列的生存预测中恢复预后特征，揭示癌症特异性生物学，为病理学家解释模型行为提供支持。

Comments Proceedings of the MICCAI Workshop on Interpretability of Machine Intelligence in Medical Image Computing (iMIMIC)

2608.00714 2026-08-04 cs.CV cs.AI 新提交 62%

Coverage-Driven Adaptive Keyframe Selection for Video Understanding

面向视频理解的覆盖驱动型自适应关键帧选择

Junyang Zhang, Puhan Luo, Chen Tang, Yuxi Shi, Xiang-Yang Li

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结本文针对视频理解中LVLM处理大量帧计算开销大的问题，提出无需训练的CSES关键帧选择器，通过覆盖驱动的自适应策略减少需评分和选择的帧数量，同时保持准确率并提升选择速度。

2607.28269 2026-07-31 cs.CV cs.AI cs.MM 新提交 62%

Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation

Theia：用于无数据蒸馏的Incidents1M数据集的大规模多模态字幕生成与自动验证

Simone Giano, Lorenzo Severini, Alessandro Galdelli, Adriano Mancini

机构 * Università Politecnica delle Marche（马尔凯理工大学）

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

AI总结本研究针对灾害领域多模态数据集的缺陷，提出方法构建并自动验证Incidents1M数据集，生成高保真字幕，其语义一致性达78.65/100，为跨模态知识蒸馏提供了可扩展框架。

2604.06614 2026-07-16 cs.CV cs.LG 版本更新 62%

Holistic Optimal Label Selection for Robust Prompt Learning under Partial Labels

整体最优标签选择用于在部分标签下的鲁棒提示学习

Yaqi Zhao, Haoliang Sun, Yating Wang, Yongshun Gong, Yilong Yin

机构 * Shandong University, Jinan, China（山东大学（济南，中国））

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

AI总结本文提出Holistic Optimal Label Selection方法，通过局部密度过滤和全局最优传输策略，提升部分标签下的提示学习性能，实验表明其在八个基准数据集上表现优异。

Comments ECCV 2026

1. 其他VLM 1565 篇

Real-World Robot Applications of Foundation Models: A Review

Synth-SONAR: Sonar Image Synthesis with Enhanced Diversity and Realism via Dual Diffusion Models and GPT Prompting

On the Evaluation of Generative Robotic Simulations

Improving Multi-label Recognition using Class Co-Occurrence Probabilities

Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective

Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs

Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction

DEAL: Disentangle and Localize Concept-level Explanations for VLMs

CountCLIP -- [Re] Teaching CLIP to Count to Ten

LADI v2: Multi-label Dataset and Classifiers for Low-Altitude Disaster Imagery

Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition

MileBench: Benchmarking MLLMs in Long Context

Semantically consistent Video-to-Audio Generation using Multimodal Language Large Model

ChimpVLM: Ethogram-Enhanced Chimpanzee Behaviour Recognition

Intriguing Differences Between Zero-Shot and Systematic Evaluations of Vision-Language Transformer Models

Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

Enhancing Medical Task Performance in GPT-4V: A Comprehensive Study on Prompt Engineering Strategies

Video Language Planning

Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training

The CLEAR Benchmark: Continual LEArning on Real-World Imagery

Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation

Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding

Coherence-Oriented Dream Scene Visualisation

SAGE: Semantic Explainability of Attention-Based Survival Models in Computational Pathology

Coverage-Driven Adaptive Keyframe Selection for Video Understanding

Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation

Holistic Optimal Label Selection for Robust Prompt Learning under Partial Labels