视觉大模型 / VLM

How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding

Zhuoran Yu, Yong Jae Lee

机构 * University of Wisconsin–Madison（威斯康星大学麦迪逊分校） 纠错

专题命中 视觉定位与Grounding ：grounding(title,abstract)；LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by COLM 2025

A Training-Free, Task-Agnostic Framework for Enhancing MLLM Performance on High-Resolution Images

Jaeseong Lee, Yeeun Choi, Heechan Choi, Hanjung Kim, Seonjoo Kim

机构 * Yonsei University（延世大学） 纠错

专题命中 视觉定位与Grounding ：MLLM(title,abstract)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at CVPR 2025 Workshop on Emergent Visual Abilities and Limits of Foundation Models

Grounding-Aware Token Pruning: Recovering from Drastic Performance Drops in Visual Grounding Caused by Pruning

Tzu-Chun Chien, Chieh-Kai Lin, Shiang-Feng Tsai, Ruei-Chi Lai, Hung-Jen Chen, Min Sun

机构 * National Tsing Hua University (NTHU)（国立清华大学） 纠错

专题命中 视觉定位与Grounding ：grounding(title,abstract)；LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis

Ruixuan Zhang, Beichen Wang, Juexiao Zhang, Zilin Bian, Chen Feng, Kaan Ozbay

机构 * New York University（纽约大学） 纠错

专题命中 视觉定位与Grounding ：multimodal large language model(title,abstract)；grounding(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents

Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, Yu Su

机构 * The Ohio State University（俄亥俄州立大学） ； Orby AI（Orby人工智能） 纠错

专题命中 视觉定位与Grounding ：grounding(title,abstract)；LLaVA(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICLR 2025 (Oral). Project Homepage: https://osu-nlp-group.github.io/UGround/

Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models

Woody Haosheng Gan, Deqing Fu, Julian Asilis, Ollie Liu, Dani Yogatama, Vatsal Sharan, Robin Jia, Willie Neiswanger

专题命中 视觉定位与Grounding ：multimodal large language model(title,abstract)；grounding(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning

Sheila Schoepp, Masoud Jafaripour, Yingyue Cao, Tianpei Yang, Fatemeh Abdollahi, Shadan Golestan, Zahin Sufiyan, Osmar R. Zaiane, Matthew E. Taylor

专题命中 视觉定位与Grounding ：VLM(title,abstract)；vision-language model(abstract)；grounding(abstract)；分类 cs.AI、cs.LG

Comments 9 pages, 4 figures

Exploring Spatial Language Grounding Through Referring Expressions

Akshar Tumu, Parisa Kordjamshidi

专题命中 视觉定位与Grounding ：grounding(title,abstract)；vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Vision Language Models are In-Context Value Learners

Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani, Peng Xu, Danny Driess, Ted Xiao, Jonathan Tompson, Osbert Bastani, Dinesh Jayaraman, Wenhao Yu, Tingnan Zhang, Dorsa Sadigh, Fei Xia

专题命中 视觉定位与Grounding ：vision language model(title)；vision-language model(abstract)；VLM(abstract)；grounding(abstract)

Comments Project website and demo: https://generative-value-learning.github.io/

Advancing Object Detection in Transportation with Multimodal Large Language Models (MLLMs): A Comprehensive Review and Empirical Testing

Huthaifa I. Ashqar, Ahmed Jaber, Taqwa I. Alhadidi, Mohammed Elhenawy

专题命中 视觉定位与Grounding ：multimodal large language model(title,abstract)；visual reasoning(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Visual Prompting in Multimodal Large Language Models: A Survey

Junda Wu, Zhehao Zhang, Yu Xia, Xintong Li, Zhaoyang Xia, Aaron Chang, Tong Yu, Sungchul Kim, Ryan A. Rossi, Ruiyi Zhang, Subrata Mitra, Dimitris N. Metaxas, Lina Yao, Jingbo Shang, Julian McAuley

专题命中 视觉定位与Grounding ：multimodal large language model(title,abstract)；grounding(abstract)；MLLM(abstract)；分类 cs.CV、cs.LG

Comments 10 pages

Genixer: Empowering Multimodal Large Language Models as a Powerful Data Generator

Henry Hengyuan Zhao, Pan Zhou, Mike Zheng Shou

专题命中 视觉定位与Grounding ：multimodal large language model(title,abstract)；grounding(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV 2024

Visual Grounding for Object-Level Generalization in Reinforcement Learning

Haobin Jiang, Zongqing Lu

专题命中 视觉定位与Grounding ：grounding(title,abstract)；vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 35 pages, 14 figures, 17 tables

Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models

Jiayun Luo, Siddhesh Khandelwal, Leonid Sigal, Boyang Li

专题命中 视觉定位与Grounding ：vision-language model(title,abstract)；VLM(abstract)；visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2024; Earlier version of this paper contained an unintentional error stemming from a bug in the code. This version corrects this error, which had to do with filtering of class names. In consultation with CVPR Program Chairs it was suggested errata be submitted as the updated (fixed) code reinforced original findings (albeit with slightly different final numbers)

Harnessing the Power of Large Vision Language Models for Synthetic Image Detection

Mamadou Keita, Wassim Hamidouche, Hassen Bougueffa, Abdenour Hadid, Abdelmalik Taleb-Ahmed

专题命中 视觉定位与Grounding ：vision language model(title)；vision-language model(abstract)；VLM(abstract)；visual language model(abstract)

Comments arXiv admin note: substantial text overlap with arXiv:2404.01959

SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection

Peng Qi, Zehong Yan, Wynne Hsu, Mong Li Lee

专题命中 视觉定位与Grounding ：multimodal large language model(title,abstract)；visual reasoning(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments To appear in CVPR 2024

Vision-Language Models as Zero-Annotation Oracles in Histopathology

视觉-语言模型作为组织病理学中的零标注预言机

Vishal Jain, Giorgio Buzzanca, Sarah Cechnicka, Maarten Naesens, Priyanka Koshy, Tri Nguyen, Jesper Kers, Candice Roufosse, Bernhard Kainz

机构 * Imperial College London（帝国理工学院） ； Leiden University Medical Center（莱顿大学医学中心） ； KU Leuven（鲁汶大学） ； University Hospitals Leuven（鲁汶大学医院） ； University Medical Center Utrecht（乌得勒支大学医学中心） ； Friedrich-Alexander University Erlangen-Nürnberg（埃尔朗根-纽伦堡大学） 纠错

专题命中 视觉定位与Grounding ：vision-language model(title,abstract)；VLM(abstract,abstract_cn)；分类 cs.CV

AI总结 提出一种粗到细方法，利用通用视觉-语言模型作为零标注预言机进行前景分割，在特殊染色上优于监督基线，并通过伪标签蒸馏轻量学生模型。

Comments 11 pages, 1 figure, 6 tables. Code available at https://github.com/VishalJ99/vlm-wsi-auto-context

HiERO-StepG @ Ego4D Step Grounding Challenge: hierarchical activity understanding enables zero-shot step grounding

HiERO-StepG @ Ego4D Step Grounding Challenge: 层次化活动理解实现零样本步骤定位

Andrea Zenotto, Simone Alberto Peirone, Francesca Pistilli, Giuseppe Averta

机构 * Politecnico di Torino（托里诺理工大学） 纠错

专题命中 视觉定位与Grounding ：grounding(title,title_cn)；分类 cs.CV

AI总结 提出HiERO-StepG方法，利用弱监督层次化表示学习和聚类，无需任务特定微调即可实现零样本步骤定位，在Ego4D挑战中达到56.27% R@1 (IoU=0.3)。

Comments Technical report for the Ego4D Goal Step - Step Grounding challenge at CVPR 2026, derived from arXiv:2505.12911

LVSum: A Benchmark for Timestamp-Aware Long Video Summarization

LVSum：一个用于时间感知长视频摘要的基准测试

Alkesh Patel, Melis Ozyildirim, Ying-Chang Cheng, Ganesh Nagarajan

机构 * Apple（苹果公司） 纠错

专题命中 视觉定位与Grounding ：MLLM(summary_cn,abstract_cn)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 本文提出LVSum基准测试，用于评估长视频摘要中时间对齐的性能，通过引入新的评估指标揭示现有MLLM在时间理解上的系统性差距。

Comments 25 pages, 5 tables, 3 figures

Where Am I? Semantic Map Grounding via Vision-Language Models for Multi-Modal Localization

我在哪里？基于视觉-语言模型的语义地图定位用于多模态定位

Suraj Borate, Aarav Shah, Madhu Vadali

机构 * IIT Gandhinagar（印度理工学院甘地讷格尔分校） 纠错

专题命中 视觉定位与Grounding ：vision-language model(title,abstract)；grounding(title)

AI总结 将机器人定位重构为语义推理任务，使用微调的Qwen2.5-VL-7B模型融合摄像头、LiDAR和语义地图预测位姿，在室内数据集上达到98.23%位置精度，并展现出跨模态互补性和对新场景的泛化能力。

Visual-Language-Guided Task Planning for Horticultural Robots

面向园艺机器人的视觉-语言引导任务规划

Jose Cuaran, Kendall Koe, Aditya Potnis, Naveen Kumar Uppalapati, Girish Chowdhary

机构 * the Siebel School of Computing and Data Science（塞比尔计算与数据科学学院） ； the Department of Agricultural and Biological Engineering（农业与生物工程系） ； National Center for Supercomputing Applications（国家超级计算中心） 纠错

专题命中 视觉定位与Grounding ：VLM(summary_cn,abstract)；vision language model(abstract)；grounding(abstract)

AI总结 提出一种模块化框架，利用视觉语言模型（VLM）主动查询异构数据源来引导机器人任务规划，在单作和混作环境中进行短期和长期作物监测任务基准测试，发现零样本VLM在短期任务中表现稳健（成功率87%），但长期多目标任务成功率低于10%，且依赖噪声语义地图时性能下降。

Comments 18 pages, 6 figures

MMCORE: MultiModal COnnection with Representation Aligned Latent Embeddings

MMCORE：多模态连接与表征对齐的潜在嵌入

Zijie Li, Yichun Shi, Jingxiang Sun, Ye Wang, Yixuan Huang, Zhiyao Guo, Xiaochen Lian, Peihao Zhu, Yu Tian, Zhonghua Zhai, Peng Wang

机构 * ByteDance Seed（字节跳动种子） 纠错

专题命中 视觉定位与Grounding ：VLM(abstract,abstract_cn)；grounding(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 MMCORE通过预训练视觉-语言模型生成语义视觉嵌入，结合扩散模型实现多模态图像生成与编辑，提升生成质量并降低计算开销。

By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Prompting

Hyungjun Yoon, Biniyam Aschalew Tolera, Taesik Gong, Kimin Lee, Sung-Ju Lee

专题命中 视觉定位与Grounding ：grounding(title)；multimodal large language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024) Main

Grounding Classical Task Planners via Vision-Language Models

Xiaohan Zhang, Yan Ding, Saeid Amiri, Hao Yang, Andy Kaminski, Chad Esselink, Shiqi Zhang

专题命中 视觉定位与Grounding ：vision-language model(title,abstract)；grounding(title)

Comments ICRA Workshop on Robot Execution Failures and Failure Management Strategies, 2023

VidForensics-M1: Meta-Detection Reinforcement Learning with Verifiable Temporal Grounding for AI-Generated Video Forensics

VidForensics-M1：用于AI生成视频取证的、具备可验证时间定位的元检测强化学习

Bowei Liu, Zheng Lu, Yuhan Bian, Xinchen Zhang, Xingming Shui, Yuesheng Huang, Xuhuan Li, Zihao Liu, Yifan Yang, Jun Zhou, Xiu Li

机构 * Tsinghua University（清华大学） ； Peking University（北京大学） ； Renmin University of China（中国人民大学） ； Microsoft（微软公司） 纠错

专题命中 视觉定位与Grounding ：grounding(title,abstract)；MLLM(abstract,abstract_cn)；分类 cs.CV

AI总结 该研究针对AI生成视频检测泛化性不足的问题，首次将元检测引入该领域，提出结合可验证时间证据的VidForensics-M1模型及相关机制，实现了鲁棒可泛化的检测。

Comments 27 pages, 15 figures

DAEP: Difficulty-Aware Evidence Planning for Medical Video Corpus Temporal Answer Grounding

DAEP：面向医学视频语料时序答案 grounding 的难度感知证据规划

Tianjian He, Yujie Liu, Zhiping Huang, Changbo Xu

机构 * TikTok, ByteDance（字节跳动TikTok） ； Beijing Institute of Graphic Communication（北京印刷学院） ； Lingnan University（岭南大学） 纠错

专题命中 视觉定位与Grounding ：grounding(title,title_cn)；分类 cs.CV

AI总结 BIGC团队提出DAEP方案，通过多模态证据排名与难度感知规划，在NLPCC 2026共享任务中以0.2728的平均得分获第一名，提升了医学视频时序答案定位的性能。

Comments 12 pages, 2 figures, 5 tables, accepted by NLPCC 2026 Shared Task Track 3

Zero-Shot Traffic Accident Detection via a Coarse-to-Fine VLM-Tracking Pipeline

基于粗到细VLM跟踪流水线的零样本交通事故检测

Dipit Saha, Shah Mohammad Abdul Mannan, Mohammad Raihan Rashid, Ruwad Naswan, Ahnaf Tahmid

机构 * Bangladesh University of Engineering and Technology（孟加拉工程技术大学） 纠错

专题命中 视觉定位与Grounding ：VLM(title,title_cn)；vision-language model(abstract)；分类 cs.CV

AI总结 本文提出一种无需训练的双通粗到细流水线，结合Qwen3-VL-32B-Instruct、YOLO11x与BoT-SORT，在零样本约束下于ACCIDENT @ CVPR基准测试中实现22%相对优势，达成0.504的三方调和均值得分。

Comments Accepted at the AUTOPILOT Workshop, CVPR 2026, Denver, CO

Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models

面向细粒度遗忘：多模态大语言模型的属性遗忘

Junkai Lin, Junkai Chen, Siqi Hou, Yuhao He, Ruiqi Liu, Chenhan Jin, Shengze Xu, Tieyong Zeng

专题命中 视觉定位与Grounding ：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.AI

AI总结 针对多模态大语言模型（MLLMs）属性遗忘的细粒度需求，提出轻量级训练无关框架CLRP，通过激活修补定位因果层并应用保留感知投影，实现目标属性遗忘同时保留同一身份信息，在多种MLLMs上验证了有效性。

MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence

MEDIC-AD：迈向医疗视觉-语言模型的临床智能

Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho, Pa Hong, Wookyoung Jeong, Yoojin Nam, Namjoon Kim, Ginny Y. Wong, Ka Chun Cheung, Jaeyoung Do

机构 * AIDAS Laboratory, Seoul National University（首尔大学AIDAS实验室） ； Samsung Changwon Hospital（三星昌原医院） ； Samsung Medical Center（三星医疗中心） ； NVIDIA, Santa Clara, USA（英伟达（美国圣克拉拉）） 纠错

专题命中 视觉定位与Grounding ：vision-language model(title,abstract)；VLM(abstract,abstract_cn)；分类 cs.CV

AI总结 MEDIC-AD通过分阶段框架提升医疗视觉-语言模型在病变检测、症状跟踪和视觉可解释性方面的性能，实现临床应用中的最优表现。

Journal ref CVPR 2026

Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigation

基于视觉语言模型的自动驾驶车辆导航GNSS欺骗检测技术发展

Mohammed Aldeen, Muhammad Sami Irfan, Sagar Dasgupta, Long Cheng, Mizanur Rahman, Mashrur Chowdhury

专题命中 视觉定位与Grounding ：vision-language model(title,abstract)；VLM(abstract,abstract_cn)；分类 cs.CV

AI总结 研究自动驾驶车辆GNSS欺骗检测问题，提出基于视觉语言模型的框架，融合多源数据，经三阶段微调及自适应推理策略，大幅提升检测准确率与效率，并生成数据集验证跨区域泛化能力，为自动驾驶提供实用道路防御层。