视觉大模型 / VLM

Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes

Just KIDDIN: 基于知识注入与蒸馏的有害表情包检测

Rahul Garg, Trilok Padhi, Hemang Jain, Ugur Kursuncu, Ponnurangam Kumaraguru

机构 * IIIT Hyderabad ； Georgia State University（佐治亚州立大学） 纠错

专题命中 视觉定位与Grounding ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 本文提出了一种结合知识蒸馏与注入的框架，用于提升有害表情包检测的性能，通过整合大规模知识图谱和视觉语言模型，实现更准确的毒性识别。

Audit After Segmentation: Reference-Free Mask Quality Assessment for Language-Referred Audio-Visual Segmentation

分段后审计：用于语言指代音频视频分段的无参考掩码质量评估

Jinxing Zhou, Yanghao Zhou, Yaoting Wang, Zongyan Han, Jiaqi Ma, Henghui Ding, Rao Muhammad Anwer, Hisham Cholakkal

机构 * MBZUAI ； National University of Singapore（国立新加坡大学） ； Fudan University（复旦大学） 纠错

专题命中 视觉定位与Grounding ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 本研究提出 MQA-RefAVS 任务，通过多模态大语言模型评估语言指代音频视频分段中掩码质量，提升分割准确性与可解释性。

GutenOCR: A Grounded Vision-Language Front-End for Documents

GutenOCR：文档的 grounded 视觉-语言前端

Hunter Heidenreich, Ben Elliott, Olivia Dinica, Yosheb Getachew

专题命中 视觉定位与Grounding ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 GutenOCR 通过微调 Qwen2.5-VL 模型，实现了文档中的 grounded OCR 和检测，提升了 OCR 得分和文本检测召回率，但存在页面线性化和公式布局方面的权衡。

ReSemAct: Advancing Fine-Grained Robotic Manipulation via Semantic Structuring and Affordance Refinement

ReSemAct：通过语义结构化和效用细化推进细粒度机器人操作

Chenyu Su, Weiwei Shang, Chen Qian, Fei Zhang, Shuang Cong

机构 * Department of Automation, University of Science and Technology of China（自动化系，中国科学技术大学） 纠错

专题命中 视觉定位与Grounding ：grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

AI总结 ReSemAct 通过语义结构化和效用细化方法，在细粒度机器人操作中实现更精确的效用目标生成与动态环境适应。

Comments Code and videos: https://github.com/scy-v/ReSemAct and https://resemact.github.io

AgriGPT-VL: Agricultural Vision-Language Understanding Suite

AgriGPT-VL：农业视觉-语言理解套件

Bo Yang, Yunkui Chen, Lanfei Feng, Yu Zhang, Xiao Xu, Jianyu Zhang, Nueraili Aierken, Runhe Huang, Hongjian Lin, Yibin Ying, Shijian Li

机构 * Zhejiang University（浙江大学） 纠错

专题命中 视觉定位与Grounding ：vision-language model(abstract)；grounding(abstract)；multimodal large language model(abstract)

AI总结 AgriGPT-VL通过构建农业专用的视觉-语言模型和评估套件，提升了农业领域的多模态理解和推理能力。

ReFineG: Synergizing Small Supervised Models and LLMs for Low-Resource Grounded Multimodal NER

Jielong Tang, Shuang Wang, Zhenxing Wang, Jianxing Yu, Jian Yin

机构 * School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院） ； Key Laboratory of Sustainable Tourism Smart Assessment Technology, Ministry of Culture and Tourism, Sun Yat-sen University（文化旅游可持续评估技术重点实验室，中华人民共和国文化和旅游部，中山大学） ； Beijing Normal University（北京师范大学） ； Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所） 纠错

专题命中 视觉定位与Grounding ：grounding(abstract)；multimodal large language model(abstract)；MLLM(abstract)

Comments CCKS 2025 Shared Task Paper

GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning

Sahiti Yerramilli, Nilay Pande, Rynaa Grover, Jayant Sravan Tamarapalli

机构 * Google（谷歌） ； Waymo 纠错

专题命中 视觉定位与Grounding ：grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Language-Guided Long Horizon Manipulation with LLM-based Planning and Visual Perception

Changshi Zhou, Haichuan Xu, Ningquan Gu, Zhipeng Wang, Bin Cheng, Pengpeng Zhang, Yanchao Dong, Mitsuhiro Hayashibe, Yanmin Zhou, Bin He

专题命中 视觉定位与Grounding ：vision-language model(abstract)；VLM(abstract)；grounding(abstract)

Temporal Preference Optimization for Long-Form Video Understanding

Rui Li, Xiaohan Wang, Yuhui Zhang, Orr Zohar, Zeyu Wang, Serena Yeung-Levy

机构 * Stanford University（斯坦福大学） 纠错

专题命中 视觉定位与Grounding ：LLaVA(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

DIS-CO: Discovering Copyrighted Content in VLMs Training Data

André V. Duarte, Xuandong Zhao, Arlindo L. Oliveira, Lei Li

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 视觉定位与Grounding ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

CleanMAP: Distilling Multimodal LLMs for Confidence-Driven Crowdsourced HD Map Updates

Ankit Kumar Shaw, Kun Jiang, Tuopu Wen, Chandan Kumar Sah, Yining Shi, Mengmeng Yang, Diange Yang, Xiaoli Lian

专题命中 视觉定位与Grounding ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Kun Jiang, Mengmeng Yang and Diange Yang are Corresponding Author. The main paper and supplementary material are both included here, total 23 pages (main paper is 10 pages and supplementary material is 13 pages), total 17 figures (6 figures in main paper and 11 figures in supplementary material), this paper is Accepted to CVPR WDFM-AD Workshop 2025, The code will be available at https://Ankit-Zefan.github.io/CleanMap/

Bayesian Modeling of Zero-Shot Classifications for Urban Flood Detection

Matt Franchi, Nikhil Garg, Wendy Ju, Emma Pierson

专题命中 视觉定位与Grounding ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments In review

Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection

Enshen Zhou, Qi Su, Cheng Chi, Zhizheng Zhang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, He Wang

专题命中 视觉定位与Grounding ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by CVPR 2025. Project page: https://zhoues.github.io/Code-as-Monitor/

Quo Vadis, Anomaly Detection? LLMs and VLMs in the Spotlight

Xi Ding, Lei Wang

专题命中 视觉定位与Grounding ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Research report

Shaking Up VLMs: Comparing Transformers and Structured State Space Models for Vision & Language Modeling

Georgios Pantazopoulos, Malvina Nikandrou, Alessandro Suglia, Oliver Lemon, Arash Eshghi

专题命中 视觉定位与Grounding ：visual language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Leveraging Semantic and Geometric Information for Zero-Shot Robot-to-Human Handover

Jiangshan Liu, Wenlong Dong, Jiankun Wang, Max Q. -H. Meng

专题命中 视觉定位与Grounding ：vision-language model(abstract)；VLM(abstract)；grounding(abstract)

Comments 6 pages, 5 figures, conference

VALE: A Multimodal Visual and Language Explanation Framework for Image Classifiers using eXplainable AI and Language Models

Purushothaman Natarajan, Athira Nambiar

专题命中 视觉定位与Grounding ：vision-language model(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 15 pages, 10 tables, 3 figures

AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents

Michael Ahn, Debidatta Dwibedi, Chelsea Finn, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Karol Hausman, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Sean Kirmani, Isabel Leal, Edward Lee, Sergey Levine, Yao Lu, Isabel Leal, Sharath Maddineni, Kanishka Rao, Dorsa Sadigh, Pannag Sanketi, Pierre Sermanet, Quan Vuong, Stefan Welker, Fei Xia, Ted Xiao, Peng Xu, Steve Xu, Zhuo Xu

专题命中 视觉定位与Grounding ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 26 pages, 9 figures, ICRA 2024 VLMNM Workshop

Recognizing Everything from All Modalities at Once: Grounded Multimodal Universal Information Extraction

Meishan Zhang, Hao Fei, Bin Wang, Shengqiong Wu, Yixin Cao, Fei Li, Min Zhang

专题命中 视觉定位与Grounding ：grounding(abstract)；multimodal large language model(abstract)；MLLM(abstract)

Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning

Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, Yang Gao

专题命中 视觉定位与Grounding ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments arXiv v2: add appendix

SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing

Zhecheng Wang, Rajanie Prabha, Tianyuan Huang, Jiajun Wu, Ram Rajagopal

专题命中 视觉定位与Grounding ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by AAAI 2024

3D-LLM: Injecting the 3D World into Large Language Models

Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, Chuang Gan

专题命中 视觉定位与Grounding ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project Page: : https://vis-www.cs.umass.edu/3dllm/

Transformers in Vision: A Survey

Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, Mubarak Shah

专题命中 视觉定位与Grounding ：visual reasoning(abstract)；grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 30 pages (Accepted in ACM Computing Surveys December 2021)

Do VLMs Need Vision Transformers? Evaluating State Space Models as Vision Encoders

视觉语言模型需要视觉变换器吗？评估状态空间模型作为视觉编码器

Shang-Jui Ray Kuo, Paola Cascante-Bonilla

机构 * Stony Brook University（石英 Brook 大学） 纠错

专题命中 视觉定位与Grounding ：VLM(abstract,comments)；grounding(abstract)；分类 cs.CV、cs.LG

AI总结 本文评估了状态空间模型作为视觉编码器在视觉语言模型中的有效性，发现其在VQA和定位任务中表现优异，并提出稳定策略提升鲁棒性。

Comments Project page: https://lab-spell.github.io/vlm-ssm-vision-encoders/ ; Code: https://github.com/raykuo18/vlm-ssm-vision-encoders

Seed2GS: Camera-Free, Training-Free Object Extraction from 3D Gaussian Scenes via a Single Reference-View Grounding

Seed2GS：通过单个参考视图定位从3D高斯溅射（3DGS）场景中进行无相机、无训练的目标提取

Zongjian Ding, Yudong Gao, Jiale Liu, Xinglin Yu, Junxing Ren, Dong Wei, Yajing Chen, Shan Huang, Mingjun Cheng, Min Li

专题命中 视觉定位与Grounding ：grounding(title)；分类 cs.CV

AI总结 Seed2GS是一种无相机、无训练的目标提取方法，通过分离目标身份与3D覆盖范围，在LERF-MASK和3D-OVS数据集上实现了高精度，且计算延迟低。

Personalized Deep Research Query Refinement with Graph-Scaffolded Evidence Grounding

基于图支架证据锚定的个性化深度研究查询优化

Soojin Yoon, Dongha Lee

专题命中 视觉定位与Grounding ：grounding(title)；分类 cs.AI

AI总结 本研究提出G-STEER方法，通过图支架证据锚定优化用户请求为个性化研究规范，在提升深度研究智能体报告个性化的同时，大幅减少向用户提问的数量。

Comments 13 pages, 4 figures

Teaching Nemotron Greek: Mining a Corpus, Adapting Retrieval, and Grounding Generation for Modern Greek across Specialist Domains

让Nemotron学习希腊语：针对专业领域现代希腊语的语料库挖掘、检索适配与生成落地

Ayoub Kirouane, Christos Petrocheilos

专题命中 视觉定位与Grounding ：grounding(title)；分类 cs.AI

AI总结 本研究针对现代希腊语缺失于Nemotron检索模型及主流多语言基准的问题，提出Nemotron检索栈的端到端适配方案，含语料库挖掘等步骤，推出首个希腊语RAG基准HERA，适配后的模型在多项指标上显著提升。

Comments 15 pages, 10 figures, 7 tables. Includes release of the HERA benchmark and Sophea Nemo RAG models

Audio-Visual World Models: Learning Physically Grounded Multisensory Dynamics

视听世界模型：为具身智能体奠定多感官想象的基础

Jiahua Wang, Leqi Zheng, Jialong Wu, Yaoxin Mao, Shijie Cheng

机构 * Tsinghua University（清华大学） ； Beijing Institute of Technology（北京理工大学） 纠错

专题命中 视觉定位与Grounding ：grounding(title)；分类 cs.CV

AI总结 提出视听世界模型（AVWM）统一框架，通过条件扩散Transformer（AV-CDiT）联合预测双耳音频与视觉动态，在30小时基准AVW-4k上实现高保真多模态预测，并验证其在具身导航中的有效性。

CeQe: Grounding Lexical Retrieval in Semantic Evidence

CeQe：在语义证据中实现词汇检索

Adam Kahirov, Umesh Deshpande, Swaminathan Sundararaman

专题命中 视觉定位与Grounding ：grounding(title)；分类 cs.AI

AI总结 针对BM25的语义词汇鸿沟问题，本文提出CE-QE方法，利用交叉编码器的语义检索结果扩展BM25查询，在多个BEIR数据集上显著提升检索性能且不修改底层索引。

LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents

LabGuard：将自然语言实验室规则转化为具身实验室代理的运行时守卫

Jingpu Yang, Fengxian Ji, Zhengzhao Lai, Zhexuan Cui, Guangxian Ouyang, Qian Jiang, Fan Zhang, Min Peng, Qianqian Xie, Preslav Nakov, Zhuohan Xie

机构 * Wuhan University（武汉大学） ； MBZUAI（穆罕默德·本·扎耶德人工智能大学） ； Northeastern University（东北大学） 纠错

专题命中 视觉定位与Grounding ：grounding(title)；分类 cs.AI

AI总结 提出LabGuard，一种将自然语言实验室规则转化为可执行规范并部署为运行时守卫的安全套件，通过三个核心组件实现规则到监控的映射，实验表明能有效降低不安全事件。

Comments First three authors are co-first authors