视觉大模型 / VLM

Open Set Domain Adaptation with Vision-language models via Gradient-aware Separation

Haoyang Chen

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

Ultrasound Report Generation with Multimodal Large Language Models for Standardized Texts

Peixuan Ge, Tongkun Su, Faqin Lv, Baoliang Zhao, Peng Zhang, Chi Hong Wong, Liang Yao, Yu Sun, Zenan Wang, Pak Kin Wong, Ying Hu

机构 * Shenzhen Institutes of Advanced Technology（深圳先进技术研究院） ； University of Macau（澳门大学） ； Chinese PLA General Hospital（中国人民解放军总医院） ； Macau University of Science and Technology（澳门科学技术大学） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(title)；分类 cs.CV、cs.AI

LongProLIP: A Probabilistic Vision-Language Model with Long Context Text

Sanghyuk Chun, Sangdoo Yun

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

Comments Accepted as a tiny paper at the 1st workshop of "Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI" at ICLR 2025; code: https://github.com/naver-ai/prolip; models: https://huggingface.co/collections/SanghyukChun/prolip-6712595dfc87fd8597350291

FCoT-VL:Advancing Text-oriented Large Vision-Language Models with Efficient Visual Token Compression

Jianjian Li, Junquan Fan, Feng Tang, Gang Huang, Shitao Zhu, Songlin Liu, Nian Xie, Wulong Liu, Yong Liao

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.AI

Comments 20 pages, 18 figures, 6 tables

A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards

Shivansh Patel, Xinchen Yin, Wenlong Huang, Shubham Garg, Hooshang Nayyeri, Li Fei-Fei, Svetlana Lazebnik, Yunzhu Li

专题命中 VLM训练与架构 ：VLM(title)；分类 cs.CV、cs.AI

Comments ICRA 2025, Project Page: https://iker-robot.github.io/

Towards Compatible Fine-tuning for Vision-Language Model Updates

Zhengbo Wang, Jian Liang, Lijun Sheng, Ran He, Zilei Wang, Tieniu Tan

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

Comments preprint

Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis

Davide Bucciarelli, Nicholas Moratelli, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中 VLM训练与架构 ：multimodal large language model(title)；分类 cs.CV、cs.AI

Comments ECCV 2024 Workshop on Green Foundation Models

Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Design

Markus J. Buehler

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

SUGARCREPE++ Dataset: Vision-Language Model Sensitivity to Semantic and Lexical Alterations

Sri Harsha Dumpala, Aman Jaiswal, Chandramouli Sastry, Evangelos Milios, Sageev Oore, Hassan Sajjad

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

Comments Added the dataset link to the abstract

FLoRA: Enhancing Vision-Language Models with Parameter-Efficient Federated Learning

Duy Phuong Nguyen, J. Pablo Munoz, Ali Jannesari

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.AI、cs.LG

Comments 10 pages, 11 figures

MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric

Haokun Lin, Haoli Bai, Zhili Liu, Lu Hou, Muyi Sun, Linqi Song, Ying Wei, Zhenan Sun

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.AI

Comments 18 pages, 8 figures, Published in CVPR2024

Journal ref In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

Text encoders bottleneck compositionality in contrastive vision-language models

Amita Kamath, Jack Hessel, Kai-Wei Chang

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

Comments EMNLP 2023

Test-Time Distribution Normalization for Contrastively Learned Vision-language Models

Yifei Zhou, Juntao Ren, Fengyu Li, Ramin Zabih, Ser-Nam Lim

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

Comments Accepted to NeurIPS 2023, project webpage: https://fengyuli-dev.github.io/dn-website/

Black Box Few-Shot Adaptation for Vision-Language models

Yassine Ouali, Adrian Bulat, Brais Martinez, Georgios Tzimiropoulos

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

Comments Published at ICCV 2023

VL-CheckList: Evaluating Pre-trained Vision-Language Models with Objects, Attributes and Relations

Tiancheng Zhao, Tianqi Zhang, Mingwei Zhu, Haozhan Shen, Kyusong Lee, Xiaopeng Lu, Jianwei Yin

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.LG

Comments 9 pages, preprint

PyramidCLIP: Hierarchical Feature Alignment for Vision-language Model Pretraining

Yuting Gao, Jinfeng Liu, Zihan Xu, Jun Zhang, Ke Li, Rongrong Ji, Chunhua Shen

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.AI

Grounding Value Alignment with Ethical Principles

Tae Wan Kim, Thomas Donaldson, John Hooker

专题命中 VLM训练与架构 ：grounding(title)；分类 cs.AI、cs.LG

Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models

对齐即全部所需：通用音频-语言模型的无指令训练

Xuanru Zhou, Yiwen Shao, Jiahong Li, Dong Yu

机构 * Zhejiang University（浙江大学） ； Tencent Hunyuan（腾讯混元） 纠错

专题命中 VLM训练与架构 ：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)

AI总结 该研究提出仅对齐的无指令大音频-语言模型LALM，仅训练轻量投影器，在多模态数据集上用更少数据达到或优于基线，证明仅靠对齐即可构建有竞争力的多模态大语言模型。

CustomDance: Customized 3D Dance Generation with Coarse-to-Fine Human-Centered Interactive Control

CustomDance：基于以人为中心的粗到细交互式控制的定制化3D舞蹈生成

Xulong Tang, Kaixing Yang, Xiaohu Guo, Prabhakaran Balakrishnan, Rawan Alghofaili

专题命中 VLM训练与架构 ：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)

AI总结 CustomDance是基于粗到细交互式控制的定制化3D舞蹈生成系统，通过三阶段流程实现AI辅助编舞，在定量和定性比较中优于基线，为用户提供全面控制。

Comments Accepted to SIGGRAPH Asia 2026

Conjunctive Poisoning in AI Supply-Chain Applications

AI供应链应用中的联合中毒攻击

Nokimul Hasan Arif, Qian Lou, Mengxin Zheng

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)

AI总结 本研究提出AI供应链中的联合中毒攻击，通过包装器与元数据的交互改变模型行为，提出TIF-BAH防御，揭示AI部署中存在未被现有防御覆盖的部署时行为风险。

Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot

Genie Sim 3.0：人形机器人综合仿真平台

Chenghao Yin, Da Huang, Di Yang, Jichao Wang, Nanshu Zhao, Chen Xu, Wenjun Sun, Linjie Hou, Zhijun Li, Junhui Wu, Zhaobo Liu, Zhen Xiao, Sheng Zhang, Lei Bao, Rui Feng, Zhenquan Pang, Jiayu Li, Qian Wang, Maoqing Yao

机构 * AgibotTech（Agibot科技） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)

AI总结 Genie Sim 3.0通过高保真场景生成和开放数据集，提升机器人学习模型的泛化能力与仿真到现实的迁移效果。

MBA: Multimodal Benchmark and Agents for Real-World Business Ideation

MBA：面向现实世界商业创意的多模态基准与智能体

Hojun Choi, Jaeyo Shin, Suin Lee, Hyunjung Shim

专题命中 VLM训练与架构 ：MLLM(abstract,abstract_cn)；分类 cs.CV、cs.AI、cs.LG

AI总结 该研究推出首个多模态商业创意基准MBA-Bench，提出MBA-b和MBA-k两种智能体，经实验其性能显著优于相关基准，为多模态商业创意智能体研究提供了重要支撑。

Comments Project page: https://hchoi256.github.io/projects/mba/ Code: https://github.com/hchoi256/MBA

Inverting the Hidden: Unveiling Multimodal Privacy Leakage in Collaborative LVLM Inference

反转隐藏内容：揭示协作式大型视觉语言模型（LVLM）推理中的多模态隐私泄露

Shuaifan Jin, Zhibo Wang, Qiyuan Wang, Yiting Han, Yajie Zhou, Yuanfan Zhang, Jiahui Hu, Xiaoyi Pang

专题命中 VLM训练与架构 ：LLaVA(abstract,abstract_cn)；vision-language model(abstract)

AI总结 该研究针对协作式LVLM推理的隐私风险，提出RASR多模态重建攻击，可从深层LVLM隐藏状态恢复隐私信息，图像重建MSE降约50%、文本恢复token准确率达99%。

StageCraft: Execution Aware Mitigation of Distractor and Obstruction Failures in VLA Models

StageCraft: 通过执行意识缓解VLA模型中干扰和障碍故障

Kartikay Milind Pangaonkar, Prabin Rath, Omkar Patil, Nakul Gopalan

机构 * Arizona State University（亚利桑那州立大学） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)

AI总结 StageCraft通过利用大规模视觉语言模型进行推理，改进预训练VLA策略性能，通过操控环境初始状态避免执行故障，实现在三个真实任务领域中性能提升40%。

Comments Accepted to IEEE International Conference on Intelligent Robots and Systems (IROS) 2026

Keypose Exploration: Efficient Automatic Trajectory Labelling and Cross-Embodiment Policy Transfer

关键姿态探索：高效的自动轨迹标注与跨实体策略迁移

Yupu Lu, Hang Xu, Yizhou Chen, Jia Pan

机构 * School of Computing and Data Science, The University of Hong Kong, HKSAR（计算与数据科学学院，香港大学，香港特别行政区） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)

AI总结 提出结合视觉语言模型与经典轨迹分析的关键姿态自动标注流程，并利用关键姿态引导扩散策略实现零样本跨实体迁移。

Comments Accepted by IROS2026. Code available at: https://github.com/YupuLu/keypose_labelling

PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation

PIXELRAG：网页截图在检索增强生成中优于文本

Yichuan Wang, Zhifei Li, Zirui Wang, Paul Teiletche, Lesheng Jin, Matei Zaharia, Joseph E. Gonzalez, Sewon Min

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；分类 cs.CV、cs.AI、cs.LG

AI总结 提出PixelRAG方法，以网页截图替代文本进行检索和阅读，利用视觉嵌入模型和对比学习，在30M截图库上实现端到端RAG，在多项任务中优于文本基线，准确率提升最高18.1%，并通过图像压缩降低3倍token成本。

Comments Our code is available at https://github.com/StarTrail-org/PixelRAG

UniFS: Unified Fast-to-Slow Hierarchical Architecture for Vision-Language-Action Models

UniFS：面向视觉-语言-动作模型的统一快慢层次架构

Lin Sun, Zhiwei Guan, Conglin Wang, Zihong Chen, Jianhai Yu, Zongsheng Li, Boyong He, Tao Sun, Jiale Cao, Lige Liu

机构 * Tianjin University（天津大学） ； Yiwu Research Institute of Fudan University（复旦大学义乌研究院） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)

AI总结 提出UniFS统一快慢层次架构，通过分层更新频率、潜向量反转和多级监督策略，解决快慢双系统视觉-语言-动作模型中的频率困境和信息耦合问题，在LIBERO上实现98.3%成功率并提速2.1倍。

Comments Code is opensourced at https://github.com/linsun449/UniFS

BIT-Nav: Brain-Inspired Trajectory Memory for Embodied Navigation

BIT-Nav: 具身导航的脑启发轨迹记忆

Rithvik Jonna, Aakash Gurram, Man Namgung, Wyatt Mackey, Tinoosh Mohsenin

机构 * Johns Hopkins University（约翰霍普金斯大学） ； DEVCOM Army Research Laboratory（DEVCOM陆军研究实验室） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)

AI总结 提出BIT-Nav框架，通过轻量级学习轨迹记忆增强冻结的视觉-语言模型导航，解决长序列中帧采样稀疏问题，以恒定token成本编码结构化运动历史。

Comments Accepted to CVPR 2026: 2nd Workshop on 3D-LLM/VLA: Bridging Language, Vision and Action in 3D Environments

Steering Autoregressive Vision-Language-Action Policies via Action Token Intervention

通过动作令牌干预引导自回归视觉-语言-动作策略

Jason Chan, Jonathan C. Kao

机构 * University of California, Los Angeles（加州大学洛杉矶分校） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)

AI总结 提出Token Steering方法，在推理时通过干预动作令牌空间动态引导VLA模型轨迹生成，无需训练或微调，显著提升家务操作任务成功率。

Comments 9 pages, 5 figures

APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies

APT: 动作专家预训练提升视觉-语言-动作策略的指令泛化能力

Kechun Xu, Zhenjie Zhu, Anzhe Chen, Rong Xiong, Yue Wang

机构 * Zhejiang University（浙江大学） ； Zhejiang Humanoid Robot Innovation Center（浙江人形机器人创新中心） 纠错

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)

AI总结 针对连续动作专家模型对分布外语言指令泛化差的问题，提出APT两阶段训练方法，先预训练动作专家作为视觉-动作先验，再通过门控融合注入语言，显著提升泛化性能。