视觉大模型 / VLM

VLM-driven Skill Selection for Robotic Assembly Tasks

Jeong-Jung Kim, Doo-Yeol Koh, Chang-Hyun Kim

机构 * Department of AI Machinery, Korea Institute of Machinery & Materials（人工智能机械系，韩国机械材料研究院） 纠错

专题命中 VLM训练与架构 ：VLM(title)；vision-language model(abstract)

PSD2Code: Automated Front-End Code Generation from Design Files via Multimodal Large Language Models

Yongxi Chen, Lei Chen

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)

An Evaluation of Interleaved Instruction Tuning on Semantic Reasoning Performance in an Audio MLLM

Jiawei Liu, Enis Berk Çoban, Zarina Schevchenko, Hao Tang, Zhigang Zhu, Michael I Mandel, Johanna Devaney

机构 * The Graduate Center, CUNY（CUNY研究生中心） ； Brooklyn College, CUNY（CUNY布鲁克林学院） ； Borough of Manhattan Community College, CUNY（CUNY曼哈顿社区学院） ； The City College of New York, CUNY（CUNY纽约城市学院） 纠错

专题命中 VLM训练与架构 ：MLLM(title,abstract)

Vision-language models learn the geometry of human perceptual space

Craig Sanders, Billy Dickson, Sahaj Singh Maini, Robert Nosofsky, Zoran Tiganj

专题命中 VLM训练与架构 ：vision-language model(title,abstract)

DOTA: Distributional Test-Time Adaptation of Vision-Language Models

Zongbo Han, Jialong Yang, Guangyu Wang, Junfan Li, Qianli Xu, Mike Zheng Shou, Changqing Zhang

机构 * State Key Laboratory of Networking and Switching Technology（网络与交换技术国家重点实验室） ； Beijing University of Posts and Telecommunications（北京邮电大学） ； College of Intelligence and Computing（智能与计算学院） ； Tianjin University（天津大学） ； School of Computer Science and Technology（计算机科学与技术学院） ； Harbin Institute of Technology Shenzhen（哈尔滨工业大学深圳学院） ； Institute for Infocomm Research, A*STAR（信息通信研究机构，A*STAR） ； Show Lab, National University of Singapore（新加坡国立大学Show实验室） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by NeurIPS 2025

Chem3DLLM: 3D Multimodal Large Language Models for Chemistry

Lei Jiang, Shuzhou Sun, Biqing Qi, Yuchen Fu, Xiaohua Xu, Yuqiang Li, Dongzhan Zhou, Tianfan Fu

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)

Comments 15 pages, 9 figures

Can Sound Replace Vision in LLaVA With Token Substitution?

Ali Vosoughi, Jing Bi, Pinxin Liu, Yunlong Tang, Chenliang Xu

专题命中 VLM训练与架构 ：LLaVA(title)；vision-language model(abstract)

Comments Project page: https://ali-vosoughi.github.io/SoundCLIP/

LLaVA-NeuMT: Selective Layer-Neuron Modulation for Efficient Multilingual Multimodal Translation

Jingxuan Wei, Caijun Jia, Qi Chen, Yujun Cai, Linzhuang Sun, Xiangxiang Zhang, Gaowei Wu, Bihui Yu

机构 * University of Chinese Academy of Sciences（中国科学院大学） ； The University of Queensland（昆士兰大学） 纠错

专题命中 VLM训练与架构 ：LLaVA(title,abstract)

Human-like object concept representations emerge naturally in multimodal large language models

Changde Du, Kaicheng Fu, Bincheng Wen, Yi Sun, Jie Peng, Wei Wei, Ying Gao, Shengpei Wang, Chuncheng Zhang, Jinpeng Li, Shuang Qiu, Le Chang, Huiguang He

机构 * State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology（脑认知与脑启发智能技术重点实验室） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Institute of Neuroscience, State Key Laboratory of Brain Cognition and Brain-Inspired Intelligence Technology（神经科学研究所） ； CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心） ； Chinese Academy of Sciences（中国科学院） 纠错

专题命中 VLM训练与架构 ：multimodal large language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments Published on Nature Machine Intelligence

Journal ref Nature Machine Intelligence, 2025

CacheFL: Privacy-Preserving and Efficient Federated Cache Model Fine-Tuning for Vision-Language Models

Mengjun Yi, Hanwen Zhang, Hui Dou, Jian Zhao, Furao Shen

专题命中 VLM训练与架构 ：vision-language model(title,abstract)

PlanGPT-VL: Enhancing Urban Planning with Domain-Specific Vision-Language Models

He Zhu, Junyou Su, Minxin Chen, Wen Wang, Yijie Deng, Guanhua Chen, Wenjia Zhang

机构 * Behavioral and Spatial AI Lab, Peking University & Tongji University（行为与空间人工智能实验室，北京大学 & 同济大学） ； Southern University of Science and Technology（南方科技大学） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)

Have Multimodal Large Language Models (MLLMs) Really Learned to Tell the Time on Analog Clocks?

Tairan Fu, Miguel González, Javier Conde, Elena Merino-Gómez, Pedro Reviriego

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)

Comments 6 pages, 5 figures, 2 tables

OmniV-Med: Scaling Medical Vision-Language Model for Universal Visual Understanding

Songtao Jiang, Yuan Wang, Sibo Song, Yan Zhang, Zijie Meng, Bohan Lei, Jian Wu, Jimeng Sun, Zuozhu Liu

机构 * Zhejiang University（浙江大学） ； Alibaba Group（阿里巴巴集团） ； UIUC（伊利诺伊大学香槟分校） 纠错

专题命中 VLM训练与架构 ：vision-language model(title,abstract)

BeMERC: Behavior-Aware MLLM-based Framework for Multimodal Emotion Recognition in Conversation

Yumeng Fu, Junjie Wu, Zhongjie Wang, Meishan Zhang, Yulin Wu, Bingquan Liu

专题命中 VLM训练与架构 ：MLLM(title,abstract)

ArchSeek: Retrieving Architectural Case Studies Using Vision-Language Models

Danrui Li, Yichao Shi, Yaluo Wang, Ziying Shi, Mubbasir Kapadia

专题命中 VLM训练与架构 ：vision-language model(title,abstract)

Comments 15 pages, 8 figures, 3 tables. Accepted by CAAD Futures 2025

M4SC: An MLLM-based Multi-modal, Multi-task and Multi-user Semantic Communication System

Feibo Jiang, Siwei Tu, Li Dong, Kezhi Wang, Kun Yang, Cunhua Pan

专题命中 VLM训练与架构 ：MLLM(title,abstract)

LLaVAC: Fine-tuning LLaVA as a Multimodal Sentiment Classifier

T. Chay-intr, Y. Chen, K. Viriyayudhakorn, T. Theeramunkong

专题命中 VLM训练与架构 ：LLaVA(title,abstract)

Fine-tuning Multimodal Large Language Models for Product Bundling

Xiaohao Liu, Jie Wu, Zhulin Tao, Yunshan Ma, Yinwei Wei, Tat-seng Chua

专题命中 VLM训练与架构 ：multimodal large language model(title)；MLLM(abstract)

Comments Accepted by KDD 2025 (CR)

ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models

Yassir Bendou, Amine Ouasfi, Vincent Gripon, Adnane Boukhayma

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments Code available at https://ybendou.github.io/ProKeR

ReStory: VLM-augmentation of Social Human-Robot Interaction Datasets

Fanjun Bu, Wendy Ju

专题命中 VLM训练与架构 ：VLM(title)；vision language model(abstract)

Comments 16th International Conference on Social Robotics +AI

Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval

Zelong Sun, Dong Jing, Guoxing Yang, Nanyi Fei, Zhiwu Lu

专题命中 VLM训练与架构 ：vision-language model(title,abstract)

Comments Accepted by AAAI 2025

Context-Informed Machine Translation of Manga using Multimodal Large Language Models

Philip Lippmann, Konrad Skublicki, Joshua Tanner, Shonosuke Ishiwatari, Jie Yang

专题命中 VLM训练与架构 ：multimodal large language model(title,abstract)

Comments COLING 2025

TrafficVLM: A Controllable Visual Language Model for Traffic Video Captioning

Quang Minh Dinh, Minh Khoi Ho, Anh Quan Dang, Hung Phong Tran

专题命中 VLM训练与架构 ：visual language model(title)；分类 cs.CV、cs.AI、cs.LG

Journal ref Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 7134-7143

Amharic LLaMA and LLaVA: Multimodal LLMs for Low Resource Languages

Michael Andersland

专题命中 VLM训练与架构 ：LLaVA(title,abstract)

SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models

Zekun Wang, Jingchang Chen, Wangchunshu Zhou, Haichao Zhu, Jiafeng Liang, Liping Shan, Ming Liu, Dongliang Xu, Qing Yang, Bing Qin

专题命中 VLM训练与架构 ：vision-language model(title,abstract)

Comments COLING-LREC 2024

Robust Fine-Tuning of Vision-Language Models for Domain Generalization

Kevin Vogt-Lowell, Noah Lee, Theodoros Tsiligkaridis, Marc Vaillant

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments In proceedings of the 27th IEEE High Performance Extreme Computing Conference

When and why vision-language models behave like bags-of-words, and what to do about it?

Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, James Zou

专题命中 VLM训练与架构 ：vision-language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2023 Oral (notable-top-5%)

Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning

Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, Cordelia Schmid

专题命中 VLM训练与架构 ：visual language model(title)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2023 Camera-Ready; Project Webpage: https://antoyang.github.io/vid2seq.html ; 18 pages; 6 figures

TransAnyText: Translating Arbitrary Text in E-commerce Images via Structured Visual Generation

TransAnyText：通过结构化视觉生成实现电商图像中任意文本的翻译

Xiaoan Liu, Lichen Ma, Zipeng Guo, Yu He, Xiaoyan Su, Shaojie Guo, Hao Yang, Jingling Fu, Xiaolong Fu, Zhen Chen, Yu Guo, Fei Wang, Xinyi Liu, Yongjun Zhang, Ke Zhang, Junshi Huang

专题命中 VLM训练与架构 ：VLM(abstract,abstract_cn)；vision-language model(abstract)；分类 cs.CV

AI总结 本文提出TransAnyText框架，将电商图像文本翻译转化为生成可渲染HTML补丁，经三阶段后训练优化，在多基准上表现优于对比方法，为跨境电商图像翻译提供有效可控的解决方案。

FIRM: Fine-Grained Intra-Token Representation of Masks for Remote Sensing Reasoning Segmentation

FIRM：面向遥感推理分割的掩码细粒度令牌内表示

Weidong Tang, Kaiyu Li, Yikai Wang, Yanan Wu, Haotian Gan, Shihong Wang, Xiangyong Cao

专题命中 VLM训练与架构 ：MLLM(abstract,abstract_cn)；multimodal large language model(abstract)；分类 cs.CV

AI总结 本文提出FIRM模型，通过预测视觉令牌内的r×r二值子单元掩码代码结合轻量连续渲染器优化边界，在5个遥感推理分割基准上取得领先结果，提升了细粒度分割性能。