多模态大模型

GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis

GAIA: 一个全球性的多模态、多尺度遥感图像分析数据集

Angelos Zavras, Dimitrios Michail, Xiao Xiang Zhu, Begüm Demir, Ioannis Papoutsis

机构 * Orion Lab, School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens（奥里昂实验室，农村测绘与地理信息工程学院，希腊雅典国家技术大学） ； Institute of Astronomy, Astrophysics, Space Applications and Remote Sensing, National Observatory of Athens（天文、天体物理、空间应用与遥感研究所，希腊雅典国家天文台） ； Department of Informatics and Telematics, Harokopio University of Athens（信息与电信技术系，雅典霍罗科皮欧大学） ； Chair of Data Science in Earth Observation, Technical University of Munich（地球观测数据科学教授职位，慕尼黑技术大学） ； Munich Center for Machine Learning（慕尼黑机器学习中心） ； Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学系，柏林技术大学） ； BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所） 纠错

专题命中 图文多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

AI总结 GAIA是一个全球性的多模态遥感图像分析数据集，通过精心构建的图像-文本对提升遥感任务的性能。

Comments 26 pages, 14 figures

How to make Medical AI Systems safer? Simulating Vulnerabilities, and Threats in Multimodal Medical RAG System

如何使医疗AI系统更安全？在多模态医疗RAG系统中模拟漏洞和威胁

Kaiwen Zuo, Zelin Liu, Raman Dutt, Ziyang Wang, Zhongtian Sun, Fan Mo, Pietro Liò

专题命中 图文多模态 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.AI

AI总结 本文提出MedThreatRAG框架，通过模拟攻击环境揭示医疗RAG系统漏洞，展示跨模态冲突注入对系统性能的严重影响。

Comments Sumbitted to 2026 ICASSP

CMMCoT: Enhancing Complex Multi-Image Comprehension via Multi-Modal Chain-of-Thought and Memory Augmentation

CMMCoT：通过多模态链式思考和记忆增强提升复杂多图像理解

Guanghao Zhang, Tao Zhong, Yan Xia, Mushui Liu, Zhelun Yu, Haoyuan Li, Wanggui He, Fangxun Shu, Dong She, Yi Wang, Hao Jiang

专题命中 图文多模态 ：multi-modal(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

AI总结 CMMCoT通过多模态链式思考和记忆增强提升多图像理解的复杂性与准确性

Comments Accepted by AAAI 2026

A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks

多模态大语言模型在视觉-语言任务中的综合综述与指南

Chia Xin Liang, Pu Tian, Caitlyn Heqi Yin, Yao Yua, Wei An-Hou, Li Ming, Xinyuan Song, Tianyang Wang, Ziqian Bi, Ming Liu

机构 * JTB Technology Corp.（JTB技术公司） ； Stockton University（斯托顿大学） ； University of Wisconsin-Madison（威斯康星大学麦迪逊分校） ； AppCubic USA（AppCubic美国公司） ； Nomad Sustaintech LTD（Nomad可持续科技有限公司） ； Georgia Institute of Technology（佐治亚理工学院） ； Emory University（埃默里大学） ； University of Liverpool（利物浦大学） ； Indiana University（印第安纳大学） ； Purdue University（普渡大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.AI

AI总结 本文综述了多模态大语言模型在视觉-语言任务中的应用，探讨了其架构、训练方法及挑战，并提供了理论与实践的全面指南。

Buffer replay enhances the robustness of multimodal learning under missing-modality

缓冲回放增强了在缺失模态下的多模态学习的鲁棒性

Hongye Zhu, Xuan Liu, Yanwen Ba, Jingye Xue, Shigeng Zhang

机构 * College of Computer Science and Electronic Engineering Hunan University（湖南大学计算机科学与电子工程学院） ； School of Computer Science Central South University（中南大学计算机学院） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；cross-modal(abstract)；vision-language-audio(abstract)；分类 cs.CV

AI总结 REP通过特征缓冲和动态初始化提升多模态学习在缺失模态下的鲁棒性

Adapter-state Sharing CLIP for Parameter-efficient Multimodal Sarcasm Detection

Soumyadeep Jana, Sahil Danayak, Sanasam Ranbir Singh

机构 * Department of Computer Science（计算机科学系） ； Engineering, Indian Institute of Technology Guwahati, India（工程系、印度理工学院瓜哇提学院、印度） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CL

Training-Free Multimodal Deepfake Detection via Graph Reasoning

Yuxin Liu, Fei Wang, Kun Li, Yiqi Nie, Junjie Chen, Yanyan Wei, Zhangling Duan, Zhaohong Jia

机构 * School of Internet, Anhui University, Hefei, China（安徽大学互联网学院） ； School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China（合肥工业大学计算机科学与信息工程学院） ； Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China（合肥综合国家科学中心人工智能研究院） ； Department of Computer Science, Hong Kong Baptist University, Hong Kong, China（香港 Baptist 大学计算机科学系） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

LLaVA-RadZ: Can Multimodal Large Language Models Effectively Tackle Zero-shot Radiology Recognition?

Bangyan Li, Wenxuan Huang, Zhenkun Gao, Yeqiang Wang, Yunhang Shen, Jingzhong Lin, Ling You, Yuxiang Shen, Shaohui Lin, Wanli Ouyang, Yuling Sun

机构 * East China Normal University（华东师范大学） ； The Chinese University of Hong Kong（香港中文大学） ； Northwest A&F University（西北农林科技大学） ； Tencent Youtu Lab（腾讯优图实验室） ； Xiamen University（厦门大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV

Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization

Alberto Compagnoni, Davide Caffagni, Nicholas Moratelli, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara

机构 * University of Modena and Reggio Emilia（摩德纳和雷吉奥艾米利亚大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments BMVC 2025

MV-CoRe: Multimodal Visual-Conceptual Reasoning for Complex Visual Question Answering

Jingwei Peng, Jiehao Chen, Mateo Alejandro Rojas, Meilin Zhang

机构 * Shaanxi University of Technology（陕西理工大学） ； Technological University of Peru（秘鲁技术大学） 纠错

专题命中 图文多模态 ：multimodal(title,abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Images

Hongyu Ge, Longkun Hao, Zihui Xu, Zhenxin Lin, Bin Li, Shoujun Zhou, Hongjin Zhao, Yihang Liu

机构 * The Hong Kong University of Sciences and Technology, Guangzhou（香港科学与技术大学（广州）） ； Beihang University（北航大学） ； Shandong University（山东大学） ； Hubei University（湖北大学） ； Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所） ； Australian National University（澳大利亚国立大学） 纠错

专题命中 图文多模态 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs

Haodong Chen, Haojian Huang, Junhao Dong, Mingzhe Zheng, Dian Shao

机构 * School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院） ； The University of Hong Kong（香港大学） ； Nanyang Technological University（南洋理工大学） ； School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院） ； Unmanned System Research Institute, Northwestern Polytechnical University（西北工业大学无人系统研究所） 纠错

专题命中 图文多模态 ：multi-modal(title,abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted to ACM MM 2024

Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources

Weizhi Wang, Yu Tian, Linjie Yang, Heng Wang, Xifeng Yan

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CL

LLaVA-UHD v2: an MLLM Integrating High-Resolution Semantic Pyramid via Hierarchical Window Transformer

Yipeng Zhang, Yifan Liu, Zonghao Guo, Yidan Zhang, Xuesong Yang, Xiaoying Zhang, Chi Chen, Jun Song, Bo Zheng, Yuan Yao, Zhiyuan Liu, Tat-Seng Chua, Maosong Sun

专题命中 图文多模态 ：MLLM(title,abstract)；multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

M^2ConceptBase: A Fine-Grained Aligned Concept-Centric Multimodal Knowledge Base

Zhiwei Zha, Jiaan Wang, Zhixu Li, Xiangru Zhu, Wei Song, Yanghua Xiao

专题命中 图文多模态 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.AI

Comments Accepted by CIKM2024. The code and data can be found at https://github.com/AwellmanZha/M2ConceptBase

Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Models

Pete Janowczyk, Linda Laurier, Ave Giulietta, Arlo Octavia, Meade Cleti

专题命中 图文多模态 ：multi-modal(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.AI

Face-MLLM: A Large Face Perception Model

Haomiao Sun, Mingjie He, Tianheng Lian, Hu Han, Shiguang Shan

专题命中 图文多模态 ：MLLM(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs

Fengzhu Zeng, Wenqian Li, Wei Gao, Yan Pang

专题命中 图文多模态 ：multimodal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CL

Comments EMNLP 2024 Findings

TAMM: TriAdapter Multi-Modal Learning for 3D Shape Understanding

Zhihao Zhang, Shengcao Cao, Yu-Xiong Wang

专题命中 图文多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments This paper is accepted by CVPR 2024

Visual Foundation Models Boost Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation

Jingyi Xu, Weidong Yang, Lingdong Kong, Youquan Liu, Rui Zhang, Qingyuan Zhou, Ben Fei

专题命中 图文多模态 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 8 pages, 6 figures

Deeply Coupled Cross-Modal Prompt Learning

Xuejing Liu, Wei Tang, Jinghui Lu, Rui Zhao, Zhaojun Guo, Fei Tan

专题命中 图文多模态 ：cross-modal(title,abstract)；multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CV

Comments Accepted by ACL 2023 findings

Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval

Yuan Yuan, Yang Zhan, Zhitong Xiong

专题命中 图文多模态 ：image-text(title,abstract)；multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

RegionBLIP: A Unified Multi-modal Pre-training Framework for Holistic and Regional Comprehension

Qiang Zhou, Chaohui Yu, Shaofeng Zhang, Sitong Wu, Zhibing Wang, Fan Wang

专题命中 图文多模态 ：multi-modal(title,abstract)；MLLM(abstract)；image-text(abstract)；分类 cs.CV

BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency

Shuo Yang, Zhaopan Xu, Kai Wang, Yang You, Hongxun Yao, Tongliang Liu, Min Xu

专题命中 图文多模态 ：cross-modal(title,abstract)；multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2023

Instruction-Following Agents with Multimodal Transformer

Hao Liu, Lisa Lee, Kimin Lee, Pieter Abbeel

专题命中 图文多模态 ：multimodal(title,abstract)；cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments fixed a typo in affiliation

VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix

Teng Wang, Wenhao Jiang, Zhichao Lu, Feng Zheng, Ran Cheng, Chengguo Yin, Ping Luo

专题命中 图文多模态 ：cross-modal(title,abstract)；multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Multimodal Fake News Detection via CLIP-Guided Learning

Yangming Zhou, Qichao Ying, Zhenxing Qian, Sheng Li, Xinpeng Zhang

专题命中 图文多模态 ：multimodal(title,abstract)；multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Submitted to CIKM 2022

SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning

Zhecan Wang, Haoxuan You, Liunian Harold Li, Alireza Zareian, Suji Park, Yiqing Liang, Kai-Wei Chang, Shih-Fu Chang

专题命中 图文多模态 ：image-text(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments AAAI 2022

Journal ref AAAI 2022

Understanding, Categorizing and Predicting Semantic Image-Text Relations

Christian Otto, Matthias Springstein, Avishek Anand, Ralph Ewerth

专题命中 图文多模态 ：image-text(title,abstract)；multimodal(abstract)；cross-modal(abstract)；分类 cs.MM

Comments 8 pages, 8 Figures, 5 tables

Journal ref In Proceedings of the 2019 on International Conference on Multimedia Retrieval (ICMR '19). ACM, New York, NY, USA, 168-176

Learning How To Ask: Cycle-Consistency Refines Prompts in Multimodal Foundation Models

Maurice Diesendruck, Jianzhe Lin, Shima Imani, Gayathri Mahalingam, Mingyang Xu, Jie Zhao

专题命中 图文多模态 ：multimodal(title)；multimodal foundation model(title)；分类 cs.CV、cs.CL