多模态大模型 - arXivDaily 专题

2405.11793 2025-08-27 cs.CV 83%

MM-Retinal: Knowledge-Enhanced Foundational Pretraining with Fundus Image-Text Expertise

Ruiqi Wu, Chenran Zhang, Jianle Zhang, Yi Zhou, Tao Zhou, Huazhu Fu

机构 * School of Computer Science and Engineering, Southeast University, China（东南大学计算机科学与工程学院）； Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, China（教育部新一代人工智能技术及其交叉应用重点实验室）； Nanjing University of Science and Technology, Nanjing, China（南京理工大学）； Agency for Science, Technology and Research (A*STAR), Singapore（新加坡科技研究局）

专题命中图文多模态：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Early Accepted by The International Conference on Medical Image Computing and Computer Assisted Intervention(MICCAI)2024

2508.17037 2025-08-26 cs.CV 83%

F4-ITS: Fine-grained Feature Fusion for Food Image-Text Search

Raghul Asokan

机构 * HyperVerge

专题命中图文多模态：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV

2508.16762 2025-08-26 cs.CL cs.CY 83%

Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation

Arka Mukherjee, Shreya Ghosh

机构 * KIIT Deemed University（KIIT大学）； Indian Institute of Technology (IIT) Bhubaneswar（印度理工学院（Bhubaneswar分校））

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

Comments Accepted at ASI @ ICCV 2025

2508.07414 2025-08-13 cs.CL cs.LG 83%

Grounding Multilingual Multimodal LLMs With Cultural Knowledge

Jean de Dieu Nyandwi, Yueqi Song, Simran Khanuja, Graham Neubig

机构 * Carnegie Mellon University（卡内基梅隆大学）

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

2508.05167 2025-08-08 cs.CV 83%

PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems

Qi Guo, Xiaojun Jia, Shanmin Pang, Simeng Qin, Lin Wang, Ju Jia, Yang Liu, Qing Guo

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2508.04175 2025-08-07 cs.CV 83%

AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization

Jingyi Liao, Yongyi Su, Rong-Cheng Tu, Zhao Jin, Wenhao Sun, Yiting Li, Dacheng Tao, Xun Xu, Xulei Yang

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2502.12513 2025-08-06 cs.CV 83%

RealSyn: An Effective and Scalable Multimodal Interleaved Document Transformation Paradigm

Tiancheng Gu, Kaicheng Yang, Chaoyi Zhang, Yin Xie, Xiang An, Ziyong Feng, Dongnan Liu, Weidong Cai, Jiankang Deng

机构 * The University of Sydney（悉尼大学）； DeepGlint ； Imperial College London（伦敦帝国学院）

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments 15 pages, 12 figures, Accepted by ACM MM2025, Webpage: https://garygutc.github.io/RealSyn

2508.01254 2025-08-05 cs.CV 83%

Self-Enhanced Image Clustering with Cross-Modal Semantic Consistency

Zihan Li, Wei Sun, Jing Hu, Jianhua Yin, Jianlong Wu, Liqiang Nie

专题命中图文多模态：cross-modal(title,abstract)；image-text(abstract)；分类 cs.CV

2507.22431 2025-07-31 cs.CV 83%

HQ-CLIP: Leveraging Large Vision-Language Models to Create High-Quality Image-Text Datasets and CLIP Models

Zhixiang Wei, Guangting Wang, Xiaoxiao Ma, Ke Mei, Huaian Chen, Yi Jin, Fengyun Rao

机构 * University of Science and Technology of China（中国科学技术大学）； WeChat Vision, Tencent Inc.（腾讯公司）

专题命中图文多模态：image-text(title,abstract)；cross-modal(abstract)；分类 cs.CV

2507.22074 2025-07-31 cs.LG cs.CL 83%

CIMR: Contextualized Iterative Multimodal Reasoning for Robust Instruction Following in LVLMs

Yangshu Yuan, Heng Chen, Xinyi Jiang, Christian Ng, Kexin Qiu

机构 * Singapore Institute of Management（新加坡管理学院）

专题命中图文多模态：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CL

2502.15910 2025-07-24 cs.CL 83%

Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models

Zheyuan Liu, Guangyao Dou, Xiangchi Yuan, Chunhui Zhang, Zhaoxuan Tan, Meng Jiang

机构 * University of Notre Dame（诺丁汉大学）； University of Pennsylvania（宾夕法尼亚大学）； Georgia Institute of Technology（佐治亚理工学院）； Dartmouth College（达特茅斯学院）

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

Comments ACL 2025 Main Conference

2503.06312 2025-07-23 cs.CV 83%

DOFA-CLIP: Multimodal Vision-Language Foundation Models for Earth Observation

Zhitong Xiong, Yi Wang, Weikang Yu, Adam J Stewart, Jie Zhao, Nils Lehmann, Thomas Dujardin, Zhenghang Yuan, Pedram Ghamisi, Xiao Xiang Zhu

机构 * Technical University of Munich（慕尼黑技术大学）； Helmholtz-Zentrum Dresden-Rossendorf（德累斯顿-罗斯托克亥姆霍尔茨中心）

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments code & weights: https://github.com/xiong-zhitong/DOFA-CLIP

2507.08410 2025-07-14 cs.CV 83%

Multi-modal Mutual-Guidance Conditional Prompt Learning for Vision-Language Models

Shijun Yang, Xiang Zhang, Wanqing Zhao, Hangzai Luo, Sheng Zhong, Jinye Peng, Jianping Fan

机构 * School of Information and Technology, Northwest University（信息与技术学院，西北大学）

专题命中图文多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 21 pages, 8 figures

2507.00926 2025-07-02 cs.MM cs.LG 83%

HyperFusion: Hierarchical Multimodal Ensemble Learning for Social Media Popularity Prediction

Liliang Ye, Yunyao Zhang, Yafeng Wu, Yi-Ping Phoebe Chen, Junqing Yu, Wei Yang, Zikai Song

机构 * Huazhong University of Science and Technology（华中科技大学）； La Trobe University（拉特罗布大学）

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.MM

2503.19564 2025-07-01 cs.LG cs.AI 83%

FedMM-X: A Trustworthy and Interpretable Framework for Federated Multi-Modal Learning in Dynamic Environments

Sree Bhargavi Balija

机构 * University of California, San Diego（加州大学圣地亚哥分校）

专题命中图文多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.AI

2506.12724 2025-06-17 cs.CV 83%

Dynamic Modality Scheduling for Multimodal Large Models via Confidence, Uncertainty, and Semantic Consistency

Hiroshi Tanaka, Anika Rao, Hana Satou, Michael Johnson, Sofia García

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

2402.04416 2025-06-11 cs.CV cs.LG 83%

Multimodal Unsupervised Domain Generalization by Retrieving Across the Modality Gap

Christopher Liao, Christian So, Theodoros Tsiligkaridis, Brian Kulis

机构 * Boston University（波士顿大学）； MIT Lincoln Laboratory（麻省理工学院林肯实验室）

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2506.01307 2025-06-03 cs.CR cs.AI 83%

Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models

Youze Wang, Wenbo Hu, Yinpeng Dong, Jing Liu, Hanwang Zhang, Richang Hong

机构 * School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）； Tsinghua University（清华大学）； Institute of automation, Chinese academy of science（中国科学院自动化研究所）； School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.AI

2506.00903 2025-06-03 cs.CV 83%

Leveraging CLIP Encoder for Multimodal Emotion Recognition

Yehun Song, Sunyoung Cho

机构 * Agency for Defense Development（国防发展机构）； Sookmyung Women’s University（苏镜女子大学）

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted at IEEE/CVF WACV 2025, pp.6115-6124, 2025

Journal ref Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp.6115-6124

2308.12636 2025-06-03 cs.MM 83%

Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learning

Youze Wang, Wenbo Hu, Yinpeng Dong, Hanwang Zhang, Hang Su, Richang Hong

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.MM

2505.20029 2025-05-27 q-bio.NC cs.AI cs.LG 83%

Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)

Subba Reddy Oota, Akshett Jindal, Ishani Mondal, Khushbu Pahwa, Satya Sai Srinath Namburi, Manish Shrivastava, Maneesh Singh, Bapi S. Raju, Manish Gupta

机构 * Technische Universität Berlin（柏林技术大学）； IIIT Hyderabad（海得拉巴国家理工学院）； Univ of Maryland（马里兰大学）； Rice Univ（Rice 大学）； Univ of Wisconsin - Madison（威斯康星大学麦迪逊分校）； Spector Inc（Spector 公司）； Microsoft（微软公司）

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.AI

Comments 30 pages, 22 figures, The Thirteenth International Conference on Learning Representations, ICLR-2025, Singapore. https://openreview.net/pdf?id=xkgfLXZ4e0

Journal ref ICLR-2025, Singapore

2505.19812 2025-05-27 cs.CV 83%

Efficient Multi-modal Long Context Learning for Training-free Adaptation

Zehong Ma, Shiliang Zhang, Longhui Wei, Qi Tian

专题命中图文多模态：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Accepted to ICML2025

2505.06356 2025-05-13 cs.CV 83%

Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA

Karthik Reddy Kanjula, Surya Guthikonda, Nahid Alam, Shayekh Bin Islam

机构 * Cohere for AI Community（Cohere for AI社区）； Cisco Meraki ； Indiana University Bloomington（印第安纳大学布卢明顿分校）； Bangladesh University of Engineering and Technology（孟加拉工程与技术大学）

专题命中图文多模态：image-text(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Accepted at ReGenAI CVPR2025 Workshop as Oral

2505.04601 2025-05-08 cs.CV 83%

OpenVision: A Fully-Open, Cost-Effective Family of Advanced Vision Encoders for Multimodal Learning

Xianhang Li, Yanqing Liu, Haoqin Tu, Hongru Zhu, Cihang Xie

机构 * University of California, Santa Cruz（加州大学圣克鲁兹分校）

专题命中图文多模态：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

2505.03703 2025-05-07 cs.CV cs.LG 83%

Fill the Gap: Quantifying and Reducing the Modality Gap in Image-Text Representation Learning

François Role, Sébastien Meyer, Victor Amblard

机构 * Université Paris-Cité（巴黎-cite大学）； Pôle d’Expertise de la Régulation Numérique (PEReN)（数字监管专家中心）

专题命中图文多模态：image-text(title,abstract)；multimodal(abstract)；分类 cs.CV

2505.02013 2025-05-06 cs.CV 83%

MLLM-Enhanced Face Forgery Detection: A Vision-Language Fusion Solution

Siran Peng, Zipei Wang, Li Gao, Xiangyu Zhu, Tianshuo Zhang, Ajian Liu, Haoyuan Zhang, Zhen Lei

机构 * CASIA ； UCAS ； CMCC ； CAIR, HKISI, CAS

专题命中图文多模态：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV

2412.04332 2025-04-14 cs.CV 83%

Liquid: Language Models are Scalable and Unified Multi-modal Generators

Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, Xiang Bai

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；MLLM(abstract)；分类 cs.CV

Comments Technical report. Project page: https://foundationvision.github.io/Liquid/

2404.18930 2025-04-03 cs.CV 83%

Hallucination of Multimodal Large Language Models: A Survey

Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, Mike Zheng Shou

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 228 references

2503.10437 2025-04-02 cs.CV 83%

4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models

Wanhua Li, Renping Zhou, Jiawei Zhou, Yingwei Song, Johannes Herter, Minghan Qin, Gao Huang, Hanspeter Pfister

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2025. Project Page: https://4d-langsplat.github.io

2503.20502 2025-04-01 cs.CV 83%

MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning

Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Jiayi Ji, Jie Lou, Debing Zhang, Rongrong Ji

专题命中图文多模态：MLLM(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Tech Report

1. 图文多模态 4642 篇

MM-Retinal: Knowledge-Enhanced Foundational Pretraining with Fundus Image-Text Expertise

F4-ITS: Fine-grained Feature Fusion for Food Image-Text Search

Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation

Grounding Multilingual Multimodal LLMs With Cultural Knowledge

PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems

AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization

RealSyn: An Effective and Scalable Multimodal Interleaved Document Transformation Paradigm

Self-Enhanced Image Clustering with Cross-Modal Semantic Consistency

HQ-CLIP: Leveraging Large Vision-Language Models to Create High-Quality Image-Text Datasets and CLIP Models

CIMR: Contextualized Iterative Multimodal Reasoning for Robust Instruction Following in LVLMs

Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models

DOFA-CLIP: Multimodal Vision-Language Foundation Models for Earth Observation

Multi-modal Mutual-Guidance Conditional Prompt Learning for Vision-Language Models

HyperFusion: Hierarchical Multimodal Ensemble Learning for Social Media Popularity Prediction

FedMM-X: A Trustworthy and Interpretable Framework for Federated Multi-Modal Learning in Dynamic Environments

Dynamic Modality Scheduling for Multimodal Large Models via Confidence, Uncertainty, and Semantic Consistency

Multimodal Unsupervised Domain Generalization by Retrieving Across the Modality Gap

Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models

Leveraging CLIP Encoder for Multimodal Emotion Recognition

Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learning

Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)

Efficient Multi-modal Long Context Learning for Training-free Adaptation

Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA

OpenVision: A Fully-Open, Cost-Effective Family of Advanced Vision Encoders for Multimodal Learning

Fill the Gap: Quantifying and Reducing the Modality Gap in Image-Text Representation Learning

MLLM-Enhanced Face Forgery Detection: A Vision-Language Fusion Solution

Liquid: Language Models are Scalable and Unified Multi-modal Generators

Hallucination of Multimodal Large Language Models: A Survey

4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models

MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning