多模态大模型 - arXivDaily 专题

2410.10491 2025-03-21 cs.CV 83%

TWIST & SCOUT: Grounding Multimodal LLM-Experts by Forget-Free Tuning

Aritra Bhowmik, Mohammad Mahdi Derakhshani, Dennis Koelma, Yuki M. Asano, Martin R. Oswald, Cees G. M. Snoek

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2501.06828 2025-03-14 cs.CV 83%

GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensing

Ruizhe Ou, Yuan Hu, Fan Zhang, Jiaxin Chen, Yu Liu

专题命中图文多模态：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

2503.06446 2025-03-11 cs.CV 83%

M$^3$amba: CLIP-driven Mamba Model for Multi-modal Remote Sensing Classification

Mingxiang Cao, Weiying Xie, Xin Zhang, Jiaqing Zhang, Kai Jiang, Jie Lei, Yunsong Li

专题命中图文多模态：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2503.06106 2025-03-11 cs.CV 83%

Vision-aware Multimodal Prompt Tuning for Uploadable Multi-source Few-shot Domain Adaptation

Kuanghong Liu, Jin Wang, Kangjian He, Dan Xu, Xuejie Zhang

专题命中图文多模态：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by AAAI 2025

2503.04353 2025-03-07 cs.CV 83%

ObjMST: An Object-Focused Multimodal Style Transfer Framework

Chanda Grover Kamra, Indra Deep Mastan, Debayan Gupta

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments 8 pages, 8 Figures, 3 Tables

Journal ref Pattern Recognition Letters, Special Issues, 2024

2503.03651 2025-03-06 cs.CV 83%

DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles

Rui Zhao, Weijia Mao, Mike Zheng Shou

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2025

2406.05127 2025-02-27 cs.CV 83%

Towards Semantic Equivalence of Tokenization in Multimodal LLM

Shengqiong Wu, Hao Fei, Xiangtai Li, Jiayi Ji, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments ICLR-2025. The project page: https://chocowu.github.io/SeTok-web/

2502.17516 2025-02-26 cs.LG cs.AI 83%

A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models

Zihao Lin, Samyadeep Basu, Mohammad Beigi, Varun Manjunatha, Ryan A. Rossi, Zichao Wang, Yufan Zhou, Sriram Balasubramanian, Arman Zarei, Keivan Rezaei, Ying Shen, Barry Menglong Yao, Zhiyang Xu, Qin Liu, Yuxiang Zhang, Yan Sun, Shilong Liu, Li Shen, Hongxuan Li, Soheil Feizi, Lifu Huang

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.AI

Comments 30 pages, 4 Figures, 10 Tables

2405.13459 2025-02-18 cs.CV 83%

Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwards

Xiaoyu Yang, Jie Lu, En Yu

专题命中图文多模态：multi-modal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments ICLR 2025 Poster

2502.09906 2025-02-17 cs.CV 83%

Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding

Thanh-Dat Truong, Hoang-Quan Nguyen, Xuan-Bac Nguyen, Ashley Dowling, Xin Li, Khoa Luu

专题命中图文多模态：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

2501.18006 2025-01-31 cs.LG cs.AI cs.CR 83%

Topological Signatures of Adversaries in Multimodal Alignments

Minh Vu, Geigh Zollicoffer, Huy Mai, Ben Nebgen, Boian Alexandrov, Manish Bhattarai

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.AI

2501.15269 2025-01-28 cs.LG cs.CR cs.CV 83%

Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink

Yining Wang, Mi Zhang, Junjie Sun, Chenyue Wang, Min Yang, Hui Xue, Jialing Tao, Ranjie Duan, Jiexi Liu

专题命中图文多模态：multi-modal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments USENIX Security 2025

2501.01720 2025-01-27 cs.CV 83%

Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models

Guosheng Zhang, Keyao Wang, Haixiao Yue, Ajian Liu, Gang Zhang, Kun Yao, Errui Ding, Jingdong Wang

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted to AAAI2025(Oral)

2501.11469 2025-01-22 cs.CV cs.LG 83%

MASS: Overcoming Language Bias in Image-Text Matching

Jiwan Chung, Seungwon Lim, Sangkyu Lee, Youngjae Yu

专题命中图文多模态：image-text(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments AAAI 2025

2411.10950 2025-01-14 cs.CL 83%

Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering

Zeping Yu, Sophia Ananiadou

专题命中图文多模态：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CL

Comments preprint

2311.07594 2025-01-09 cs.CL cs.AI cs.CV cs.MM 83%

How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model

Shezheng Song, Xiaopeng Li, Shasha Li, Shan Zhao, Jie Yu, Jun Ma, Xiaoguang Mao, Weimin Zhang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by TKDE

2409.05840 2025-01-03 cs.CL 83%

MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct

Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang, Minzheng Wang, Pengpeng Zeng, Lianli Gao, Heng Tao Shen, Yunshui Li, Xiaobo Xia, Fei Huang, Jingkuan Song, Yongbin Li

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CL

2406.08394 2025-01-03 cs.CV 83%

VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks

Jiannan Wu, Muyan Zhong, Sen Xing, Zeqiang Lai, Zhaoyang Liu, Zhe Chen, Wenhai Wang, Xizhou Zhu, Lewei Lu, Tong Lu, Ping Luo, Yu Qiao, Jifeng Dai

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 44 pages

2412.20927 2024-12-31 cs.CV 83%

Enhanced Multimodal RAG-LLM for Accurate Visual Question Answering

Junxiao Xue, Quan Deng, Fei Yu, Yanhao Wang, Jun Wang, Yuehua Li

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 6 pages, 3 figures, under review

2412.18826 2024-12-30 cs.CL 83%

RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting

Yilei Jiang, Yingshui Tan, Xiangyu Yue

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

2412.16869 2024-12-24 cs.CV 83%

CoF: Coarse to Fine-Grained Image Understanding for Multi-modal Large Language Models

Yeyuan Wang, Dehong Gao, Bin Li, Rujiao Long, Lei Yi, Xiaoyan Cai, Libin Yang, Jinxia Zhang, Shanqing Yu, Qi Xuan

专题命中图文多模态：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 5 pages, Accepted by ICASSP2025, full paper

2412.11124 2024-12-24 cs.CV 83%

Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning

Shengqiong Wu, Hao Fei, Liangming Pan, William Yang Wang, Shuicheng Yan, Tat-Seng Chua

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 16 pages, 10 figures, accepted by AAAI 25

2412.15813 2024-12-23 cs.CV 83%

Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations

Yi Zhang, Chun-Wun Cheng, Junyi He, Zhihai He, Carola-Bibiane Schönlieb, Yuyan Chen, Angelica I Aviles-Rivero

专题命中图文多模态：cross-modal(title,abstract)；image-text(abstract)；分类 cs.CV

2411.01756 2024-12-17 cs.CV 83%

ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model

Yiming Sun, Fan Yu, Shaoxiang Chen, Yu Zhang, Junwei Huang, Chenhui Li, Yang Li, Changbo Wang

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2412.10372 2024-12-16 cs.CV 83%

UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities

Muhammad Uzair Khattak, Shahina Kunhimon, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan

专题命中图文多模态：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Code, models and demo available at https://github.com/mbzuai-oryx/UniMed-CLIP

2412.09616 2024-12-16 cs.CV 83%

V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

Junqi Ge, Ziyi Chen, Jintao Lin, Jinguo Zhu, Xihui Liu, Jifeng Dai, Xizhou Zhu

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments The code and models will be available at https://github.com/OpenGVLab/V2PE

2412.07119 2024-12-11 cs.CV 83%

DiffCLIP: Few-shot Language-driven Multimodal Classifier

Jiaqing Zhang, Mingxiang Cao, Xue Yang, Kai Jiang, Yunsong Li

专题命中图文多模态：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV

2412.03665 2024-12-06 cs.CV cs.AI cs.CL cs.MM 83%

Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis

Davide Bucciarelli, Nicholas Moratelli, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ECCV 2024 Workshop on Green Foundation Models

2406.02884 2024-11-27 cs.CV 83%

PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM

Tao Yang, Yingmin Luo, Zhongang Qi, Yang Wu, Ying Shan, Chang Wen Chen

专题命中图文多模态：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 13 pages; with PosterGen as extension; IEEE template

2411.09301 2024-11-15 cs.CV 83%

LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation

Zhenshi Li, Dilxat Muhtar, Feng Gu, Xueliang Zhang, Pengfeng Xiao, Guangjun He, Xiaoxiang Zhu

专题命中图文多模态：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

1. 图文多模态 4634 篇

TWIST & SCOUT: Grounding Multimodal LLM-Experts by Forget-Free Tuning

GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensing

M$^3$amba: CLIP-driven Mamba Model for Multi-modal Remote Sensing Classification

Vision-aware Multimodal Prompt Tuning for Uploadable Multi-source Few-shot Domain Adaptation

ObjMST: An Object-Focused Multimodal Style Transfer Framework

DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles

Towards Semantic Equivalence of Tokenization in Multimodal LLM

A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models

Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwards

Insect-Foundation: A Foundation Model and Large Multimodal Dataset for Vision-Language Insect Understanding

Topological Signatures of Adversaries in Multimodal Alignments

Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink

Interpretable Face Anti-Spoofing: Enhancing Generalization with Multimodal Large Language Models

MASS: Overcoming Language Bias in Image-Text Matching

Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering

How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model

MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct

VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks

Enhanced Multimodal RAG-LLM for Accurate Visual Question Answering

RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Prompting

CoF: Coarse to Fine-Grained Image Understanding for Multi-modal Large Language Models

Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning

Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations

ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model

UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities

V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

DiffCLIP: Few-shot Language-driven Multimodal Classifier

Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis

PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM

LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation