多模态大模型 - arXivDaily 专题

2406.10701 2024-10-15 cs.CL 79%

MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding

Baixuan Xu, Weiqi Wang, Haochen Shi, Wenxuan Ding, Huihao Jing, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Long Chen, Yangqiu Song

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments EMNLP 2024 main conference

2212.07050 2024-10-14 cs.LG cs.CV eess.IV 79%

Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders

Jongseong Jang, Daeun Kyung, Seung Hwan Kim, Honglak Lee, Kyunghoon Bae, Edward Choi

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Journal ref Sci Rep 14, 23199 (2024)

2410.07783 2024-10-11 cs.CV 79%

CLIP Multi-modal Hashing for Multimedia Retrieval

Jian Zhu, Mingkai Sheng, Zhangmin Huang, Jingfei Chang, Jinling Jiang, Jian Long, Cheng Luo, Lei Liu

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by 31st International Conference on MultiMedia Modeling (MMM2025)

2410.04426 2024-10-08 cs.CV 79%

CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection

Devank, Jayateja Kalla, Soma Biswas

专题命中图文多模态：multi-modal(title)；image-text(abstract)；分类 cs.CV

Comments Accepted in ACCV 2024

2410.00905 2024-10-02 cs.CV 79%

Removing Distributional Discrepancies in Captions Improves Image-Text Alignment

Yuheng Li, Haotian Liu, Mu Cai, Yijun Li, Eli Shechtman, Zhe Lin, Yong Jae Lee, Krishna Kumar Singh

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

2409.18753 2024-09-30 cs.CV 79%

Enhancing Explainability in Multimodal Large Language Models Using Ontological Context

Jihen Amara, Birgitta König-Ries, Sheeba Samuel

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

2409.16261 2024-09-25 cs.CV 79%

CDChat: A Large Multimodal Model for Remote Sensing Change Description

Mubashir Noman, Noor Ahsan, Muzammal Naseer, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Fahad Shahbaz Khan

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

2409.13407 2024-09-23 cs.CV 79%

Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model

Li Zhou, Xu Yuan, Zenghui Sun, Zikun Zhou, Jingsong Lan

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Code and dataset will be released at https://github.com/lizhou-cs/mglmm. 7 pages, 4 figures with Supplementary Material

2409.07463 2024-09-13 cs.CV cs.LG 79%

Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis

Sakhinana Sagar Srinivas, Geethan Sannidhi, Venkataramana Runkana

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；分类 cs.CV

Comments Paper published at AAAI 2024 Spring Symposium Series

2312.06726 2024-09-05 cs.CV 79%

Filter & Align: Leveraging Human Knowledge to Curate Image-Text Data

Lei Zhang, Fangxun Shu, Tianyang Liu, Sucheng Ren, Hao Jiang, Cihang Xie

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

2408.05097 2024-08-12 cs.LG cs.AI 79%

Hyperbolic Learning with Multimodal Large Language Models

Paolo Mandica, Luca Franco, Konstantinos Kallidromitis, Suzanne Petryk, Fabio Galasso

专题命中图文多模态：multimodal(title)；multi-modal(abstract)；分类 cs.AI

Comments ECCV 2024 - Beyond Euclidean Workshop

2403.01988 2024-08-07 cs.CL 79%

FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLMs

Xuannan Liu, Peipei Li, Huaibo Huang, Zekun Li, Xing Cui, Jiahao Liang, Lixiong Qin, Weihong Deng, Zhaofeng He

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by ACM MM 2024.Project page: https://liuxuannan.github.io/FKA_Owl.github.io/

2408.02718 2024-08-07 cs.CV 79%

MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

Fanqing Meng, Jin Wang, Chuanhao Li, Quanfeng Lu, Hao Tian, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, Kaipeng Zhang, Wenqi Shao

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Project Page: https://mmiu-bench.github.io/

2408.02484 2024-08-06 cs.CV 79%

Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection

Ting Lei, Shaofeng Yin, Yuxin Peng, Yang Liu

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

2406.14343 2024-07-23 cs.AI 79%

IWISDM: Assessing instruction following in multimodal models at scale

Xiaoxuan Lei, Lucas Gomez, Hao Yuan Bai, Pouya Bashivan

专题命中图文多模态：multimodal(title,abstract)；分类 cs.AI

2404.12467 2024-07-18 cs.CV cs.LG 79%

Towards Multi-modal Transformers in Federated Learning

Guangyu Sun, Matias Mendieta, Aritra Dutta, Xin Li, Chen Chen

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments 2024 European Conference on Computer Vision (ECCV)

2407.08196 2024-07-12 cs.AI 79%

SoupLM: Model Integration in Large Language and Multi-Modal Models

Yue Bai, Zichen Zhang, Jiasen Lu, Yun Fu

专题命中图文多模态：multi-modal(title)；multimodal(abstract)；分类 cs.AI

2403.06407 2024-07-09 cs.CV 79%

Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

Jiawei Chen, Yue Jiang, Dingkang Yang, Mingcheng Li, Jinjie Wei, Ziyun Qian, Lihua Zhang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by MICCAI 2024

2407.02586 2024-07-04 cs.CV 79%

Improving Visual Storytelling with Multimodal Large Language Models

Xiaochuan Lin, Xiangyong Chen

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments 10 pages

2406.17057 2024-06-26 astro-ph.IM astro-ph.GA cs.AI 79%

At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models

Dimitrios Tanoglidis, Bhuvnesh Jain

专题命中图文多模态：multimodal(title,abstract)；分类 cs.AI

Comments 5 pages, 3 images. Prepared for submission to RNAAS

2406.17047 2024-06-26 cs.CV 79%

Enhancing Scientific Figure Captioning Through Cross-modal Learning

Mateo Alejandro Rojas, Rafael Carranza

专题命中图文多模态：cross-modal(title)；multimodal(abstract)；分类 cs.CV

Comments 7 pages

2406.16141 2024-06-25 cs.CV 79%

Multimodal Multilabel Classification by CLIP

Yanming Guo

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

2406.14830 2024-06-24 cs.CV 79%

CLIP-Decoder : ZeroShot Multilabel Classification using Multimodal CLIP Aligned Representation

Muhammad Ali, Salman Khan

专题命中图文多模态：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Comments Accepted at ICCVW- VLAR

2406.14481 2024-06-21 cs.LG cs.AI cs.NE q-bio.NC 79%

Revealing Vision-Language Integration in the Brain with Multimodal Networks

Vighnesh Subramaniam, Colin Conwell, Christopher Wang, Gabriel Kreiman, Boris Katz, Ignacio Cases, Andrei Barbu

专题命中图文多模态：multimodal(title,abstract)；分类 cs.AI

Comments ICML 2024; 23 pages, 11 figures

2406.11820 2024-06-18 cs.CV 79%

Composing Object Relations and Attributes for Image-Text Matching

Khoi Pham, Chuong Huynh, Ser-Nam Lim, Abhinav Shrivastava

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted to CVPR'24

2406.04675 2024-06-10 cs.CV 79%

OVMR: Open-Vocabulary Recognition with Multi-Modal References

Zehong Ma, Shiliang Zhang, Longhui Wei, Qi Tian

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments CVPR2024

2406.01906 2024-06-05 cs.CV cs.IR 79%

ProGEO: Generating Prompts through Image-Text Contrastive Learning for Visual Geo-localization

Chen Mao, Jingqi Hu

专题命中图文多模态：image-text(title)；multi-modal(abstract)；分类 cs.CV

2405.20735 2024-06-03 cs.CV 79%

Language Augmentation in CLIP for Improved Anatomy Detection on Multi-modal Medical Images

Mansi Kakkar, Dattesh Shanbhag, Chandan Aladahalli, Gurunath Reddy M

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments $©$ 2024 IEEE. Accepted in 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2024

2312.02546 2024-05-30 cs.CV 79%

Machine Vision Therapy: Multimodal Large Language Models Can Enhance Visual Robustness via Denoising In-Context Learning

Zhuo Huang, Chang Liu, Yinpeng Dong, Hang Su, Shibao Zheng, Tongliang Liu

专题命中图文多模态：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Comments ICML 2024

2405.16301 2024-05-28 cs.CV cs.LG 79%

Active Learning for Finely-Categorized Image-Text Retrieval by Selecting Hard Negative Unpaired Samples

Dae Ung Jo, Kyuewang Lee, JaeHo Chung, Jin Young Choi

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

1. 图文多模态 4644 篇

MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding

Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders

CLIP Multi-modal Hashing for Multimedia Retrieval

CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection

Removing Distributional Discrepancies in Captions Improves Image-Text Alignment

Enhancing Explainability in Multimodal Large Language Models Using Ontological Context

CDChat: A Large Multimodal Model for Remote Sensing Change Description

Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model

Multi-Modal Instruction-Tuning Small-Scale Language-and-Vision Assistant for Semiconductor Electron Micrograph Analysis

Filter & Align: Leveraging Human Knowledge to Curate Image-Text Data

Hyperbolic Learning with Multimodal Large Language Models

FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLMs

MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection

IWISDM: Assessing instruction following in multimodal models at scale

Towards Multi-modal Transformers in Federated Learning

SoupLM: Model Integration in Large Language and Multi-Modal Models

Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

Improving Visual Storytelling with Multimodal Large Language Models

At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models

Enhancing Scientific Figure Captioning Through Cross-modal Learning

Multimodal Multilabel Classification by CLIP

CLIP-Decoder : ZeroShot Multilabel Classification using Multimodal CLIP Aligned Representation

Revealing Vision-Language Integration in the Brain with Multimodal Networks

Composing Object Relations and Attributes for Image-Text Matching

OVMR: Open-Vocabulary Recognition with Multi-Modal References

ProGEO: Generating Prompts through Image-Text Contrastive Learning for Visual Geo-localization

Language Augmentation in CLIP for Improved Anatomy Detection on Multi-modal Medical Images

Machine Vision Therapy: Multimodal Large Language Models Can Enhance Visual Robustness via Denoising In-Context Learning

Active Learning for Finely-Categorized Image-Text Retrieval by Selecting Hard Negative Unpaired Samples