多模态大模型

DRIM: Learning Disentangled Representations from Incomplete Multimodal Healthcare Data

Lucas Robinet, Ahmad Berjaoui, Ziad Kheil, Elizabeth Cohen-Jonathan Moyal

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Recall: Empowering Multimodal Embedding for Edge Devices

Dongqi Cai, Shangguang Wang, Chen Peng, Zeling Zhang, Mengwei Xu

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Multimodal Generalized Category Discovery

Yuchang Su, Renping Zhou, Siyu Huang, Xingjian Li, Tianyang Wang, Ziyue Wang, Min Xu

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis

Sijie Mai, Yu Zhao, Ying Zeng, Jianhua Yao, Haifeng Hu

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles

Qiujing Lu, Xuanhan Wang, Yiwei Jiang, Guangming Zhao, Mingyue Ma, Shuo Feng

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Functional Graph Convolutional Networks: A unified multi-task and multi-modal learning framework to facilitate health and social-care insights

Tobia Boschi, Francesca Bonin, Rodrigo Ordonez-Hurtado, Cécile Rousseau, Alessandra Pascale, John Dinsmore

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.AI

Journal ref IJCAI 2024

FashionLOGO: Prompting Multimodal Large Language Models for Fashion Logo Embeddings

Zhen Wang, Da Li, Yulin Su, Min Yang, Minghui Qiu, Walton Wang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

CCPL: Cross-modal Contrastive Protein Learning

Jiangbin Zheng, Stan Z. Li

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.AI

Comments Accepted to ICPR 2024

Rethinking Sparse Lexical Representations for Image Retrieval in the Age of Rising Multi-Modal Large Language Models

Kengo Nakata, Daisuke Miyashita, Youyang Ng, Yasuto Hoshi, Jun Deguchi

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted to ECCV 2024 Workshops: 2nd Workshop on Traditional Computer Vision in the Age of Deep Learning (TradiCV)

Parkinson's Disease Classification Using Contrastive Graph Cross-View Learning with Multimodal Fusion of SPECT Images and Clinical Features

Jun-En Ding, Chien-Chin Hsu, Feng Liu

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Multimodal self-supervised learning for lesion localization

Hao Yang, Hong-Yu Zhou, Cheng Li, Weijian Huang, Jiarun Liu, Yong Liang, Guangming Shi, Hairong Zheng, Qiegen Liu, Shanshan Wang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

FMiFood: Multi-modal Contrastive Learning for Food Image Classification

Xinyue Pan, Jiangpeng He, Fengqing Zhu

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Structure-Aware Residual-Center Representation for Self-Supervised Open-Set 3D Cross-Modal Retrieval

Yang Xu, Yifan Feng, Yu Jiang

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.MM

Comments ICME 2024

Multimodal Pre-training Framework for Sequential Recommendation via Contrastive Learning

Lingzi Zhang, Xin Zhou, Zhiwei Zeng, Zhiqi Shen

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.MM

Comments 22 pages

ETSCL: An Evidence Theory-Based Supervised Contrastive Learning Framework for Multi-modal Glaucoma Grading

Zhiyuan Yang, Bo Zhang, Yufei Shi, Ningze Zhong, Johnathan Loh, Huihui Fang, Yanwu Xu, Si Yong Yeo

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by Ophthalmic Medical Image Analysis Workshop at MICCAI'24

Multimodal Label Relevance Ranking via Reinforcement Learning

Taian Guo, Taolin Zhang, Haoqian Wu, Hanjun Li, Ruizhi Qiao, Xing Sun

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to ECCV2024

LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

Ran Chen, Xueqi Yao, Xuhui Jiang

专题命中 跨模态检索 ：multi-modal(title)；cross-modal(abstract)；分类 cs.AI

SEMINAR: Search Enhanced Multi-modal Interest Network and Approximate Retrieval for Lifelong Sequential Recommendation

Kaiming Shen, Xichen Ding, Zixiang Zheng, Yuqi Gong, Qianqian Li, Zhongyi Liu, Guannan Zhang

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.AI

Comments 9 pages,code released

Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval

Hanwen Su, Ge Song, Kai Huang, Jiyan Wang, Ming Yang

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Zero-shot Composed Image Retrieval Considering Query-target Relationship Leveraging Masked Image-text Pairs

Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama

专题命中 跨模态检索 ：image-text(title,abstract)；分类 cs.CV

Comments Accepted as a conference paper in IEEE ICIP 2024

Data curation via joint example selection further accelerates multimodal learning

Talfan Evans, Nikhil Parthasarathy, Hamza Merzic, Olivier J. Henaff

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Comments Main text: 9 pages, 5 figures, 3 tables, 1 algorithm. Appendix: 7 pages, 5 figures, 1 table, 2. algorithm

Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning

Chi-Sheng Chen, Chun-Shu Wei

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Comments 19 pages, 14 figures

Towards a multimodal framework for remote sensing image change retrieval and captioning

Roger Ferrod, Luigi Di Caro, Dino Ienco

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

High-level Codes and Fine-grained Weights for Online Multi-modal Hashing Retrieval

Yu-Wei Zhan, Xiao-Ming Wu, Xin Luo, Yinwei Wei, Xin-Shun Xu

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.MM

Comments 32 pages, 4 figures

MORE: Multi-mOdal REtrieval Augmented Generative Commonsense Reasoning

Wanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CL

Comments Published as a conference paper at ACL Findings 2024

Open-World Human-Object Interaction Detection via Multi-modal Prompts

Jie Yang, Bingliang Li, Ailing Zeng, Lei Zhang, Ruimao Zhang

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Comments CVPR24. arXiv admin note: text overlap with arXiv:2305.12252

Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities

Mingcheng Li, Dingkang Yang, Xiao Zhao, Shuaibing Wang, Yan Wang, Kun Yang, Mingyang Sun, Dongliang Kou, Ziyun Qian, Lihua Zhang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learning

Alex Jinpeng Wang, Linjie Li, Yiqi Lin, Min Li, Lijuan Wang, Mike Zheng Shou

专题命中 跨模态检索 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV

Comments 12 pages. The website is \url{https://fingerrec.github.io/visincontext}

FinEmbedDiff: A Cost-Effective Approach of Classifying Financial Documents with Vector Sampling using Multi-modal Embedding Models

Anjanava Biswas, Wrick Talukdar

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.AI

Comments 10 pages, 3 figures

Journal ref International Research Journal of Modernization in Engineering Technology and Science: Vol. 06 (2024): No. 5, 6142-6152

MGI: Multimodal Contrastive pre-training of Genomic and Medical Imaging

Jiaying Zhou, Mingzhou Jiang, Junde Wu, Jiayuan Zhu, Ziyue Wang, Yueming Jin

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV