多模态大模型

RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training

Muhe Ding, Yang Ma, Pengda Qin, Jianlong Wu, Yuhong Li, Liqiang Nie

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments 10 pages, 6 figures, Journal

Adapting Large Multimodal Models to Distribution Shifts: The Role of In-Context Learning

Guanglin Zhou, Zhongyi Han, Shiming Chen, Biwei Huang, Liming Zhu, Salman Khan, Xin Gao, Lina Yao

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 9 figures, 7 tables

UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generation

Xiangyu Zhao, Yuehan Zhang, Wenlong Zhang, Xiao-Ming Wu

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by EMNLP 2024, main conference

Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval

Kyle Buettner, Adriana Kovashka

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2024 Main - Short

Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment

Zhiyuan Li, Hailong Li, Anca L. Ralescu, Jonathan R. Dillman, Mekibib Altaye, Kim M. Cecil, Nehal A. Parikh, Lili He

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 35 pages. Submitted to journal

Journal ref Artificial Intelligence in Medicine, Volume 157, 2024, 102993

Leveraging Retrieval Augment Approach for Multimodal Emotion Recognition Under Missing Modalities

Qi Fan, Hongyu Yuan, Haolin Zuo, Rui Liu, Guanglai Gao

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Under reviewing

Find Rhinos without Finding Rhinos: Active Learning with Multimodal Imagery of South African Rhino Habitats

Lucia Gordon, Nikhil Behari, Samuel Collier, Elizabeth Bondi-Kelly, Jackson A. Killian, Catherine Ressijac, Peter Boucher, Andrew Davies, Milind Tambe

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 9 figures, IJCAI 2023 Special Track on AI for Good

Journal ref Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence. AI for Good. Pages 5977-5985. 2023

A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios

Christian Ganhör, Marta Moscati, Anna Hausberger, Shah Nawaz, Markus Schedl

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments Accepted at 18th ACM Conference on Recommender Systems (RecSys '24)

AsthmaBot: Multi-modal, Multi-Lingual Retrieval Augmented Generation For Asthma Patient Support

Adil Bahaj, Mounir Ghogho

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages

Retrieve, Annotate, Evaluate, Repeat: Leveraging Multimodal LLMs for Large-Scale Product Retrieval Evaluation

Kasra Hosseini, Thomas Kober, Josip Krapac, Roland Vollgraf, Weiwei Cheng, Ana Peleteiro Ramallo

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 13 pages, 5 figures, 4 Tables

Cross-Modal Denoising: A Novel Training Paradigm for Enhancing Speech-Image Retrieval

Lifeng Zhou, Yuke Li, Rui Deng, Yuting Yang, Haoqi Zhu

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CL、eess.AS

Comments arXiv admin note: substantial text overlap with arXiv:2408.13119

A Multi-Modal Deep Learning Based Approach for House Price Prediction

Md Hasebul Hasan, Md Abid Jahan, Mohammed Eunus Ali, Yuan-Fang Li, Timos Sellis

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 22 pages

Leveraging Contrastive Learning and Self-Training for Multimodal Emotion Recognition with Limited Labeled Samples

Qi Fan, Yutong Li, Yi Xin, Xinyu Cheng, Guanglai Gao, Miao Ma

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI、eess.AS

Comments Accepted by ACM MM Workshop 2024

Large Language Multimodal Models for 5-Year Chronic Disease Cohort Prediction Using EHR Data

Jun-En Ding, Phan Nguyen Minh Thao, Wen-Chih Peng, Jian-Zhe Wang, Chun-Cheng Chug, Min-Chen Hsieh, Yun-Chien Tseng, Ling Chen, Dongsheng Luo, Chi-Te Wang, Pei-fu Chen, Feng Liu, Fang-Ming Hung

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task

Ali Asgarov, Samir Rustamov

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models

Yeji Park, Deokyeong Lee, Junsuk Choe, Buru Chang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments First two authors contributed equally. Source code is available at https://github.com/yejipark-m/ConVis

Multimodal Contrastive In-Context Learning

Yosuke Miyanishi, Minh Le Nguyen

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

UniMEL: A Unified Framework for Multimodal Entity Linking with Large Language Models

Liu Qi, He Yongyi, Lian Defu, Zheng Zhi, Xu Tong, Liu Che, Chen Enhong

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments CIKM 2024. The first two authors contributed equally to this work

HiQuE: Hierarchical Question Embedding Network for Multimodal Depression Detection

Juho Jung, Chaewon Kang, Jeewoo Yoon, Seungbae Kim, Jinyoung Han

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments 11 pages, 6 figures, Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (CIKM '24)

Journal ref Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (CIKM '24), October 21-25, 2024, Boise, ID, USA

Contrastive Learning-based Multi Modal Architecture for Emoticon Prediction by Employing Image-Text Pairs

Ananya Pandey, Dinesh Kumar Vishwakarma

专题命中 跨模态检索 ：image-text(title)；multimodal(abstract)；分类 cs.CV、cs.AI

MMPKUBase: A Comprehensive and High-quality Chinese Multi-modal Knowledge Graph

Xuan Yi, Yanzeng Li, Lei Zou

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CL、cs.MM

Enhancing Conceptual Understanding in Multimodal Contrastive Learning through Hard Negative Samples

Philipp J. Rösch, Norbert Oswald, Michaela Geierhos, Jindřich Libovický

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Missing Modality Prediction for Unpaired Multimodal Learning via Joint Embedding of Unimodal Models

Donggeun Kim, Taesup Kim

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments ECCV 2024

Multimodal Reranking for Knowledge-Intensive Visual Question Answering

Haoyang Wen, Honglei Zhuang, Hamed Zamani, Alexander Hauptmann, Michael Bendersky

专题命中 跨模态检索 ：multimodal(title)；multi-modal(abstract)；分类 cs.CL、cs.AI

RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Models

M. Abdul Khaliq, P. Chang, M. Ma, B. Pflugfelder, F. Miletić

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages, submitted to ACL Rolling Review June 2024

Multimodal contrastive learning for spatial gene expression prediction using histology images

Wenwen Min, Zhiceng Shi, Jun Zhang, Jun Wan, Changmiao Wang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments BIB, Code: https://github.com/shizhiceng/mclSTExp

Transforming LLMs into Cross-modal and Cross-lingual Retrieval Systems

Frank Palma Gomez, Ramon Sanabria, Yun-hsuan Sung, Daniel Cer, Siddharth Dalmia, Gustavo Hernandez Abrego

专题命中 跨模态检索 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CL、eess.AS

Research on Optimization of Natural Language Processing Model Based on Multimodal Deep Learning

Dan Sun, Yaxin Liang, Yining Yang, Yuhan Ma, Qishi Zhan, Erdi Gao

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Don't Buy it! Reassessing the Ad Understanding Abilities of Contrastive Multimodal Models

A. Bavaresco, A. Testoni, R. Fernández

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to the main conference ACL 2024

RNG: Reducing Multi-level Noise and Multi-grained Semantic Gap for Joint Multimodal Aspect-Sentiment Analysis

Yaxin Liu, Yan Zhou, Ziming Li, Jinchuan Zhang, Yu Shang, Chenyang Zhang, Songlin Hu

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ICME 2024