多模态大模型

CANDLE: A Cross-Modal Agentic Knowledge Distillation Framework for Interpretable Sarcopenia Diagnosis

Yuqi Jin, Zhenhao Shuai, Zihan Hu, Weiteng Zhang, Weihao Xie, Jianwei Shuai, Xian Shen, Zhen Feng

专题命中 跨模态检索 ：cross-modal(title)；分类 cs.AI

Comments 11 pages, 4 figures, 5 tables

NaviSense: A Multimodal Assistive Mobile application for Object Retrieval by Persons with Visual Impairment

Ajay Narayanan Sridhar, Fuli Qiao, Nelson Daniel Troncoso Aldas, Yanpei Shi, Mehrdad Mahdavi, Laurent Itti, Vijaykrishnan Narayanan

机构 * The Pennsylvania State University（宾夕法尼亚州立大学） ； Independent Researcher（独立研究者） ； University of Southern California（南加州大学） 纠错

专题命中 跨模态检索 ：multimodal(title)；分类 cs.AI

Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMs

Andrew Yeo, Daeseon Choi

机构 * Ranchview High School（拉文斯维尔高中） ； Soongsil University（松山大学） 纠错

专题命中 跨模态检索 ：multimodal(title)；分类 cs.AI

Comments 8 pages, 4 figures, 2 tables

SHREC 2025: Retrieval of Optimal Objects for Multi-modal Enhanced Language and Spatial Assistance (ROOMELSA)

Trong-Thuan Nguyen, Viet-Tham Huynh, Quang-Thuc Nguyen, Hoang-Phuc Nguyen, Long Le Bao, Thai Hoang Minh, Minh Nguyen Anh, Thang Nguyen Tien, Phat Nguyen Thuan, Huy Nguyen Phong, Bao Huynh Thai, Vinh-Tiep Nguyen, Duc-Vu Nguyen, Phu-Hoa Pham, Minh-Huy Le-Hoang, Nguyen-Khang Le, Minh-Chinh Nguyen, Minh-Quan Ho, Ngoc-Long Tran, Hien-Long Le-Hoang, Man-Khoi Tran, Anh-Duong Tran, Kim Nguyen, Quan Nguyen Hung, Dat Phan Thanh, Hoang Tran Van, Tien Huynh Viet, Nhan Nguyen Viet Thien, Dinh-Khoi Vo, Van-Loc Nguyen, Trung-Nghia Le, Tam V. Nguyen, Minh-Triet Tran

机构 * University of Science, VNU-HCM（越南胡志明市科学大学） ； University of Information Technology, VNU-HCM（越南胡志明市信息技术大学） ； Vietnam National University（越南国家大学） ； University of Dayton（戴维森大学） 纠错

专题命中 跨模态检索 ：multi-modal(title)；分类 cs.CV

Privileged Contrastive Pretraining for Multimodal Affect Modelling

Kosmas Pinitas, Konstantinos Makantasis, Georgios N. Yannakakis

机构 * Institute of Digital Games University of Malta（数字游戏研究所马耳他大学） ； Department of Artificial Intelligence University of Malta（人工智能系马耳他大学） 纠错

专题命中 跨模态检索 ：multimodal(title)；分类 cs.MM

Multimodal-to-Text Prompt Engineering in Large Language Models Using Feature Embeddings for GNSS Interference Characterization

Harshith Manjunath, Lucas Heublein, Tobias Feigl, Felix Ott

机构 * Fraunhofer Institute for Integrated Circuits IIS（弗劳恩霍夫集成电路研究所） 纠错

专题命中 跨模态检索 ：multimodal(title)；分类 cs.AI

Journal ref IEEE Wireless Communications and Networking Conference (WCNC), March 2025, Milan, Italy

Self-Supervised Generative-Contrastive Learning of Multi-Modal Euclidean Input for 3D Shape Latent Representations: A Dynamic Switching Approach

Chengzhi Wu, Julius Pfrommer, Mingyuan Zhou, Jürgen Beyerer

机构 * Institute for Anthropomatics and Robotics（人机化与机器人研究所） ； Karlsruhe Institute of Technology（卡尔斯鲁厄理工大学） ； Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB（弗劳恩霍夫光学、系统技术与图像利用研究所） 纠错

专题命中 跨模态检索 ：multi-modal(title)；分类 cs.CV

sEEG-based Encoding for Sentence Retrieval: A Contrastive Learning Approach to Brain-Language Alignment

Yijun Liu

专题命中 跨模态检索 ：multimodal(abstract,comments)；multimodal foundation model(abstract,comments)；分类 cs.CL

Comments Accepted for poster presentation at the CVPR 2025 Workshop on Multimodal Foundation Models (MMFM3)

GT2Vec: Large Language Models as Multi-Modal Encoders for Text and Graph-Structured Data

Jiacheng Lin, Kun Qian, Haoyu Han, Nurendra Choudhary, Tianxin Wei, Zhongruo Wang, Sahika Genc, Edward W Huang, Sheng Wang, Karthik Subbian, Danai Koutra, Jimeng Sun

专题命中 跨模态检索 ：multi-modal(title)；分类 cs.CL

Vision-Driven Prompt Optimization for Large Language Models in Multimodal Generative Tasks

Leo Franklin, Apiradee Boonmee, Kritsada Wongsuwan

专题命中 跨模态检索 ：multimodal(title)；分类 cs.CV

Improving Text-based Person Search via Part-level Cross-modal Correspondence

Jicheol Park, Boseung Jeong, Dongwon Kim, Suha Kwak

专题命中 跨模态检索 ：cross-modal(title)；分类 cs.CV

CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning

Yiping Wang, Yifang Chen, Wendan Yan, Alex Fang, Wenjing Zhou, Kevin Jamieson, Simon Shaolei Du

专题命中 跨模态检索 ：multimodal(title)；分类 cs.CV

Comments This paper supercedes our previous VAS paper (arXiv:2402.02055). It's accepted by NeurIPS2024 as spotlight paper. DataComp benchmark: https://www.datacomp.ai/dcclip/leaderboard.html

DRUM: Learning Demonstration Retriever for Large MUlti-modal Models

Ellen Yi-Ge, Jiechao Gao, Wei Han, Wei Zhu

专题命中 跨模态检索 ：multi-modal(title)；分类 cs.CL

Mind the Gap: A Generalized Approach for Cross-Modal Embedding Alignment

Arihan Yadav, Alan McMillan

专题命中 跨模态检索 ：cross-modal(title)；分类 cs.CL

Comments 18 pages, 3 figures

Cross-Modal Self-Supervised Learning with Effective Contrastive Units for LiDAR Point Clouds

Mu Cai, Chenxu Luo, Yong Jae Lee, Xiaodong Yang

专题命中 跨模态检索 ：cross-modal(title)；分类 cs.CV

Comments IROS 2024

Enhancing Social Media Personalization: Dynamic User Profile Embeddings and Multimodal Contextual Analysis Using Transformer Models

Pranav Vachharajani

专题命中 跨模态检索 ：multimodal(title)；分类 cs.AI

Comments 21 pages, 13 figures. Mentor: Prof Pritam Ranjan

Aligning Programming Language and Natural Language: Exploring Design Choices in Multi-Modal Transformer-Based Embedding for Bug Localization

Partha Chakraborty, Venkatraman Arumugam, Meiyappan Nagappan

专题命中 跨模态检索 ：multi-modal(title)；分类 cs.AI

Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training

Vedant Dave, Fotios Lygerakis, Elmar Rueckert

专题命中 跨模态检索 ：multimodal(title)；分类 cs.AI

Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing

Qihua Chen, Xuejin Chen, Chenxuan Wang, Yixiong Liu, Zhiwei Xiong, Feng Wu

专题命中 跨模态检索 ：multimodal(title)；分类 cs.CV

Comments 9 pages, 6 figures, AAAI 2024 accepted

On Task-personalized Multimodal Few-shot Learning for Visually-rich Document Entity Retrieval

Jiayi Chen, Hanjun Dai, Bo Dai, Aidong Zhang, Wei Wei

专题命中 跨模态检索 ：multimodal(title)；分类 cs.AI

Comments Paper published at Findings of the Association for Computational Linguistics: EMNLP, 2023

Automated Cardiovascular Record Retrieval by Multimodal Learning between Electrocardiogram and Clinical Report

Jielin Qiu, Jiacheng Zhu, Shiqi Liu, William Han, Jingqi Zhang, Chaojing Duan, Michael Rosenberg, Emerson Liu, Douglas Weber, Ding Zhao

专题命中 跨模态检索 ：multimodal(title)；分类 cs.CV

Comments Accepted to the ML4H 2023 Proceedings track

RGB-T Tracking via Multi-Modal Mutual Prompt Learning

Yang Luo, Xiqing Guo, Hui Feng, Lei Ao

专题命中 跨模态检索 ：multi-modal(title)；分类 cs.CV

Comments 9 pages, 5 figures, 5 tables

Towards multi-modal anatomical landmark detection for ultrasound-guided brain tumor resection with contrastive learning

Soorena Salari, Amirhossein Rasoulian, Hassan Rivaz, Yiming Xiao

专题命中 跨模态检索 ：multi-modal(title)；分类 cs.CV

Comments Accepted in MICCAI 2023

Towards Zero-shot Relation Extraction in Web Mining: A Multimodal Approach with Relative XML Path

Zilong Wang, Jingbo Shang

专题命中 跨模态检索 ：multimodal(title)；分类 cs.CL

(LC)$^2$: LiDAR-Camera Loop Constraints For Cross-Modal Place Recognition

Alex Junho Lee, Seungwon Song, Hyungtae Lim, Woojoo Lee, Hyun Myung

专题命中 跨模态检索 ：cross-modal(title)；分类 cs.AI

Comments 8 pages, 11 figures, Accepted to IEEE Robotics and Automation Letters (RA-L)

DDMM-Synth: A Denoising Diffusion Model for Cross-modal Medical Image Synthesis with Sparse-view Measurement Embedding

Xiaoyue Li, Kai Shang, Gaoang Wang, Mark D. Butala

专题命中 跨模态检索 ：cross-modal(title)；分类 cs.CV

Comments llncs.cls v2.20,12 pages with 6 figures

Human in the loop approaches in multi-modal conversational task guidance system development

Ramesh Manuvinakurike, Sovan Biswas, Giuseppe Raffa, Richard Beckwith, Anthony Rhodes, Meng Shi, Gesem Gudino Mejia, Saurav Sahay, Lama Nachman

专题命中 跨模态检索 ：multi-modal(title)；分类 cs.CL

Comments SCAI @ SIGIR

Learning to embed semantic similarity for joint image-text retrieval

Noam Malali, Yosi Keller

专题命中 跨模态检索 ：image-text(title)；分类 cs.CV

Comments in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

Multimodal contrastive learning for remote sensing tasks

Umangi Jain, Alex Wilson, Varun Gulshan

专题命中 跨模态检索 ：multimodal(title)；分类 cs.CV

COEM: Cross-Modal Embedding for MetaCell Identification

Haiyi Mao, Minxue Jia, Jason Xiaotian Dou, Haotian Zhang, Panayiotis V. Benos

专题命中 跨模态检索 ：cross-modal(title)；分类 cs.AI

Comments 5 pages, 2 figures, ICML workshop on computational biology