多模态大模型

LInK: Learning Joint Representations of Design and Performance Spaces through Contrastive Learning for Mechanism Synthesis

Amin Heyrani Nobari, Akash Srivastava, Dan Gutfreund, Kai Xu, Faez Ahmed

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Journal ref Transactions on Machine Learning Research, 2835-885, 2024

Multi-Modality Transformer for E-Commerce: Inferring User Purchase Intention to Bridge the Query-Product Gap

Srivatsa Mallapragada, Ying Xie, Varsha Rani Chawan, Zeyad Hailat, Yuanbo Wang

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.AI

Comments Published in IEEE Big Data Conference 2024, Washington DC

Journal ref 2024 IEEE International Conference on Big Data (BigData)

StreamingRAG: Real-time Contextual Retrieval and Generation Framework

Murugan Sankaradas, Ravi K. Rajendran, Srimat T. Chakradhar

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted and Presented at AI4Sys, HPDC 2024

LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detection

Pengcheng Zhao, Zhixian He, Fuwei Zhang, Shujin Lin, Fan Zhou

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

PsyDI: Towards a Personalized and Progressively In-depth Chatbot for Psychological Measurements

Xueyan Li, Xinyan Chen, Yazhe Niu, Shuai Hu, Yu Liu

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.AI

Comments 29 pages, 15 figures

SenseRAG: Constructing Environmental Knowledge Bases with Proactive Querying for LLM-Based Autonomous Driving

Xuewen Luo, Fan Ding, Fengze Yang, Yang Zhou, Junnyong Loo, Hwa Hui Tew, Chenxi Liu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Comments This paper has been accepted for presentation at WACV Workshop LLMAD 2025

Forecasting Credit Ratings: A Case Study where Traditional Methods Outperform Generative LLMs

Felix Drinkall, Janet B. Pierrehumbert, Stefan Zohren

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL

HST-MRF: Heterogeneous Swin Transformer with Multi-Receptive Field for Medical Image Segmentation

Xiaofei Huang, Hongfang Gong, Jin Zhang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Journal ref IEEE Journal of Biomedical and Health Informatics, 2024

Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval

Yuanmin Tang, Xiaoting Qin, Jue Zhang, Jing Yu, Gaopeng Gou, Gang Xiong, Qingwei Ling, Saravan Rajmohan, Dongmei Zhang, Qi Wu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

ScreenMark: Watermarking Arbitrary Visual Content on Screen

Xiujian Liang, Gaozhi Liu, Yichao Si, Xiaoxiao Hu, Zhenxing Qian

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

UniLoc: Towards Universal Place Recognition Using Any Single Modality

Yan Xia, Zhendong Li, Yun-Jin Li, Letian Shi, Hu Cao, João F. Henriques, Daniel Cremers

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Comments 14 pages, 10 figures

Leveraging User-Generated Metadata of Online Videos for Cover Song Identification

Simon Hachmeier, Robert Jäschke

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.MM

Comments accepted for presentation at NLP for Music and Audio (NLP4MusA) 2024

Leveraging Retrieval-Augmented Tags for Large Vision-Language Understanding in Complex Scenes

Antonio Carlos Rivera, Anthony Moore, Steven Robinson

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

CATALOG: A Camera Trap Language-guided Contrastive Learning Model

Julian D. Santamaria, Claudia Isaza, Jhony H. Giraldo

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

Embeddings are all you need! Achieving High Performance Medical Image Classification through Training-Free Embedding Analysis

Raj Hansini Khoiwal, Alan B. McMillan

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments 15 pages, 7 figures, 3 tables

Exploring Masked Autoencoders for Sensor-Agnostic Image Retrieval in Remote Sensing

Jakob Hackstein, Gencer Sumbul, Kai Norman Clasen, Begüm Demir

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted at the IEEE Transactions on Geoscience and Remote Sensing. Our code is available at https://github.com/jakhac/CSMAE

Embedding and Enriching Explicit Semantics for Visible-Infrared Person Re-Identification

Neng Dong, Shuanglin Yan, Liyan Zhang, Jinhui Tang

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV

Accelerating Manufacturing Scale-Up from Material Discovery Using Agentic Web Navigation and Retrieval-Augmented AI for Process Engineering Schematics Design

Sakhinana Sagar Srinivas, Akash Das, Shivam Gupta, Venkataramana Runkana

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

SimC3D: A Simple Contrastive 3D Pretraining Framework Using RGB Images

Jiahua Dong, Tong Wu, Rui Qian, Jiaqi Wang

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

DAug: Diffusion-based Channel Augmentation for Radiology Image Retrieval and Classification

Ying Jin, Zhuoran Zhou, Haoquan Fang, Jenq-Neng Hwang

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV

Bridging Text and Image for Artist Style Transfer via Contrastive Learning

Zhi-Song Liu, Li-Wen Wang, Jun Xiao, Vicky Kalogeiton

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV

Comments 18 pages, 8 figures. arXiv admin note: substantial text overlap with arXiv:2202.13562

EchoSight: Advancing Visual-Language Models with Wiki Knowledge

Yibin Yan, Weidi Xie

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by EMNLP 2024 findings; Project Page: https://go2heart.github.io/echosight

Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-training

Haicheng Wang, Chen Ju, Weixiong Lin, Shuai Xiao, Mengting Chen, Yixuan Huang, Chang Liu, Mingshuai Yao, Jinsong Lan, Ying Chen, Qingwen Liu, Yanfeng Wang

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV

Self-Training Boosted Multi-Factor Matching Network for Composed Image Retrieval

Haokun Wen, Xuemeng Song, Jianhua Yin, Jianlong Wu, Weili Guan, Liqiang Nie

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.MM

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 3665-3678, May 2024

Leverage Task Context for Object Affordance Ranking

Haojie Huang, Hongchen Luo, Wei Zhai, Yang Cao, Zheng-Jun Zha

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Partial Scene Text Retrieval

Hao Wang, Minghui Liao, Zhouyi Xie, Wenyu Liu, Xiang Bai

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted on TPAMI

SCAN: Bootstrapping Contrastive Pre-training for Data Efficiency

Yangyang Guo, Mohan Kankanhalli

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV

Tumor Location-weighted MRI-Report Contrastive Learning: A Framework for Improving the Explainability of Pediatric Brain Tumor Diagnosis

Sara Ketabi, Matthias W. Wagner, Cynthia Hawkins, Uri Tabori, Birgit Betina Ertl-Wagner, Farzad Khalvati

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Language-guided Hierarchical Fine-grained Image Forgery Detection and Localization

Xiao Guo, Xiaohong Liu, Iacopo Masi, Xiaoming Liu

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted by IJCV2024. arXiv admin note: substantial text overlap with arXiv:2303.17111

AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoning

Shirley Wu, Shiyu Zhao, Qian Huang, Kexin Huang, Michihiro Yasunaga, Kaidi Cao, Vassilis N. Ioannidis, Karthik Subbian, Jure Leskovec, James Zou

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL

Comments NeurIPS 2024 main conference