多模态大模型

VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos

Xubin Ren, Lingrui Xu, Long Xia, Shuaiqiang Wang, Dawei Yin, Chao Huang

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Mitigating Hallucinations in Large Vision-Language Models with Internal Fact-based Contrastive Decoding

Chao Wang, Xuancheng Zhou, Weiwei Fu, Yang Zhou

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Dissecting Representation Misalignment in Contrastive Learning via Influence Function

Lijie Hu, Chenyang Ren, Huanyi Xie, Khouloud Saadi, Shu Yang, Zhen Tan, Jingfeng Zhang, Di Wang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 33 pages

A Training-free Sub-quadratic Cost Transformer Model Serving Framework With Hierarchically Pruned Attention

Heejun Lee, Geon Park, Youngwan Lee, Jaduk Suh, Jina Kim, Wonyoung Jeong, Bumsik Kim, Hyemin Lee, Myeongjae Jeon, Sung Ju Hwang

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 44 pages

Enhancing textual textbook question answering with large language models and retrieval augmented generation

Hessa Abdulrahman Alawwad, Areej Alhothali, Usman Naseem, Ali Alkhathlan, Amani Jamal

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Journal ref Pattern Recognition, Volume 162, 2025, Article 111332

FOCUS: Towards Universal Foreground Segmentation

Zuyao You, Lingyu Kong, Lingchen Meng, Zuxuan Wu

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Hyperbolic Contrastive Learning for Hierarchical 3D Point Cloud Embedding

Yingjie Liu, Pengyu Zhang, Ziyao He, Mingsong Chen, Xuan Tang, Xian Wei

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio Captioning

Xiquan Li, Wenxi Chen, Ziyang Ma, Xuenan Xu, Yuzhe Liang, Zhisheng Zheng, Qiuqiang Kong, Xie Chen

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.AI、eess.AS

Semi-Supervised Contrastive Learning for Controllable Video-to-Music Retrieval

Shanti Stewart, Gouthaman KV, Lie Lu, Andrea Fanelli

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments Accepted at ICASSP 2025

Advanced ingestion process powered by LLM parsing for RAG system

Arnau Perez, Xavier Vizcaino

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 3 figures

I0T: Embedding Standardization Method Towards Zero Modality Gap

Na Min An, Eunki Kim, James Thorne, Hyunjung Shim

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 16 figures, 8 figures, 7 tables

Rebalanced Vision-Language Retrieval Considering Structure-Aware Distillation

Yang Yang, Wenjuan Xi, Luping Zhou, Jinhui Tang

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Wills Aligner: Multi-Subject Collaborative Brain Visual Decoding

Guangyin Bao, Qi Zhang, Zixuan Gong, Jialei Zhou, Wei Fan, Kun Yi, Usman Naseem, Liang Hu, Duoqian Miao

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments AAAI 2025, 16 pages

A dual contrastive framework

Yuan Sun, Zhao Zhang, Jorge Ortiz

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

WordVIS: A Color Worth A Thousand Words

Umar Khan, Saifullah, Stefan Agne, Andreas Dengel, Sheraz Ahmed

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Dynamic Modality-Camera Invariant Clustering for Unsupervised Visible-Infrared Person Re-identification

Yiming Yang, Weipeng Hu, Haifeng Hu

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Emotion-Aligned Contrastive Learning Between Images and Music

Shanti Stewart, Kleanthis Avramidis, Tiantian Feng, Shrikanth Narayanan

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments Published at ICASSP 2024. Code: https://github.com/shantistewart/Emo-CLIM

A Framework For Image Synthesis Using Supervised Contrastive Learning

Yibin Liu, Jianyu Zhang, Li Zhang, Shijian Li, Gang Pan

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Relation-Aware Meta-Learning for Zero-shot Sketch-Based Image Retrieval

Yang Liu, Jiale Du, Xinbo Gao, Jungong Han

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval

Zijun Min, Bingshuai Liu, Liang Zhang, Jia Song, Jinsong Su, Song He, Xiaochen Bo

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CL、cs.AI

Comments BIBM 2024 Regular Paper

Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversion

Matteo Ferrante, Tommaso Boccato, Grigorii Rashkov, Nicola Toschi

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

CleanerCLIP: Fine-grained Counterfactual Semantic Augmentation for Backdoor Defense in Contrastive Learning

Yuan Xun, Siyuan Liang, Xiaojun Jia, Xinwei Liu, Xiaochun Cao

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Pseudo-triplet Guided Few-shot Composed Image Retrieval

Bohan Hou, Haoqiang Lin, Haokun Wen, Meng Liu, Mingzhu Xu, Xuemeng Song

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 10pages

Plug, Play, and Fuse: Zero-Shot Joint Decoding via Word-Level Re-ranking Across Diverse Vocabularies

Sai Koneru, Matthias Huck, Miriam Exel, Jan Niehues

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments WMT 2024

BiVLC: Extending Vision-Language Compositionality Evaluation with Text-to-Image Retrieval

Imanol Miranda, Ander Salaberria, Eneko Agirre, Gorka Azkune

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to NeurIPS 24 Datasets and Benchmarks Track; Project page at: https://imirandam.github.io/BiVLC_project_page/

G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models

Pengyue Jia, Yiding Liu, Xiaopeng Li, Yuhao Wang, Yantong Du, Xiao Han, Xuetao Wei, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS2024

Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learning

Jingbiao Mei, Jinghong Chen, Weizhe Lin, Bill Byrne, Marcus Tomalin

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments ACL 2024 Main. The code is available from: https://github.com/JingbiaoMei/RGCL

Multi-path Exploration and Feedback Adjustment for Text-to-Image Person Retrieval

Bin Kang, Bin Chen, Junjie Wang, Yong Xu

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Eliminating the Language Bias for Visual Question Answering with fine-grained Causal Intervention

Ying Liu, Ge Bai, Chenji Lu, Shilong Li, Zhang Zhang, Ruifang Liu, Wenbin Guo

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Journal ref 2024 IEEE International Conference on Multimedia and Expo (ICME), Niagara Falls, ON, Canada, 2024, pp. 1-6

VERIFIED: A Video Corpus Moment Retrieval Benchmark for Fine-Grained Video Understanding

Houlun Chen, Xin Wang, Hong Chen, Zeyang Zhang, Wei Feng, Bin Huang, Jia Jia, Wenwu Zhu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by 38th NeurIPS Datasets & Benchmarks Track (NeurIPS 2024)