多模态大模型

Enhancing Image Retrieval : A Comprehensive Study on Photo Search using the CLIP Mode

Naresh Kumar Lahajal, Harini S

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Are we describing the same sound? An analysis of word embedding spaces of expressive piano performance

Silvan David Peter, Shreyan Chowdhury, Carlos Eduardo Cancino-Chacón, Gerhard Widmer

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CL、cs.AI

Journal ref Proceedings of the Forum for Information Retrieval Evaluation, FIRE, 2023, Panjim, India

TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection

Hao Sun, Mingyao Zhou, Wenjing Chen, Wei Xie

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by AAAI-24

Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks

Wenhan Yang, Jingdong Gao, Baharan Mirzasoleiman

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Towards Generalized Multi-stage Clustering: Multi-view Self-distillation

Jiatai Wang, Zhiwei Xu, Xin Wang, Tao Li

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding

Yingjie Niu, Ming Ding, Keisuke Fujii, Kento Ohtani, Alexander Carballo, Kazuya Takeda

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

HKUST at SemEval-2023 Task 1: Visual Word Sense Disambiguation with Context Augmentation and Visual Assistance

Zhuohao Yin, Xin Huang

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection

Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, Xiu Li

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

3D-MIR: A Benchmark and Empirical Study on 3D Medical Image Retrieval in Radiology

Asma Ben Abacha, Alberto Santamaria-Pang, Ho Hin Lee, Jameson Merkow, Qin Cai, Surya Teja Devarakonda, Abdullah Islam, Julia Gong, Matthew P. Lungren, Thomas Lin, Noel C Codella, Ivan Tarapov

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Visualizing the Diversity of Representations Learned by Bayesian Neural Networks

Dennis Grinwald, Kirill Bykov, Shinichi Nakajima, Marina M. -C. Höhne

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 16 pages, 18 figures

Journal ref Published in Transactions on Machine Learning Research (11/2023)

(Un)likelihood Training for Interpretable Embedding

Jiaxin Wu, Chong-Wah Ngo, Wing-Kwong Chan, Zhijian Hou

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments accepted in ACM Transactions on Information Systems

Three Towers: Flexible Contrastive Learning with Pretrained Image Models

Jannik Kossen, Mark Collier, Basil Mustafa, Xiao Wang, Xiaohua Zhai, Lucas Beyer, Andreas Steiner, Jesse Berent, Rodolphe Jenatton, Efi Kokiopoulou

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication at NeurIPS 2023

Jaeger: A Concatenation-Based Multi-Transformer VQA Model

Jieting Long, Zewei Shi, Penghao Jiang, Yidong Gan

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments This paper is the technical research paper of CIKM 2023 DocIU challenges. The authors received the CIKM 2023 DocIU Winner Award, sponsored by Google, Microsoft, and the Centre for data-driven geoscience

Sentence-level Prompts Benefit Composed Image Retrieval

Yang Bai, Xinxing Xu, Yong Liu, Salman Khan, Fahad Khan, Wangmeng Zuo, Rick Siow Mong Goh, Chun-Mei Feng

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors

Paul S. Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Ethan Cohen, Aidan J. Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, Kenneth A. Norman, Tanishq Mathew Abraham

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Project Page at https://medarc.ai/mindeye. Code at https://github.com/MedARC-AI/fMRI-reconstruction-NSD/. Published as a conference paper at NeurIPS 2023

Contrastive Latent Space Reconstruction Learning for Audio-Text Retrieval

Kaiyi Luo, Xulong Zhang, Jianzong Wang, Huaxiong Li, Ning Cheng, Jing Xiao

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments Accepted by The 35th IEEE International Conference on Tools with Artificial Intelligence. (ICTAI 2023)

Knowledge Graph Embeddings for Multi-Lingual Structured Representations of Radiology Reports

Tom van Sonsbeek, Xiantong Zhen, Marcel Worring

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CL、cs.AI

FArMARe: a Furniture-Aware Multi-task methodology for Recommending Apartments based on the user interests

Ali Abdari, Alex Falcon, Giuseppe Serra

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments accepted for presentation at the ICCV2023 CV4Metaverse workshop

An Empirical Analysis for Zero-Shot Multi-Label Classification on COVID-19 CT Scans and Uncurated Reports

Ethan Dack, Lorenzo Brigato, Matthew McMurray, Matthias Fontanellaz, Thomas Frauenfelder, Hanno Hoppe, Aristomenis Exadaktylos, Thomas Geiser, Manuela Funke-Chambour, Andreas Christe, Lukas Ebner, Stavroula Mougiakakou

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops 2023

Contrastive Feature Masking Open-Vocabulary Vision Transformer

Dahun Kim, Anelia Angelova, Weicheng Kuo

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICCV 2023

TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis

Mathis Petrovich, Michael J. Black, Gül Varol

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments ICCV 2023 Camera Ready, project page: https://mathis.petrovich.fr/tmr/

Fairness in Image Search: A Study of Occupational Stereotyping in Image Retrieval and its Debiasing

Swagatika Dash

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 20 Pages, Work uses Proprietary Search Systems from the year 2021

Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark

Shuyu Yang, Yinan Zhou, Yaxiong Wang, Yujiao Wu, Li Zhu, Zhedong Zheng

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.MM

Adapt and Align to Improve Zero-Shot Sketch-Based Image Retrieval

Shiyin Dong, Mingrui Zhu, Nannan Wang, Xinbo Gao

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 7 figures, 6 tables

ScribbleVC: Scribble-supervised Medical Image Segmentation with Vision-Class Embedding

Zihan Li, Yuan Zheng, Xiangde Luo, Dandan Shan, Qingqi Hong

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ACM MM 2023, project page: https://github.com/HUANGLIZI/ScribbleVC

Fine-Tuned but Zero-Shot 3D Shape Sketch View Similarity and Retrieval

Gianluca Berardi, Yulia Gryaditskaya

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Substance or Style: What Does Your Image Embedding Know?

Cyrus Rashtchian, Charles Herrmann, Chun-Sung Ferng, Ayan Chakrabarti, Dilip Krishnan, Deqing Sun, Da-Cheng Juan, Andrew Tomkins

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 27 pages, 9 figures

Sticker820K: Empowering Interactive Retrieval with Stickers

Sijie Zhao, Yixiao Ge, Zhongang Qi, Lin Song, Xiaohan Ding, Zehua Xie, Ying Shan

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

ConaCLIP: Exploring Distillation of Fully-Connected Knowledge Interaction Graph for Lightweight Text-Image Retrieval

Jiapeng Wang, Chengyu Wang, Xiaodan Wang, Jun Huang, Lianwen Jin

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments ACL 2023 Industry Track

MALM: Mask Augmentation based Local Matching for Food-Recipe Retrieval

Bhanu Prakash Voutharoja, Peng Wang, Lei Wang, Vivienne Guan

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.MM

Comments Under review. Link to the dataset repo - https://github.com/torralba-lab/im2recipe-Pytorch#recipe1m-dataset