多模态大模型

Joint Moment Retrieval and Highlight Detection Via Natural Language Queries

Richard Luo, Austin Peng, Heidi Yap, Koby Beard

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL

MEDIMP: 3D Medical Images with clinical Prompts from limited tabular data for renal transplantation

Leo Milecki, Vicky Kalogeiton, Sylvain Bodard, Dany Anglicheau, Jean-Michel Correas, Marc-Olivier Timsit, Maria Vakalopoulou

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answering

Alireza Salemi, Juan Altmayer Pizzorno, Hamed Zamani

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL

CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment

Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2023 (Highlight paper, 2.5% acceptance rate); Open source

VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners

Shen Yan, Tao Zhu, Zirui Wang, Yuan Cao, Mi Zhang, Soham Ghosh, Yonghui Wu, Jiahui Yu

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.MM

Comments Tech report. arXiv v3: update text

TQ-Net: Mixed Contrastive Representation Learning For Heterogeneous Test Questions

He Zhu, Xihua Li, Xuemin Zhao, Yunbo Cao, Shan Yu

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments This paper has been accepted for the AAAI2023 AI4Edu Workshop

Unified Vision-Language Representation Modeling for E-Commerce Same-Style Products Retrieval

Ben Chen, Linbo Jin, Xinxin Wang, Dehong Gao, Wen Jiang, Wei Ning

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.AI、cs.MM

Comments Accepted in The Web Conference (WWW2023) Industry Track. 5 pages, 3 figures

You Need to Read Again: Multi-granularity Perception Network for Moment Retrieval in Videos

Xin Sun, Xuan Wang, Jialin Gao, Qiong Liu, Xi Zhou

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments in SIGIR 2022

HateProof: Are Hateful Meme Detection Systems really Robust?

Piush Aggarwal, Pranit Chawla, Mithun Das, Punyajoy Saha, Binny Mathew, Torsten Zesch, Animesh Mukherjee

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Accepted at TheWebConf'2023 (WWW'2023)

LaT: Latent Translation with Cycle-Consistency for Video-Text Retrieval

Jinbin Bai, Chunhui Liu, Feiyue Ni, Haofan Wang, Mengying Hu, Xiaofeng Guo, Lele Cheng

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Multi-queue Momentum Contrast for Microvideo-Product Retrieval

Yali Du, Yinwei Wei, Wei Ji, Fan Liu, Xin Luo, Liqiang Nie

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining (WSDM '23), February 27-March 3, 2023, Singapore, Singapore

Image Retrieval from Contextual Descriptions

Benno Krojer, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, Siva Reddy

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments accepted to ACL 2022

Face-to-Face Contrastive Learning for Social Intelligence Question-Answering

Alex Wilf, Martin Q. Ma, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

COFAR: Commonsense and Factual Reasoning in Image Search

Prajwal Gatti, Abhirama Subramanyam Penamakuri, Revant Teotia, Anand Mishra, Shubhashis Sengupta, Roshni Ramnani

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted in AACL-IJCNLP 2022

Matching Text and Audio Embeddings: Exploring Transfer-learning Strategies for Language-based Audio Retrieval

Benno Weck, Miguel Pérez Fernández, Holger Kirchhoff, Xavier Serra

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CL、eess.AS

Comments 5 pages, 2 figures. Accepted at Detection and Classification of Acoustic Scenes and Events 2022 (DCASE2022)

CMSBERT-CLR: Context-driven Modality Shifting BERT with Contrastive Learning for linguistic, visual, acoustic Representations

Junghun Kim, Jihie Kim

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments Accepted by IJCNN 2022

CAIBC: Capturing All-round Information Beyond Color for Text-based Person Retrieval

Zijie Wang, Aichun Zhu, Jingyi Xue, Xili Wan, Chao Liu, Tian Wang, Yifeng Li

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted on ACM MM '22

Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-training

Haoxuan You, Luowei Zhou, Bin Xiao, Noel Codella, Yu Cheng, Ruochen Xu, Shih-Fu Chang, Lu Yuan

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ECCV 2022, 22 pages, 4 figures

A Feature Memory Rearrangement Network for Visual Inspection of Textured Surface Defects Toward Edge Intelligent Manufacturing

Haiming Yao, Wenyong Yu, Xue Wang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Revision to IEEE transactions on automation science and engineering

LiT: Zero-Shot Transfer with Locked-image text Tuning

Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, Lucas Beyer

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Xiaohua, Xiao, Basil, Andreas and Lucas contributed equally; CVPR 2022

PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Map

Chenfeng Xu, Tian Li, Chen Tang, Lingfeng Sun, Kurt Keutzer, Masayoshi Tomizuka, Alireza Fathi, Wei Zhan

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments The code is available at https://github.com/chenfengxu714/PreTraM.git

Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation

Jogendra Nath Kundu, Siddharth Seth, Anirudh Jamkhandi, Pradyumna YM, Varun Jampani, Anirban Chakraborty, R. Venkatesh Babu

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2021. Project page: https://sites.google.com/view/sa3dhp

SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddings

Rasmus Laurvig Haugaard, Anders Glent Buch

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Neural Machine Translation with Phrase-Level Universal Visual Representations

Qingkai Fang, Yang Feng

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments ACL 2022 main conference

A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering

Feng Gao, Qing Ping, Govind Thattai, Aishwarya Reganti, Ying Nian Wu, Prem Natarajan

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL

The Curious Layperson: Fine-Grained Image Recognition without Expert Labels

Subhabrata Choudhury, Iro Laina, Christian Rupprecht, Andrea Vedaldi

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments To appear in BMVC 2021 (Oral). Project page: https://www.robots.ox.ac.uk/~vgg/research/clever/

Simple Dialogue System with AUDITED

Yusuf Tas, Piotr Koniusz

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by the BMVC 2021

Two-stage Visual Cues Enhancement Network for Referring Image Segmentation

Yang Jiao, Zequn Jie, Weixin Luo, Jingjing Chen, Yu-Gang Jiang, Xiaolin Wei, Lin Ma

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ACM MM 2021

CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

Zhijian Hou, Chong-Wah Ngo, Wing Kwong Chan

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.AI、cs.MM

Comments 10 pages, 4 figures, 2021 MultiMedia, code: https://github.com/houzhijian/CONQUER

Dynamic Modeling of Hand-Object Interactions via Tactile Sensing

Qiang Zhang, Yunzhu Li, Yiyue Luo, Wan Shou, Michael Foshey, Junchi Yan, Joshua B. Tenenbaum, Wojciech Matusik, Antonio Torralba

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments IROS 2021. First two authors contributed equally. Project page: http://phystouch.csail.mit.edu/