多模态大模型

Transformers in Vision: A Survey

Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, Mubarak Shah

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 30 pages (Accepted in ACM Computing Surveys December 2021)

Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding

Zhenzhi Wang, Limin Wang, Tao Wu, Tianhao Li, Gangshan Wu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments AAAI 2022 Camera Ready Version

Relation-aware Video Reading Comprehension for Temporal Language Grounding

Jialin Gao, Xin Sun, Mengmeng Xu, Xi Zhou, Bernard Ghanem

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by EMNLP-21

Fully Automatic Page Turning on Real Scores

Florian Henkel, Stephanie Schwaiger, Gerhard Widmer

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、eess.AS

Comments ISMIR 2021 Late Breaking/Demo

Towards Debiasing Temporal Sentence Grounding in Video

Hao Zhang, Aixin Sun, Wei Jing, Joey Tianyi Zhou

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 13 pages, 6 figures, 11 tables

Will You Ever Become Popular? Learning to Predict Virality of Dance Clips

Jiahao Wang, Yunhong Wang, Nina Weng, Tianrui Chai, Annan Li, Faxi Zhang, Sansi Yu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by TOMM

Visual Keyword Spotting with Attention

K R Prajwal, Liliane Momeni, Triantafyllos Afouras, Andrew Zisserman

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Appears in: British Machine Vision Conference 2021 (BMVC 2021)

GraphTMT: Unsupervised Graph-based Topic Modeling from Video Transcripts

Lukas Stappen, Jason Thies, Gerhard Hagerer, Björn W. Schuller, Georg Groh

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.MM

Comments JT and LS contributed equally to this work

AttendAffectNet: Self-Attention based Networks for Predicting Affective Responses from Movies

Ha Thi Phuong Thao, Balamurali B. T., Dorien Herremans, Gemma Roig

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments 8 pages, 6 figures

Journal ref Proceedings of the International Conference on Pattern Recognition (ICPR2020)

A CLIP-Enhanced Method for Video-Language Understanding

Guohao Li, Feng He, Zhifan Feng

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments 3 pages, 1 figure, 2 tables. Technical report

Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble

Songyao Jiang, Bin Sun, Lichen Wang, Yue Bai, Kunpeng Li, Yun Fu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Grounding Spatio-Temporal Language with Transformers

Tristan Karch, Laetitia Teodorescu, Katja Hofmann, Clément Moulin-Frier, Pierre-Yves Oudeyer

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Contains main article and supplementaries

Journal ref Neurips 2021

Contrastive Video-Language Segmentation

Chen Liang, Yawei Luo, Yu Wu, Yi Yang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Joint Multimedia Event Extraction from Video and Article

Brian Chen, Xudong Lin, Christopher Thomas, Manling Li, Shoya Yoshida, Lovish Chum, Heng Ji, Shih-Fu Chang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments To be presented at EMNLP 2021 findings

Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding

Daizong Liu, Xiaoye Qu, Pan Zhou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted as a long paper in the main conference of EMNLP 2021

WhyAct: Identifying Action Reasons in Lifestyle Vlogs

Oana Ignat, Santiago Castro, Hanwen Miao, Weiji Li, Rada Mihalcea

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at EMNLP 2021

SlowFast Rolling-Unrolling LSTMs for Action Anticipation in Egocentric Videos

Nada Osman, Guglielmo Camporese, Pasquale Coscia, Lamberto Ballan

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to EPIC@ICCV 2021

SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environments

Muhammad Zubair Irshad, Niluthpol Chowdhury Mithun, Zachary Seymour, Han-Pang Chiu, Supun Samarasekera, Rakesh Kumar

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 10 pages, 4 figures

Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition

Ning Wang, Guangming Zhu, Liang Zhang, Peiyi Shen, Hongsheng Li, Cong Hua

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments ACM MM Oral paper

Mounting Video Metadata on Transformer-based Language Model for Open-ended Video Question Answering

Donggeon Lee, Seongho Choi, Youwon Jang, Byoung-Tak Zhang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 1 figure

Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading

Shahd Elashmawy, Marian Ramsis, Hesham M. Eraqi, Farah Eldeshnawy, Hadeel Mabrouk, Omar Abugabal, Nourhan Sakr

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.AI

Hybrid Reasoning Network for Video-based Commonsense Captioning

Weijiang Yu, Jian Liang, Lei Ji, Lu Li, Yuejian Fang, Nong Xiao, Nan Duan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 11 pages, 6 figures

Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery

Yonghao Long, Jie Ying Wu, Bo Lu, Yueming Jin, Mathias Unberath, Yun-Hui Liu, Pheng Ann Heng, Qi Dou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted for ICRA 2021

Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering

Ahjeong Seo, Gi-Cheon Kang, Joonhan Park, Byoung-Tak Zhang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments ACL 2021

PyKale: Knowledge-Aware Machine Learning from Multiple Sources in Python

Haiping Lu, Xianyuan Liu, Robert Turner, Peizhen Bai, Raivo E Koot, Shuo Zhou, Mustafa Chasmai, Lawrence Schobs

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments This library is available at https://github.com/pykale/pykale

Relation-aware Hierarchical Attention Framework for Video Question Answering

Fangtao Li, Ting Bai, Chenyu Cao, Zihe Liu, Chenghao Yan, Bin Wu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, This paper is accepted by ICMR 2021

Video-aided Unsupervised Grammar Induction

Songyang Zhang, Linfeng Song, Lifeng Jin, Kun Xu, Dong Yu, Jiebo Luo

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments This paper is accepted by NAACL'21

Towards Natural Language Question Answering over Earth Observation Linked Data using Attention-based Neural Machine Translation

Abhishek V. Potnis, Rajat C. Shinde, Surya S. Durbha

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Accepted at IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2020

Dual Encoding for Video Retrieval by Text

Jianfeng Dong, Xirong Li, Chaoxi Xu, Xun Yang, Gang Yang, Xun Wang, Meng Wang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence. Code and data will be available at https://github.com/danieljf24/hybrid_space. Conference version: arXiv:1809.06181

Rethinking movie genre classification with fine-grained semantic clustering

Edward Fish, Jon Weinbren, Andrew Gilbert

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM