多模态大模型

Temporal Perceiving Video-Language Pre-training

Fan Ma, Xiaojie Jin, Heng Wang, Jingjia Huang, Linchao Zhu, Jiashi Feng, Yi Yang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Inferring latent neural sources via deep transcoding of simultaneously acquired EEG and fMRI

Xueqing Liu, Tao Tu, Paul Sajda

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2022): Workshop and Shared Task Report

Ali Hürriyetoğlu, Hristo Tanev, Vanni Zavarella, Reyyan Yeniterzi, Osman Mutlu, Erdem Yörük

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments to appear at CASE 2022 @ EMNLP 2022

Holistic Interaction Transformer Network for Action Detection

Gueter Josmy Faure, Min-Hung Chen, Shang-Hong Lai

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted for WACV 2023. Code: https://github.com/joslefaure/HIT

Flamingo: a Visual Language Model for Few-Shot Learning

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, Karen Simonyan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 54 pages. In Proceedings of Neural Information Processing Systems (NeurIPS) 2022

OWL (Observe, Watch, Listen): Audiovisual Temporal Context for Localizing Actions in Egocentric Videos

Merey Ramazanova, Victor Escorcia, Fabian Caba Heilbron, Chen Zhao, Bernard Ghanem

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、eess.AS

VTC: Improving Video-Text Retrieval with User Comments

Laura Hanu, James Thewlis, Yuki M. Asano, Christian Rupprecht

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted paper at the European Conference on Computer Vision (ECCV) 2022

STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognition

Dasom Ahn, Sangwon Kim, Hyunsu Hong, Byoung Chul Ko

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by WACV 2023

RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval

Xing Wu, Chaochen Gao, Zijia Lin, Zhongyuan Wang, Jizhong Han, Songlin Hu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2022

Behavior Transformers: Cloning $k$ modes with one stone

Nur Muhammad Mahi Shafiullah, Zichen Jeff Cui, Ariuntuya Altanzaya, Lerrel Pinto

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Code and data available at https://github.com/notmahi/bet

Zero-Shot Video Question Answering via Frozen Bidirectional Language Models

Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, Cordelia Schmid

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments NeurIPS 2022 Camera-Ready; Project Webpage: https://antoyang.github.io/frozenbilm.html; 25 pages; 5 figures

VLSNR:Vision-Linguistics Coordination Time Sequence-aware News Recommendation

Songhao Han, Wei Huang, Xiaotian Luan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.AI、cs.MM

Comments 10 pages, 5 figures

Semantic Segmentation of Vegetation in Remote Sensing Imagery Using Deep Learning

Alexandru Munteanu, Marian Neagul

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Masters thesis presented in 2021 at the West University of Timisoara, faculty of Mathematics and Computer Science

Text-Adaptive Multiple Visual Prototype Matching for Video-Text Retrieval

Chengzhi Lin, Ancong Wu, Junwei Liang, Jun Zhang, Wenhang Ge, Wei-Shi Zheng, Chunhua Shen

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments NIPS2022

Journal ref NIPS2022

LAVIS: A Library for Language-Vision Intelligence

Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Silvio Savarese, Steven C. H. Hoi

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Preprint of LAVIS technical report

AutoTransition: Learning to Recommend Video Transition Effects

Yaojie Shen, Libo Zhang, Kai Xu, Xiaojie Jin

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments To appear at ECCV 2022

CLIP2TV: Align, Match and Distill for Video-Text Retrieval

Zijian Gao, Jingyu Liu, Weiqi Sun, Sheng Chen, Dedan Chang, Lili Zhao

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Visually-aware Acoustic Event Detection using Heterogeneous Graphs

Amir Shirian, Krishna Somandepalli, Victor Sanchez, Tanaya Guha

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.MM、eess.AS

Technical Report for CVPR 2022 LOVEU AQTC Challenge

Hyeonyu Kim, Jongeun Kim, Jeonghun Kang, Sanguk Park, Dongchan Park, Taehwan Kim

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 4 pages, 3 figures, technical report for track3 of CVPR 2022 LOVEU challenge

The Second Place Solution for The 4th Large-scale Video Object Segmentation Challenge--Track 3: Referring Video Object Segmentation

Leilei Cao, Zhuang Li, Bo Yan, Feng Zhang, Fengliang Qi, Yuchen Hu, Hongbin Wang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments 4 pages, 2 figures

TubeDETR: Spatio-Temporal Video Grounding with Transformers

Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, Cordelia Schmid

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Updated vIoU results compared to the CVPR'22 camera-ready version; 17 pages; 8 figures

Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentation

Zihan Ding, Tianrui Hui, Junshi Huang, Xiaoming Wei, Jizhong Han, Si Liu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by CVPR 2022

Entity-aware and Motion-aware Transformers for Language-driven Action Localization in Videos

Shuo Yang, Xinxiao Wu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments accepted by IJCAI-22, Codes are available at https://github.com/shuoyang129/EAMAT

Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answering

Min Peng, Chongyang Wang, Yuan Gao, Yu Shi, Xiang-Dong Zhou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IJCAI 2022. arXiv admin note: text overlap with arXiv:2109.04735

Learning Commonsense-aware Moment-Text Alignment for Fast Video Temporal Grounding

Ziyue Wu, Junyu Gao, Shucheng Huang, Changsheng Xu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Submitted to IEEE TMM; Code is available at https://github.com/ZiyueWu59/CCA

Spatio-Temporal SAR-Optical Data Fusion for Cloud Removal via a Deep Hierarchical Model

Alessandro Sebastianelli, Artur Nowakowski, Erika Puglisi, Maria Pia Del Rosso, Jamila Mifdal, Fiora Pirri, Pierre Philippe Mathieu, Silvia Liberata Ullo

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Ego4D: Around the World in 3,000 Hours of Egocentric Video

Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhongcong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Cartillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli, Christoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Abrham Gebreselasie, Cristina Gonzalez, James Hillis, Xuhua Huang, Yifei Huang, Wenqi Jia, Weslie Khoo, Jachym Kolar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li, Yanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Modhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will Price, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran Somasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao, Minh Vo, Yuchen Wang, Xindi Wu, Takuma Yagi, Ziwei Zhao, Yunyi Zhu, Pablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria Farinella, Christian Fuegen, Bernard Ghanem, Vamsi Krishna Ithapu, C. V. Jawahar, Hanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude Oliva, Hyun Soo Park, James M. Rehg, Yoichi Sato, Jianbo Shi, Mike Zheng Shou, Antonio Torralba, Lorenzo Torresani, Mingfei Yan, Jitendra Malik

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.AI

Comments To appear in the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. This version updates the baseline result numbers for the Hands and Objects benchmark (appendix)

Neural Marionette: Unsupervised Learning of Motion Skeleton and Latent Dynamics from Volumetric Video

Jinseok Bae, Hojun Jang, Cheol-Hui Min, Hyungun Choi, Young Min Kim

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 7 pages (main), 10 pages (appendix) and to be appeared in AAAI2022

NEWSKVQA: Knowledge-Aware News Video Question Answering

Pranay Gupta, Manish Gupta

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Deep Video Prior for Video Consistency and Propagation

Chenyang Lei, Yazhou Xing, Hao Ouyang, Qifeng Chen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by TPAMI in Dec 2021; extension of NeurIPS2020 Blind Video Temporal Consistency via Deep Video Prior. arXiv admin note: substantial text overlap with arXiv:2010.11838