多模态大模型

Change Detection Between Optical Remote Sensing Imagery and Map Data via Segment Anything Model (SAM)

Hongruixuan Chen, Jian Song, Naoto Yokoya

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Panel Transitions for Genre Analysis in Visual Narratives

Yi-Chun Chen, Arnav Jhala

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation

Shilin Yan, Renrui Zhang, Ziyu Guo, Wenchao Chen, Wei Zhang, Hongyang Li, Yu Qiao, Hao Dong, Zhongjiang He, Peng Gao

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by AAAI 2024. Code is released at https://github.com/OpenGVLab/MUTR

Modular Blended Attention Network for Video Question Answering

Mingjie Zhou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments I will not add others' names since this work has not been published

Conditional Modeling Based Automatic Video Summarization

Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen, Min-Hung Chen, Marcel Worring

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments This work has been submitted to the IEEE for possible publication. arXiv admin note: substantial text overlap with arXiv:2305.00455

Generative Action Description Prompts for Skeleton-based Action Recognition

Wangmeng Xiang, Chao Li, Yuxuan Zhou, Biao Wang, Lei Zhang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by ICCV23

Exploiting Spatial-temporal Data for Sleep Stage Classification via Hypergraph Learning

Yuze Liu, Ziming Zhao, Tiehua Zhang, Kang Wang, Xin Chen, Xiaowei Huang, Jun Yin, Zhishu Shen

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)

Improving Social Media Popularity Prediction with Multiple Post Dependencies

Zhizhen Zhang, Xiaohui Xie, Mengyu Yang, Ye Tian, Yong Jiang, Yong Cui

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI、cs.MM

Verbs in Action: Improving verb understanding in video-language models

Liliane Momeni, Mathilde Caron, Arsha Nagrani, Andrew Zisserman, Cordelia Schmid

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning

Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, Cordelia Schmid

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2023 Camera-Ready; Project Webpage: https://antoyang.github.io/vid2seq.html ; 18 pages; 6 figures

COVID-VTS: Fact Extraction and Verification on Short Video Platforms

Fuxiao Liu, Yaser Yacoob, Abhinav Shrivastava

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 11 pages, 5 figures, accepted to EACL2023

LGDN: Language-Guided Denoising Network for Video-Language Modeling

Haoyu Lu, Mingyu Ding, Nanyi Fei, Yuqi Huo, Zhiwu Lu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by NeurIPS2022

Temporal Feature Alignment in Contrastive Self-Supervised Learning for Human Activity Recognition

Bulat Khaertdinov, Stylianos Asteriadis

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to IJCB 2022

Extending Compositional Attention Networks for Social Reasoning in Videos

Christina Sartzetaki, Georgios Paraskevopoulos, Alexandros Potamianos

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Journal ref Proc. Interspeech 2022, 1116-1120

What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data

Oier Mees, Lukas Hermann, Wolfram Burgard

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted for publication at IEEE Robotics and Automation Letters (RAL). Codebase and trained models available at http://hulc.cs.uni-freiburg.de

Learning to Retrieve Videos by Asking Questions

Avinash Madasu, Junier Oliva, Gedas Bertasius

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Journal ref ACM Multimedia 2022

Revisiting the "Video" in Video-Language Understanding

Shyamal Buch, Cristóbal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, Juan Carlos Niebles

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2022 (Oral)

Video as Conditional Graph Hierarchy for Multi-Granular Question Answering

Junbin Xiao, Angela Yao, Zhiyuan Liu, Yicong Li, Wei Ji, Tat-Seng Chua

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments AAAI'22 (Oral)

Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval

Jianfeng Dong, Yabing Wang, Xianke Chen, Xiaoye Qu, Xirong Li, Yuan He, Xun Wang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by IEEE Transactions on Circuits and Systems for Video Technology. Code is available at https://github.com/LiJiaBei-7/rivrl

Decompose the Sounds and Pixels, Recompose the Events

Varshanth R. Rao, Md Ibrahim Khalil, Haoda Li, Peng Dai, Juwei Lu

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted at AAAI 2022

Human-centric Spatio-Temporal Video Grounding With Visual Transformers

Zongheng Tang, Yue Liao, Si Liu, Guanbin Li, Xiaojie Jin, Hongxu Jiang, Qian Yu, Dong Xu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accept at TCSVT

SEA: Sentence Encoder Assembly for Video Retrieval by Textual Queries

Xirong Li, Fangming Zhou, Chaoxi Xu, Jiaqi Ji, Gang Yang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments accepted for publication as a REGULAR paper in the IEEE Transactions on Multimedia

COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning

Simon Ging, Mohammadreza Zolfaghari, Hamed Pirsiavash, Thomas Brox

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 27 pages, 5 figures, 19 tables. To be published in the 34th conference on Neural Information Processing Systems (NeurIPS 2020). The first two authors contributed equally to this work

What is More Likely to Happen Next? Video-and-Language Future Event Prediction

Jie Lei, Licheng Yu, Tamara L. Berg, Mohit Bansal

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2020 (17 pages)

Video Question Answering on Screencast Tutorials

Wentian Zhao, Seokhwan Kim, Ning Xu, Hailin Jin

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Analogical Reasoning for Visually Grounded Language Acquisition

Bo Wu, Haoyu Qin, Alireza Zareian, Carl Vondrick, Shih-Fu Chang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 12 pages

VIOLIN: A Large-Scale Dataset for Video-and-Language Inference

Jingzhou Liu, Wenhu Chen, Yu Cheng, Zhe Gan, Licheng Yu, Yiming Yang, Jingjing Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to CVPR2020

Game-Based Video-Context Dialogue

Ramakanth Pasunuru, Mohit Bansal

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2018 (14 pages) (fixed Table5 typo in v2)

Learning to Separate Object Sounds by Watching Unlabeled Video

Ruohan Gao, Rogerio Feris, Kristen Grauman

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.MM、eess.AS

Comments Published in ECCV 2018; Project Page: http://vision.cs.utexas.edu/projects/separating_object_sounds/

Do You See What I Mean? Visual Resolution of Linguistic Ambiguities

Yevgeni Berzak, Andrei Barbu, Daniel Harari, Boris Katz, Shimon Ullman

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2015

Journal ref Conference on Empirical Methods in Natural Language Processing (EMNLP), 2015, pages 1477--1487