多模态大模型

Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Model

Peng Wu, Jing Liu, Xiangteng He, Yuxin Peng, Peng Wang, Yanning Zhang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments This work has been accepted to the IEEE TIP. Copyright has been transferred

LVCHAT: Facilitating Long Video Comprehension

Yu Wang, Zeyuan Zhang, Julian McAuley, Zexue He

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 17 pages; 8 figures

Perceptual Video Quality Assessment: A Survey

Xiongkuo Min, Huiyu Duan, Wei Sun, Yucheng Zhu, Guangtao Zhai

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.MM

YTCommentQA: Video Question Answerability in Instructional Videos

Saelyne Yang, Sunghyun Park, Yunseok Jang, Moontae Lee

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments AAAI 2024

An Open Software Suite for Event-Based Video

Andrew C. Freeman

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Memory-Inspired Temporal Prompt Interaction for Text-Image Classification

Xinyao Yu, Hao Sun, Ziwei Niu, Rui Qin, Zhenjia Bai, Yen-Wei Chen, Lanfen Lin

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge

Dimitrios Psychogyios, Emanuele Colleoni, Beatrice Van Amsterdam, Chih-Yang Li, Shu-Yu Huang, Yuchong Li, Fucang Jia, Baosheng Zou, Guotai Wang, Yang Liu, Maxence Boels, Jiayu Huo, Rachel Sparks, Prokar Dasgupta, Alejandro Granados, Sebastien Ourselin, Mengya Xu, An Wang, Yanan Wu, Long Bai, Hongliang Ren, Atsushi Yamada, Yuriko Harai, Yuto Ishikawa, Kazuyuki Hayashi, Jente Simoens, Pieter DeBacker, Francesco Cisternino, Gabriele Furnari, Alex Mottrie, Federica Ferraguti, Satoshi Kondo, Satoshi Kasai, Kousuke Hirasawa, Soohee Kim, Seung Hyun Lee, Kyu Eun Lee, Hyoun-Joong Kong, Kui Fu, Chao Li, Shan An, Stefanie Krell, Sebastian Bodenstedt, Nicolas Ayobi, Alejandra Perez, Santiago Rodriguez, Juanita Puentes, Pablo Arbelaez, Omid Mohareri, Danail Stoyanov

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, Wancai Zhang, Zhifeng Li, Wei Liu, Li Yuan

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICLR 2024

Affective Video Content Analysis: Decade Review and New Perspectives

Junxiao Xue, Jie Wang, Xuecheng Wu, Qian Zhang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VKIE: The Application of Key Information Extraction on Video Text

Siyu An, Ye Liu, Haoyuan Peng, Di Yin

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

STAIR: Spatial-Temporal Reasoning with Auditable Intermediate Results for Video Question Answering

Yueqian Wang, Yuxuan Wang, Kai Chen, Dongyan Zhao

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments To appear in AAAI 2024

SkateboardAI: The Coolest Video Action Recognition for Skateboarding

Hanxiao Chen

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments The original first-author work has been accepted and presented by CVPR 2022 WiCV Workshop (This is the long-version paper)

EvPlug: Learn a Plug-and-Play Module for Event and Image Fusion

Jianping Jiang, Xinyu Zhou, Peiqi Duan, Boxin Shi

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalities

Hammad A. Ayyubi, Christopher Thomas, Lovish Chum, Rahul Lokesh, Long Chen, Yulei Niu, Xudong Lin, Xuande Feng, Jaywon Koo, Sounak Ray, Shih-Fu Chang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments AAAI 2024

TF-CLIP: Learning Text-free CLIP for Video-based Person Re-Identification

Chenyang Yu, Xuehu Liu, Yingquan Wang, Pingping Zhang, Huchuan Lu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments This work is accepted by AAAI2024

Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlines

Yoo Yeon Sung, Jordan Boyd-Graber, Naeemul Hassan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2023 Main Paper

PG-Video-LLaVA: Pixel Grounding Large Video-Language Models

Shehan Munasinghe, Rusiru Thushara, Muhammad Maaz, Hanoona Abdul Rasheed, Salman Khan, Mubarak Shah, Fahad Khan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Technical Report

Navigating Open Set Scenarios for Skeleton-based Action Recognition

Kunyu Peng, Cheng Yin, Junwei Zheng, Ruiping Liu, David Schneider, Jiaming Zhang, Kailun Yang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to AAAI 2024. The benchmark, code, and models will be released at https://github.com/KPeng9510/OS-SAR

SPOT! Revisiting Video-Language Models for Event Understanding

Gengyuan Zhang, Jinhe Bi, Jindong Gu, Yanyu Chen, Volker Tresp

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challenges

Tongtong Yuan, Xuange Zhang, Kun Liu, Bo Liu, Chen Chen, Jian Jin, Zhenzhen Jiao

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

LLVMs4Protest: Harnessing the Power of Large Language and Vision Models for Deciphering Protests in the News

Yongjun Zhang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

FFINet: Future Feedback Interaction Network for Motion Forecasting

Miao Kang, Shengqi Wang, Sanping Zhou, Ke Ye, Jingjing Jiang, Nanning Zheng

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 8 figures, 12 tables

Temporal Shift -- Multi-Objective Loss Function for Improved Anomaly Fall Detection

Stefan Denkovski, Shehroz S. Khan, Alex Mihailidis

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Long Story Short: a Summarize-then-Search Method for Long Video Question Answering

Jiwan Chung, Youngjae Yu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Published in BMVC 2023

Video Referring Expression Comprehension via Transformer with Content-conditioned Query

Ji Jiang, Meng Cao, Tengtao Song, Long Chen, Yi Wang, Yuexian Zou

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ACM International Conference on Multimedia Workshop (ACM MM), 2023. arXiv admin note: substantial text overlap with arXiv:2210.02953

Paxion: Patching Action Knowledge in Video-Language Foundation Models

Zhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu, Jaemin Cho, Zineng Tang, Mohit Bansal, Heng Ji

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments NeurIPS 2023 spotlight

Information-Theoretic Text Hallucination Reduction for Video-grounded Dialogue

Sunjae Yoon, Eunseop Yoon, Hee Suk Yoon, Junyeong Kim, Chang D. Yoo

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 12 pages, Accepted in EMNLP 2022

Learning to Locate Visual Answer in Video Corpus Using Question

Bin Li, Yixuan Weng, Bin Sun, Shutao Li

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ICASSP 2023

DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population

Ningyu Zhang, Xin Xu, Liankuan Tao, Haiyang Yu, Hongbin Ye, Shuofei Qiao, Xin Xie, Xiang Chen, Zhoubo Li, Lei Li, Xiaozhuan Liang, Yunzhi Yao, Shumin Deng, Peng Wang, Wen Zhang, Zhenru Zhang, Chuanqi Tan, Qiang Chen, Feiyu Xiong, Fei Huang, Guozhou Zheng, Huajun Chen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2022 System Demonstrations and the project website is http://deepke.zjukg.cn/

Pretrained Language Models as Visual Planners for Human Assistance

Dhruvesh Patel, Hamid Eghbalzadeh, Nitin Kamra, Michael Louis Iuzzolino, Unnat Jain, Ruta Desai

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at ICCV 2023