视频大模型

MAMBA4D: Efficient Long-Sequence Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Models

Jiuming Liu, Jinru Han, Lihao Liu, Angelica I. Aviles-Rivero, Chaokang Jiang, Zhe Liu, Hesheng Wang

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

Comments Accepted by CVPR 2025. The first two authors contribute equally

Do Egocentric Video-Language Models Truly Understand Hand-Object Interactions?

Boshen Xu, Ziheng Wang, Yang Du, Zhinan Song, Sipeng Zheng, Qin Jin

专题命中 视频理解 ：video-language(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments Accepted by ICLR 2025. Code: https://github.com/xuboshen/EgoNCEpp

Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition

Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, Wynne Hsu

专题命中 视频理解 ：video reasoning(title,abstract)；video understanding(abstract)；分类 cs.CV

Comments Accepted by ICML 2024

Apollo: An Exploration of Video Understanding in Large Multimodal Models

Orr Zohar, Xiaohan Wang, Yann Dubois, Nikhil Mehta, Tong Xiao, Philippe Hansen-Estruch, Licheng Yu, Xiaofang Wang, Felix Juefei-Xu, Ning Zhang, Serena Yeung-Levy, Xide Xia

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

Comments https://apollo-lmms.github.io

Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding

Yan Shu, Zheng Liu, Peitian Zhang, Minghao Qin, Junjie Zhou, Zhengyang Liang, Tiejun Huang, Bo Zhao

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understanding

Kangsan Kim, Geon Park, Youngwan Lee, Woongyeong Yeo, Sung Ju Hwang

专题命中 视频理解 ：video understanding(title,abstract)；video-language(abstract)；分类 cs.CV

Look Every Frame All at Once: Video-Ma$^2$mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing

Hosu Lee, Junho Kim, Hyunjun Kim, Yong Man Ro

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

Comments Project page: https://ivy-lvlm.github.io/Video-MA2MBA/

TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability

Shimin Chen, Xiaohan Lan, Yitian Yuan, Zequn Jie, Lin Ma

专题命中 视频理解 ：video understanding(title)；video-language(abstract)；long video(abstract)；分类 cs.CV

TV-TREES: Multimodal Entailment Trees for Neuro-Symbolic Video Reasoning

Kate Sanders, Nathaniel Weir, Benjamin Van Durme

专题命中 视频理解 ：video reasoning(title)；video understanding(abstract)；video-language(abstract)；分类 cs.CV

Comments 9 pages, EMNLP 2024

OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding

Tiancheng Zhao, Qianqian Zhang, Kyusong Lee, Peng Liu, Lu Zhang, Chunxin Fang, Jiajia Liao, Kelei Jiang, Yibo Ma, Ruochen Xu

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

Comments 14 pages

Hallucination Mitigation Prompts Long-term Video Understanding

Yiwei Sun, Zhihang Liu, Chuanbin Liu, Bowei Pu, Zhihan Zhang, Hongtao Xie

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization

Yang Jin, Zhicheng Sun, Kun Xu, Kun Xu, Liwei Chen, Hao Jiang, Quzhe Huang, Chengru Song, Yuliang Liu, Di Zhang, Yang Song, Kun Gai, Yadong Mu

专题命中 视频理解 ：video-language(title,abstract)；video understanding(abstract)；分类 cs.CV

Learning text-to-video retrieval from image captioning

Lucas Ventura, Cordelia Schmid, Gül Varol

专题命中 视频理解 ：text-to-video(title,abstract)；video understanding(abstract)；分类 cs.CV

Comments A short version of this work appeared at CVPR 2023 Workshops. Project page: https://imagine.enpc.fr/~ventural/multicaps/

Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding

Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, Limin Wang

专题命中 视频理解 ：video understanding(title,abstract)；video-language(abstract)；分类 cs.CV

Comments Technical Report

VideoMamba: State Space Model for Efficient Video Understanding

Kunchang Li, Xinhao Li, Yi Wang, Yinan He, Yali Wang, Limin Wang, Yu Qiao

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

Comments 19 Pages, 7 Figures, 8 Tables

Slot-VLM: SlowFast Slots for Video-Language Modeling

Jiaqi Xu, Cuiling Lan, Wenxuan Xie, Xuejin Chen, Yan Lu

专题命中 视频理解 ：video-language(title,abstract)；video understanding(abstract)；分类 cs.CV

Comments 16 pages, 10 figures

Revisiting Kernel Temporal Segmentation as an Adaptive Tokenizer for Long-form Video Understanding

Mohamed Afham, Satya Narayan Shukla, Omid Poursaeed, Pengchuan Zhang, Ashish Shah, Sernam Lim

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

HierVL: Learning Hierarchical Video-Language Embeddings

Kumar Ashutosh, Rohit Girdhar, Lorenzo Torresani, Kristen Grauman

专题命中 视频理解 ：video-language(title,abstract)；long video(abstract)；分类 cs.CV

Comments CVPR 2023

Building Scalable Video Understanding Benchmarks through Sports

Aniket Agarwal, Alex Zhang, Karthik Narasimhan, Igor Gilitschenski, Vishvak Murahari, Yash Kant

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

VideoPipe 2022 Challenge: Real-World Video Understanding for Urban Pipe Inspection

Yi Liu, Xuan Zhang, Ying Li, Guixin Liang, Yabing Jiang, Lixia Qiu, Haiping Tang, Fei Xie, Wei Yao, Yi Dai, Yu Qiao, Yali Wang

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

Comments VideoPipe Challenge @ ICPR2022. Homepage: https://videopipe.github.io/

FitCLIP: Refining Large-Scale Pretrained Image-Text Models for Zero-Shot Video Understanding Tasks

Santiago Castro, Fabian Caba Heilbron

专题命中 视频理解 ：video understanding(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments Accepted at BMVC 2022. It includes the supplementary material. The margins and page size were modified to fit the arXiv ID stamp on the left side

The End-of-End-to-End: A Video Understanding Pentathlon Challenge (2020)

Samuel Albanie, Yang Liu, Arsha Nagrani, Antoine Miech, Ernesto Coto, Ivan Laptev, Rahul Sukthankar, Bernard Ghanem, Andrew Zisserman, Valentin Gabeur, Chen Sun, Karteek Alahari, Cordelia Schmid, Shizhe Chen, Yida Zhao, Qin Jin, Kaixu Cui, Hui Liu, Chen Wang, Yudong Jiang, Xiaoshuai Hao

专题命中 视频理解 ：video understanding(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments Individual reports, dataset information, rules, and released source code can be found at the competition webpage (https://www.robots.ox.ac.uk/~vgg/challenges/video-pentathlon)

Comprehensive Video Understanding: Video summarization with content-based video recommender design

Yudong Jiang, Kaixu Cui, Bo Peng, Changliang Xu

专题命中 视频理解 ：video understanding(title,abstract)；long video(abstract)；分类 cs.CV

Comments 2019 International Conference on Computer Vision Workshop (ICCVW 2019)

Mosaic: Cross-Modal Clustering for Efficient Video Understanding

Mosaic：用于高效视频理解的跨模态聚类

Tuowei Wang, He Zhou, Chengru Song, Qiushi Li, Ju Ren

专题命中 视频理解 ：video understanding(title,abstract)；video reasoning(abstract)

AI总结 Mosaic是首个针对流式长视频理解的跨模态聚类驱动VLM推理系统，通过优化KVCache管理实现1.38倍速度提升。

Optimizing Multimodal LLMs for Egocentric Video Understanding: A Solution for the HD-EPIC VQA Challenge

优化多模态大语言模型以实现视角视频理解：HD-EPIC VQA挑战的解决方案

Sicheng Yang, Yukai Huang, Shitong Sun, Weitong Cai, Jiankang Deng, Jifei Song, Zhensong Zhang

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV、eess.IV、cs.MM

AI总结 本文提出一种优化多模态大语言模型的方法，通过预处理、微调和时间链式思考提示，在HD-EPIC VQA挑战中实现了41.6%的准确率。

Comments 4 pages, 1 figure, CVPR 2025 EgoVis Workshop, 2nd Place in HD-EPIC Challenge

Watch, Remember, Reason: Human-View Video Understanding with MLLMs

Watch, Remember, Reason: 基于多模态大语言模型的人类视角视频理解

Jiahao Meng, Yue Tan, Qi Xu, Kuan Gao, Weisong Liu, Yanwei Li, Jason Li, Lingdong Kong, Haochen Wang, Qianyu Zhou, Jiangning Zhang, Guangliang Cheng, Yunhai Tong, Lu Qi, Minghsuan Yang

机构 * School of Intelligence Science and Technology, Peking University（北京理工大学智能科学与技术学院） ； Wuhan University（武汉大学） ； Shanghai Jiao Tong University（上海交通大学） ； Nanyang Technological University（南洋理工大学） ； CASIA（中国科学院自动化研究所） ； University of Tokyo（东京大学） ； University of Liverpool（利物浦大学） ； Zhejiang University（浙江大学） ； National University of Singapore（新加坡国立大学） ； UC Merced（加州大学默塞德分校） 纠错

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV、cs.MM

AI总结 提出人类视角下视频理解的三个功能能力（观看、记忆、推理），构建统一框架分析视频MLLM的感知、记忆、推理和预测，并总结挑战、方法、应用及未来方向。

StreamingEval: A Unified Evaluation Protocol towards Realistic Streaming Video Understanding

StreamingEval: 一种面向真实流媒体视频理解的统一评估协议

Guowei Tang, Tianwen Qian, Huanran Zheng, Yifei Wang, Xiaoling Wang

机构 * East China Normal University（华东师范大学） 纠错

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV、cs.MM

AI总结 本文提出StreamingEval评估框架，用于评估视频大语言模型在真实资源约束下的流媒体视频理解能力，通过标准化协议测试效率、存储与准确性的平衡，揭示当前模型与实际应用需求的差距。

Data-Efficient Learning for Generalizable Surgical Video Understanding

Sahar Nasirihaghighi

机构 * Department of Information Technology (ITEC), University of Klagenfurt（信息科技系，克雷根弗特大学） 纠错

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV、eess.IV

VidCompress: Memory-Enhanced Temporal Compression for Video Understanding in Large Language Models

Xiaohan Lan, Yitian Yuan, Zequn Jie, Lin Ma

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV、cs.MM

Comments 9 pages, 4 figures

A Coding Framework and Benchmark towards Low-Bitrate Video Understanding

Yuan Tian, Guo Lu, Yichao Yan, Guangtao Zhai, Li Chen, Zhiyong Gao

专题命中 视频理解 ：video understanding(title,abstract)；分类 cs.CV、eess.IV

Comments TPAMI2024