视频大模型

Deep Temporal Reasoning in Video Language Models: A Cross-Linguistic Evaluation of Action Duration and Completion through Perfect Times

Olga Loginova, Sofía Ortega Loguinova

机构 * University of Trento（特伦托大学） ； Maastricht University（马斯特里赫特大学） 纠错

专题命中 动作与事件理解 ：video language model(title)；video-language(abstract)；分类 cs.CV

This&That: Language-Gesture Controlled Video Generation for Robot Planning

Boyang Wang, Nikhil Sridhar, Chao Feng, Mark Van der Merwe, Adam Fishman, Nima Fazeli, Jeong Joon Park

机构 * University of Michigan（密歇根大学） ； University of Washington（华盛顿大学） 纠错

专题命中 动作与事件理解 ：video generation(title,abstract)；分类 cs.CV

EasyGenNet: An Efficient Framework for Audio-Driven Gesture Video Generation Based on Diffusion Model

Renda Li, Xiaohua Qi, Qiang Ling, Jun Yu, Ziyi Chen, Peng Chang, Mei HanJing Xiao

专题命中 动作与事件理解 ：video generation(title,abstract)；分类 cs.CV

Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level

Andong Deng, Tongjia Chen, Shoubin Yu, Taojiannan Yang, Lincoln Spencer, Yapeng Tian, Ajmal Saeed Mian, Mohit Bansal, Chen Chen

专题命中 动作与事件理解 ：video reasoning(title,abstract)；分类 cs.CV

Comments CVPR 2025

Motion Prompting: Controlling Video Generation with Motion Trajectories

Daniel Geng, Charles Herrmann, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevara, Carl Doersch, Yusuf Aytar, Michael Rubinstein, Chen Sun, Oliver Wang, Andrew Owens, Deqing Sun

专题命中 动作与事件理解 ：video generation(title,abstract)；分类 cs.CV

Comments CVPR 2025 camera ready. Project page: https://motion-prompting.github.io/

Synergizing Motion and Appearance: Multi-Scale Compensatory Codebooks for Talking Head Video Generation

Shuling Zhao, Fa-Ting Hong, Xiaoshui Huang, Dan Xu

专题命中 动作与事件理解 ：video generation(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2025. Project page: https://shaelynz.github.io/synergize-motion-appearance/

Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation

Kun Yuan, Vinkle Srivastav, Nassir Navab, Nicolas Padoy

专题命中 动作与事件理解 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted at the 38th Conference on Neural Information Processing Systems (NeurIPS 2024 Spolight)

Mind the Time: Temporally-Controlled Multi-Event Video Generation

Ziyi Wu, Aliaksandr Siarohin, Willi Menapace, Ivan Skorokhodov, Yuwei Fang, Varnith Chordia, Igor Gilitschenski, Sergey Tulyakov

专题命中 动作与事件理解 ：video generation(title)；video diffusion(abstract)；分类 cs.CV

Comments CVPR 2025. Project Page: https://mint-video.github.io/

The Devil is in Temporal Token: High Quality Video Reasoning Segmentation

Sitong Gong, Yunzhi Zhuge, Lu Zhang, Zongxin Yang, Pingping Zhang, Huchuan Lu

专题命中 动作与事件理解 ：video reasoning(title,abstract)；分类 cs.CV

Journal ref CVPR 2025

iMOVE: Instance-Motion-Aware Video Understanding

Jiaze Li, Yaya Shi, Zongyang Ma, Haoran Xu, Feng Cheng, Huihui Xiao, Ruiwen Kang, Fan Yang, Tingting Gao, Di Zhang

专题命中 动作与事件理解 ：video understanding(title,abstract)；分类 cs.CV

EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generation

Xiaofeng Wang, Kang Zhao, Feng Liu, Jiayu Wang, Guosheng Zhao, Xiaoyi Bao, Zheng Zhu, Yingya Zhang, Xingang Wang

专题命中 动作与事件理解 ：video generation(title,abstract)；分类 cs.CV

Comments Project Page: https://egovid.github.io/

GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation

Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, Hanbo Zhang, Minzhao Zhu

专题命中 动作与事件理解 ：video-language(title)；video generation(abstract)；分类 cs.CV

Comments Tech Report. Authors are listed in alphabetical order. Project page: https://gr2-manipulation.github.io

E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding

Ye Liu, Zongyang Ma, Zhongang Qi, Yang Wu, Ying Shan, Chang Wen Chen

专题命中 动作与事件理解 ：video-language(title)；video understanding(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2024 Datasets and Benchmarks Track

NAVERO: Unlocking Fine-Grained Semantics for Video-Language Compositionality

Chaofan Tao, Gukyeong Kwon, Varad Gunjal, Hao Yang, Zhaowei Cai, Yonatan Dukler, Ashwin Swaminathan, R. Manmatha, Colin Jon Taylor, Stefano Soatto

专题命中 动作与事件理解 ：video-language(title,abstract)；分类 cs.CV

Video Generation with Learned Action Prior

Meenakshi Sarkar, Devansh Bhardwaj, Debasish Ghose

专题命中 动作与事件理解 ：video generation(title,abstract)；分类 cs.CV

Can't make an Omelette without Breaking some Eggs: Plausible Action Anticipation using Large Video-Language Models

Himangi Mittal, Nakul Agarwal, Shao-Yuan Lo, Kwonjoon Lee

专题命中 动作与事件理解 ：video-language(title,abstract)；分类 cs.CV

Comments CVPR 2024

Disentangling Foreground and Background Motion for Enhanced Realism in Human Video Generation

Jinlin Liu, Kai Yu, Mengyang Feng, Xiefan Guo, Miaomiao Cui

专题命中 动作与事件理解 ：video generation(title,abstract)；分类 cs.CV

STOA-VLP: Spatial-Temporal Modeling of Object and Action for Video-Language Pre-training

Weihong Zhong, Mao Zheng, Duyu Tang, Xuan Luo, Heng Gong, Xiaocheng Feng, Bing Qin

专题命中 动作与事件理解 ：video-language(title,abstract)；分类 cs.CV

Comments AAAI 2023, 7 pages, 3 figures

Paxion: Patching Action Knowledge in Video-Language Foundation Models

Zhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu, Jaemin Cho, Zineng Tang, Mohit Bansal, Heng Ji

专题命中 动作与事件理解 ：video-language(title,abstract)；分类 cs.CV

Comments NeurIPS 2023 spotlight

Verbs in Action: Improving verb understanding in video-language models

Liliane Momeni, Mathilde Caron, Arsha Nagrani, Andrew Zisserman, Cordelia Schmid

专题命中 动作与事件理解 ：video-language(title,abstract)；分类 cs.CV

Audio-Driven Co-Speech Gesture Video Generation

Xian Liu, Qianyi Wu, Hang Zhou, Yuanqi Du, Wayne Wu, Dahua Lin, Ziwei Liu

专题命中 动作与事件理解 ：video generation(title,abstract)；分类 cs.CV

Comments Accepted by Advances in Neural Information Processing Systems (NeurIPS), 2022 (Spotlight Presentation). Camera-Ready Version, 19 Pages

Frame-wise Action Representations for Long Videos via Sequence Contrastive Learning

Minghao Chen, Fangyun Wei, Chong Li, Deng Cai

专题命中 动作与事件理解 ：long video(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2022

Hybrid Dynamic-static Context-aware Attention Network for Action Assessment in Long Videos

Ling-An Zeng, Fa-Ting Hong, Wei-Shi Zheng, Qi-Zhi Yu, Wei Zeng, Yao-Wei Wang, Jian-Huang Lai

专题命中 动作与事件理解 ：long video(title,abstract)；分类 cs.CV

Comments ACM International Conference on Multimedia 2020

Long Activity Video Understanding using Functional Object-Oriented Network

Ahmad Babaeian Jelodar, David Paulius, Yu Sun

专题命中 动作与事件理解 ：video understanding(title,abstract)；分类 cs.CV

Comments 12 pages, 12 figures

In-Context Ensemble Learning from Pseudo Labels Improves Video-Language Models for Low-Level Workflow Understanding

Moucheng Xu, Evangelos Chatzaroulas, Luc McCutcheon, Abdul Ahad, Hamzah Azeem, Janusz Marecki, Ammar Anwar

专题命中 动作与事件理解 ：video-language(title,abstract)

Comments To appear in NeurIPS Workshop on Video-Language Models 2024

HAIC: Improving Human Action Understanding and Generation with Better Captions for Multi-modal Large Language Models

Xiao Wang, Jingyun Hua, Weihong Lin, Yuanxing Zhang, Fuzheng Zhang, Jianlong Wu, Di Zhang, Liqiang Nie

机构 * Kuaishou Technology（快手科技） ； Shandong University（山东省大学） 纠错

专题命中 动作与事件理解 ：video generation(abstract)；video understanding(abstract)；text-to-video(abstract)；分类 cs.CV、cs.MM

STEER: Structured Event Evidence for Video Reasoning via Multi-Objective Reinforcement Learning

STEER: 通过多目标强化学习进行视频推理的结构事件证据

Zinuo Li, Yongxin Guo, Jun Liu, Jiawei Zhan, Xi Jiang, Chengjie Wang, Mohammed Bennamoun, Farid Boussaid, Feng Zheng, Qiuhong Ke

机构 * University of Western Australia（西澳大学） ； CUHKSZ（香港科技大学） ； Tencent Youtu Lab（腾讯优图实验室） ； SUSTech（南方科技大学） ； Monash University（墨尔本大学） 纠错

专题命中 动作与事件理解 ：video reasoning(title)；video understanding(abstract)

AI总结 本文提出结构事件证据方法，通过多目标强化学习解决视频推理中事件结构缺失问题，提出STEER-60K数据集和P-FAB算法提升推理效率与准确性。

MotionV2V: Editing Motion in a Video

MotionV2V: 视频中运动的编辑

Ryan Burgert, Charles Herrmann, Forrester Cole, Michael S Ryoo, Neal Wadhwa, Andrey Voynov, Nataniel Ruiz

机构 * Google（谷歌） ； Stony Brook University（石溪大学） 纠错

专题命中 动作与事件理解 ：video generation(abstract)；video diffusion(abstract)；text-to-video(abstract)；分类 cs.CV

AI总结 MotionV2V通过直接编辑稀疏轨迹实现视频运动编辑，利用生成性主干提升视频编辑能力，在用户研究中表现优异。

VideoPerceiver: Enhancing Fine-Grained Temporal Perception in Video Multimodal Large Language Models

VideoPerceiver: 提升视频多模态大语言模型的细粒度时序感知能力

Fufangchen Zhao, Liao Zhang, Daiqi Shi, Yuanjun Gao, Chen Ye, Yang Cai, Jian Gao, Danfeng Yan

机构 * State Key Laboratory of Networking and Switching Technology, BUPT（网络与交换技术国家重点实验室） 纠错

专题命中 动作与事件理解 ：video understanding(abstract)；video-language(abstract)；long video(abstract)；分类 cs.CV

AI总结 VideoPerceiver通过两阶段训练框架提升视频多模态大语言模型对细粒度动作和短暂事件的感知能力，显著优于现有模型。

E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning

Qiang Qu, Yiran Shen, Xiaoming Chen, Yuk Ying Chung, Tongliang Liu

专题命中 动作与事件理解 ：video generation(title)；分类 cs.CV、cs.MM

Comments Accepted in AAAI2024