视频大模型

Vidi: Large Multimodal Models for Video Understanding and Editing

Vidi Team, Celong Liu, Chia-Wen Kuo, Dawei Du, Fan Chen, Guang Chen, Jiamin Yuan, Lingxi Zhang, Lu Guo, Lusha Li, Longyin Wen, Qingyu Chen, Rachel Deng, Sijie Zhu, Stuart Siew, Tong Jin, Wei Lu, Wen Zhong, Xiaohui Shen, Xin Gu, Xing Mei, Xueqiong Qu, Zhenfang Chen

机构 * Intelligent Editing Team（智能编辑团队） ； Intelligent Creation, ByteDance Inc.（智能创作，字节跳动公司） 纠错

专题命中 视频数据与评测 ：video understanding(title)；long video(abstract)；分类 cs.CV

InstructionBench: An Instructional Video Understanding Benchmark

Haiwan Wei, Yitian Yuan, Xiaohan Lan, Wei Ke, Lin Ma

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?

Junhao Cheng, Yuying Ge, Teng Wang, Yixiao Ge, Jing Liao, Ying Shan

机构 * ARC Lab, Tencent PCG（腾讯PCG广告实验室） ； City University of Hong Kong（香港城市大学） 纠错

专题命中 视频数据与评测 ：video reasoning(title,abstract)；分类 cs.CV

Comments Homepage: https://github.com/TencentARC/Video-Holmes

H2VU-Benchmark: A Comprehensive Benchmark for Hierarchical Holistic Video Understanding

Qi Wu, Quanlong Zheng, Yanhao Zhang, Junlin Xie, Jinguo Luo, Kuo Wang, Peng Liu, Qingsong Xie, Ru Zhen, Zhenyu Yang, Haonan Lu

机构 * OPPO AI Center（OPPO人工智能中心） 纠错

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

SurveillanceVQA-589K: A Benchmark for Comprehensive Surveillance Video-Language Understanding with Large Models

Bo Liu, Pengfei Qiao, Minhan Ma, Xuange Zhang, Yinan Tang, Peng Xu, Kun Liu, Tongtong Yuan

机构 * Department of Computer Science, Beijing University of Technology（北京理工大学计算机科学系） ； Inspur Electronic Information Industry Co., Ltd（Inspur电子信息产业有限公司） ； Department of Electronic Engineering, Tsinghua University（清华大学电子工程系） ； JD Explore Academy（京东探索研究院） 纠错

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments The dataset and code are publicly available at: https://huggingface.co/datasets/fei213/SurveillanceVQA-589K

Video-Bench: Human-Aligned Video Generation Benchmark

Hui Han, Siyuan Li, Jiaqi Chen, Yiwen Yuan, Yuling Wu, Chak Tou Leong, Hanwen Du, Junchen Fu, Youhua Li, Jie Zhang, Chi Zhang, Li-jia Li, Yongxin Ni

机构 * Shanghai Jiao Tong University（上海交通大学） ； Stanford University（斯坦福大学） ； Fellou AI（Fellou人工智能） ； Fudan University（复旦大学） ； Carnegie Mellon University（卡内基梅隆大学） ； Peking University（北京大学） ； Hong Kong Polytechnic University（香港理工大学） ； University of Glasgow（格拉斯哥大学） ； City University of Hong Kong（香港城市大学） ； Westlake University（西湖大学） ； LiveX AI（LiveX人工智能） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

Comments Accepted by CVPR'25

VideoGen-Eval: Agent-based System for Video Generation Evaluation

Yuhang Yang, Ke Fan, Shangkun Sun, Hongxiang Li, Ailing Zeng, FeiLin Han, Wei Zhai, Wei Liu, Yang Cao, Zheng-Jun Zha

机构 * USTC（中国科学技术大学） ； SJTU（上海交通大学） ； PKUSZ（北京大学软件学院） ； Tencent（腾讯） ； BFA（北京航空航天大学） 纠错

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

Comments project:https://github.com/AILab-CVC/VideoGen-Eval

TVPR: Text-to-Video Person Retrieval and a New Benchmark

Xu Zhang, Fan Ni, Guan-Nan Dong, Aichun Zhu, Jianhui Wu, Mingcheng Ni, Hui Liu

机构 * Nanjing Tech University（南京理工大学） 纠错

专题命中 视频数据与评测 ：text-to-video(title,abstract)；分类 cs.CV

Comments 9 pages, 8 figures, Proceedings of the 32nd ACM International Conference on Multimedia

Journal ref The 32nd ACM International Conference on Multimedia. 2024: 10105-10113

ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentation

Ali Athar, Xueqing Deng, Liang-Chieh Chen

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2025. Project page: https://ali2500.github.io/vicas-project/

Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation

Hongcheng Gao, Jiashu Qu, Jingyi Tang, Baolong Bi, Yue Liu, Hongyu Chen, Li Liang, Li Su, Qingming Huang

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation

Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, Kai-Wei Chang

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

Comments 41 pages, 33 Figures

Is Your Video Language Model a Reliable Judge?

Ming Liu, Wensheng Zhang

专题命中 视频数据与评测 ：video language model(title,abstract)；分类 cs.CV

OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation

Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, Siyu Zhu

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

Comments 11 pages, 8 figures, 5 tables

What You See Is What Matters: A Novel Visual and Physics-Based Metric for Evaluating Video Generation Quality

Zihan Wang, Songlin Li, Lingyan Hao, Xinyu Hu, Bowen Song

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection

Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi, Yue Liao, Si Liu

专题命中 视频数据与评测 ：video reasoning(title,abstract)；分类 cs.CV

Comments 14 pages, 14 figures

Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability,Reproducibility, and Practicality

Tianle Zhang, Langtian Ma, Yuchen Yan, Yuchen Zhang, Kai Wang, Yue Yang, Ziyao Guo, Wenqi Shao, Yang You, Yu Qiao, Ping Luo, Kaipeng Zhang

专题命中 视频数据与评测 ：text-to-video(title,abstract)；分类 cs.CV

VERIFIED: A Video Corpus Moment Retrieval Benchmark for Fine-Grained Video Understanding

Houlun Chen, Xin Wang, Hong Chen, Zeyang Zhang, Wei Feng, Bin Huang, Jia Jia, Wenwu Zhu

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Comments Accepted by 38th NeurIPS Datasets & Benchmarks Track (NeurIPS 2024)

Beyond FVD: Enhanced Evaluation Metrics for Video Generation Quality

Ge Ya Luo, Gian Mario Favero, Zhi Hao Luo, Alexia Jolicoeur-Martineau, Christopher Pal

专题命中 视频数据与评测 ：video generation(title,abstract)；分类 cs.CV

VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models

Shicheng Li, Lei Li, Shuhuai Ren, Yuanxin Liu, Yi Liu, Rundong Gao, Xu Sun, Lu Hou

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2024. 19 pages, 3 figures, 8 tables. Data is available at https://github.com/lscpku/VITATECS

PiTe: Pixel-Temporal Alignment for Large Video-Language Model

Yang Liu, Pengxiang Ding, Siteng Huang, Min Zhang, Han Zhao, Donglin Wang

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

DeVAn: Dense Video Annotation for Video-Language Models

Tingkai Liu, Yunzhe Tao, Haogeng Liu, Qihang Fan, Ding Zhou, Huaibo Huang, Ran He, Hongxia Yang

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Published in 62nd ACL (2024) Main Track

Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment

Tengchuan Kou, Xiaohong Liu, Zicheng Zhang, Chunyi Li, Haoning Wu, Xiongkuo Min, Guangtao Zhai, Ning Liu

专题命中 视频数据与评测 ：text-to-video(title,abstract)；分类 cs.CV

Comments Accepted by ACMMM 24

VideoVista: A Versatile Benchmark for Video Understanding and Reasoning

Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, Min Zhang

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Comments 38 pages, 44 figures

Investigating Video Reasoning Capability of Large Language Models with Tropes in Movies

Hung-Ting Su, Chun-Tong Chao, Ya-Ching Hsu, Xudong Lin, Yulei Niu, Hung-Yi Lee, Winston H. Hsu

专题命中 视频数据与评测 ：video reasoning(title,abstract)；分类 cs.CV

Comments Project page: https://ander1119.github.io/TiM

MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, Yu Qiao

专题命中 视频数据与评测 ：video understanding(title,abstract)；分类 cs.CV

Comments CVPR 2024 highlight: updated version with Mistral and better performances for MVBench/NExT-QA/STAR/TVQA/EgoSchema/IntentQA

SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge

Andong Wang, Bo Wu, Sunli Chen, Zhenfang Chen, Haotian Guan, Wei-Ning Lee, Li Erran Li, Chuang Gan

专题命中 视频数据与评测 ：video reasoning(title,abstract)；分类 cs.CV

Comments CVPR

Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detection

Yang Liu, Tongfei Shen, Dong Zhang, Qingying Sun, Shoushan Li, Guodong Zhou

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted by ICMR 2024

VidLA: Video-Language Alignment at Scale

Mamshad Nayeem Rizve, Fan Fei, Jayakrishnan Unnikrishnan, Son Tran, Benjamin Z. Yao, Belinda Zeng, Mubarak Shah, Trishul Chilimbi

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2024

Fighting FIRe with FIRE: Assessing the Validity of Text-to-Video Retrieval Benchmarks

Pedro Rodriguez, Mahmoud Azab, Becka Silvert, Renato Sanchez, Linzy Labson, Hardik Shah, Seungwhan Moon

专题命中 视频数据与评测 ：text-to-video(title,abstract)；分类 cs.CV

Comments EACL 2023 Camera Ready

Synopses of Movie Narratives: a Video-Language Dataset for Story Understanding

Yidan Sun, Qin Chao, Yangfeng Ji, Boyang Li

专题命中 视频数据与评测 ：video-language(title,abstract)；分类 cs.CV

Comments 25 pages, 17 figures