视频大模型

WorldSimBench: Towards Video Generation Models as World Simulators

Yiran Qin, Zhelun Shi, Jiwen Yu, Xijun Wang, Enshen Zhou, Lijun Li, Zhenfei Yin, Xihui Liu, Lu Sheng, Jing Shao, Lei Bai, Wanli Ouyang, Ruimao Zhang

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Allegro: Open the Black Box of Commercial-Level Video Generation Model

Yuan Zhou, Qiuyue Wang, Yuxuan Cai, Huan Yang

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Fundus to Fluorescein Angiography Video Generation as a Retinal Generative Foundation Model

Weiyi Zhang, Jiancheng Yang, Ruoyu Chen, Siyu Huang, Pusheng Xu, Xiaolan Chen, Shanfu Lu, Hongyu Cao, Mingguang He, Danli Shi

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

DragEntity: Trajectory Guided Video Generation using Entity and Positional Relationships

Zhang Wan, Sheng Tang, Jiawei Wei, Ruize Zhang, Juan Cao

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments ACM MM2024 Oral

VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation

Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang, Quy Duc Do, Yuansheng Ni, Bohan Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, Yuchen Lin, Wenhu Chen

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

HARIVO: Harnessing Text-to-Image Models for Video Generation

Mingi Kwon, Seoung Wug Oh, Yang Zhou, Difan Liu, Joon-Young Lee, Haoran Cai, Baqiao Liu, Feng Liu, Youngjung Uh

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments ECCV2024

Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

Fanqing Meng, Jiaqi Liao, Xinyu Tan, Wenqi Shao, Quanfeng Lu, Kaipeng Zhang, Yu Cheng, Dianqi Li, Yu Qiao, Ping Luo

专题命中 视频生成 ：video generation(title)；text-to-video(abstract)；分类 cs.CV

Comments Project Page: https://phygenbench123.github.io/

Noise Crystallization and Liquid Noise: Zero-shot Video Generation using Image Diffusion Models

Muhammad Haaris Khan, Hadrien Reynaud, Bernhard Kainz

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation

Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y. Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, William T. Freeman

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments Project website at: https://physdreamer.github.io/ Appear on ECCV 2024

VideoPhy: Evaluating Physical Commonsense for Video Generation

Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, Aditya Grover

专题命中 视频生成 ：video generation(title)；text-to-video(abstract)；分类 cs.CV

Comments 43 pages, 29 figures, 12 tables. Added CogVideo and Dream Machine in v2

COMUNI: Decomposing Common and Unique Video Signals for Diffusion-based Video Generation

Mingzhen Sun, Weining Wang, Xinxin Zhu, Jing Liu

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

SurGen: Text-Guided Diffusion Model for Surgical Video Generation

Joseph Cho, Samuel Schmidgall, Cyril Zakka, Mrudang Mathur, Dhamanpreet Kaur, Rohan Shad, William Hiesinger

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

DiffTED: One-shot Audio-driven TED Talk Video Generation with Diffusion-based Co-speech Gestures

Steven Hogue, Chenxu Zhang, Hamza Daruger, Yapeng Tian, Xiaohu Guo

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generation

Wei Wu, Xi Guo, Weixuan Tang, Tingxuan Huang, Chiyu Wang, Dongyue Chen, Chenjing Ding

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments Homepage: https://metadrivescape.github.io/papers_project/drivescapev1/index.html

DriveGenVLM: Real-world Video Generation for Vision Language Model based Autonomous Driving

Yongjie Fu, Anmol Jain, Xuan Di, Xu Chen, Zhaobin Mo

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

TVG: A Training-free Transition Video Generation Method with Diffusion Models

Rui Zhang, Yaosen Chen, Yuegen Liu, Wei Wang, Xuming Wen, Hongxia Wang

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Towards Understanding Unsafe Video Generation

Yan Pang, Aiping Xiong, Yang Zhang, Tianhao Wang

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments 18 pages

Learning Online Scale Transformation for Talking Head Video Generation

Fa-Ting Hong, Dan Xu

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insights

Wentao Lei, Jinting Wang, Fengji Ma, Guanjie Huang, Li Liu

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

ID-Animator: Zero-Shot Identity-Preserving Human Video Generation

Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, Jie Zhang

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments Project Page: https://id-animator.github.io/

Dreamitate: Real-World Visuomotor Policy Learning via Video Generation

Junbang Liang, Ruoshi Liu, Ege Ozguroglu, Sruthi Sudhakar, Achal Dave, Pavel Tokmakov, Shuran Song, Carl Vondrick

专题命中 视频生成 ：video generation(title)；video diffusion(abstract)；分类 cs.CV

Comments Project page: https://dreamitate.cs.columbia.edu/

V-Express: Conditional Dropout for Progressive Training of Portrait Video Generation

Cong Wang, Kuan Tian, Jun Zhang, Yonghang Guan, Feng Luo, Fei Shen, Zhiwei Jiang, Qing Gu, Xiao Han, Wei Yang

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

VideoPoet: A Large Language Model for Zero-Shot Video Generation

Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, Krishna Somandepalli, Hassan Akbari, Yair Alon, Yong Cheng, Josh Dillon, Agrim Gupta, Meera Hahn, Anja Hauth, David Hendon, Alonso Martinez, David Minnen, Mikhail Sirotenko, Kihyuk Sohn, Xuan Yang, Hartwig Adam, Ming-Hsuan Yang, Irfan Essa, Huisheng Wang, David A. Ross, Bryan Seybold, Lu Jiang

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments To appear at ICML 2024; Project page: http://sites.research.google/videopoet/

Contextualized Diffusion Models for Text-Guided Image and Video Generation

Ling Yang, Zhilong Zhang, Zhaochen Yu, Jingwei Liu, Minkai Xu, Stefano Ermon, Bin Cui

专题命中 视频生成 ：video generation(title)；text-to-video(abstract)；分类 cs.CV

Comments ICLR 2024. Project: https://github.com/YangLing0818/ContextDiff

MAVIN: Multi-Action Video Generation with Diffusion Models via Transition Video Infilling

Bowen Zhang, Xiaofei Xie, Haotian Lu, Na Ma, Tianlin Li, Qing Guo

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

CamViG: Camera Aware Image-to-Video Generation with Multimodal Transformers

Andrew Marmon, Grant Schindler, José Lezama, Dan Kondratyuk, Bryan Seybold, Irfan Essa

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

OneTo3D: One Image to Re-editable Dynamic 3D Model and Video Generation

Jinwei Lin

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments 24 pages, 13 figures, 2 tables

Matten: Video Generation with Mamba-Attention

Yu Gao, Jiancheng Huang, Xiaopeng Sun, Zequn Jie, Yujie Zhong, Lin Ma

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Sora Detector: A Unified Hallucination Detection for Large Text-to-Video Models

Zhixuan Chu, Lei Zhang, Yichen Sun, Siqiao Xue, Zhibo Wang, Zhan Qin, Kui Ren

专题命中 视频生成 ：text-to-video(title,abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2306.08302, arXiv:2403.05131 by other authors

PEEKABOO: Interactive Video Generation via Masked-Diffusion

Yash Jain, Anshul Nasery, Vibhav Vineet, Harkirat Behl

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV

Comments Project webpage - https://jinga-lala.github.io/projects/Peekaboo/