视频大模型

S2DM: Sector-Shaped Diffusion Models for Video Generation

Haoran Lang, Yuxuan Ge, Zheng Tian

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments 17 pages, 6 figures

Context-aware Talking Face Video Generation

Meidai Xuanyuan, Yuwang Wang, Honglei Guo, Qionghai Dai

专题命中 视频生成 ：video generation(title,abstract)；video diffusion(abstract)；分类 cs.CV

Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis

Willi Menapace, Aliaksandr Siarohin, Ivan Skorokhodov, Ekaterina Deyneka, Tsai-Shien Chen, Anil Kag, Yuwei Fang, Aleksei Stoliar, Elisa Ricci, Jian Ren, Sergey Tulyakov

专题命中 视频生成 ：text-to-video(title,abstract)；video generation(abstract)；分类 cs.CV

Inflation with Diffusion: Efficient Temporal Adaptation for Text-to-Video Super-Resolution

Xin Yuan, Jinoo Baek, Keyang Xu, Omer Tov, Hongliang Fei

专题命中 视频生成 ：text-to-video(title,abstract)；video generation(abstract)；分类 cs.CV

Comments WACV'24 workshop

WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens

Xiaofeng Wang, Zheng Zhu, Guan Huang, Boyuan Wang, Xinze Chen, Jiwen Lu

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments project page: https://world-dreamer.github.io/

Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions

David Junhao Zhang, Dongxu Li, Hung Le, Mike Zheng Shou, Caiming Xiong, Doyen Sahoo

专题命中 视频生成 ：video generation(title,abstract)；video diffusion(abstract)；分类 cs.CV

Comments project page: https://showlab.github.io/Moonshot/

Leveraging Generative Language Models for Weakly Supervised Sentence Component Analysis in Video-Language Joint Learning

Zaber Ibn Abdul Hakim, Najibul Haque Sarker, Rahul Pratap Singh, Bishmoy Paul, Ali Dabouei, Min Xu

专题命中 视频生成 ：video-language(title,abstract)；text-to-video(abstract)；分类 cs.CV

Fine-grained Controllable Video Generation via Object Appearance and Context

Hsin-Ping Huang, Yu-Chuan Su, Deqing Sun, Lu Jiang, Xuhui Jia, Yukun Zhu, Ming-Hsuan Yang

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments Project page: https://hhsinping.github.io/factor

FlowZero: Zero-Shot Text-to-Video Synthesis with LLM-Driven Dynamic Scene Syntax

Yu Lu, Linchao Zhu, Hehe Fan, Yi Yang

专题命中 视频生成 ：text-to-video(title,abstract)；video generation(abstract)；分类 cs.CV

Comments Project page: https://flowzero-video.github.io

Breathing Life Into Sketches Using Text-to-Video Priors

Rinon Gal, Yael Vinker, Yuval Alaluf, Amit H. Bermano, Daniel Cohen-Or, Ariel Shamir, Gal Chechik

专题命中 视频生成 ：text-to-video(title,abstract)；video diffusion(abstract)；分类 cs.CV

Comments Project page: https://livesketch.github.io/

Make Pixels Dance: High-Dynamic Video Generation

Yan Zeng, Guoqiang Wei, Jiani Zheng, Jiaxin Zou, Yang Wei, Yuchen Zhang, Hang Li

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments 12 pages

VideoCrafter1: Open Diffusion Models for High-Quality Video Generation

Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, Ying Shan

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments Tech Report; Github: https://github.com/AILab-CVC/VideoCrafter Homepage: https://ailab-cvc.github.io/videocrafter/

Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator

Hanzhuo Huang, Yufan Feng, Cheng Shi, Lan Xu, Jingyi Yu, Sibei Yang

专题命中 视频生成 ：text-to-video(title,abstract)；video generation(abstract)；分类 cs.CV

Comments NeurIPS 2023; Project available at: https://github.com/SooLab/Free-Bloom

SimDA: Simple Diffusion Adapter for Efficient Video Generation

Zhen Xing, Qi Dai, Han Hu, Zuxuan Wu, Yu-Gang Jiang

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation

Yingqing He, Menghan Xia, Haoxin Chen, Xiaodong Cun, Yuan Gong, Jinbo Xing, Yong Zhang, Xintao Wang, Chao Weng, Ying Shan, Qifeng Chen

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments Github: https://github.com/VideoCrafter/Animate-A-Story Project page: https://videocrafter.github.io/Animate-A-Story

Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance

Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, Ying Shan, Tien-Tsin Wong

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

Comments 13 pages, 8 figures. Project page: https://doubiiu.github.io/projects/Make-Your-Video/

Sketching the Future (STF): Applying Conditional Control Techniques to Text-to-Video Models

Rohan Dhesikan, Vignesh Rajmohan

专题命中 视频生成 ：text-to-video(title,abstract)；video generation(abstract)；分类 cs.CV

Comments 9 pages, 8 figures

Video Generation Beyond a Single Clip

Hsin-Ping Huang, Yu-Chuan Su, Ming-Hsuan Yang

专题命中 视频生成 ：video generation(title,abstract)；long video(abstract)；分类 cs.CV

Learning Trajectory-Word Alignments for Video-Language Tasks

Xu Yang, Zhangzikang Li, Haiyang Xu, Hanwang Zhang, Qinghao Ye, Chenliang Li, Ming Yan, Yu Zhang, Fei Huang, Songfang Huang

专题命中 视频生成 ：video-language(title,abstract)；text-to-video(abstract)；分类 cs.CV

Phenaki: Variable Length Video Generation From Open Domain Textual Description

Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, Dumitru Erhan

专题命中 视频生成 ：video generation(title,abstract)；long video(abstract)；分类 cs.CV

Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions

Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo

专题命中 视频生成 ：video-language(title,abstract)；text-to-video(abstract)；分类 cs.CV

Journal ref published in CVPR 2022

Sync-DRAW: Automatic Video Generation using Deep Recurrent Attentive Architectures

Gaurav Mittal, Tanya Marwah, Vineeth N. Balasubramanian

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)；分类 cs.CV

RAVEN-Eval: Rubric-Guided Automatic Evaluation for AI Video Generation Models Based on LMM Preference Judgement

RAVEN-Eval：基于大语言多模态模型（LMM）偏好判断的、采用评分准则引导的AI视频生成模型自动评估框架

Ziheng Jia, Jiaying Qian, Zicheng Zhang, Xiaorong Zhu, Lancheng Gao, Xiongkuo Min

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)

AI总结 本文提出RAVEN-Eval框架，基于LMM偏好判断与评分准则引导，可自动评估AI视频生成模型，已筛选任务、收集数据、评估模型与LMM并建立排行榜，为AIVGMs评估提供可扩展路径。

Bring Music The Horizon: Music-Driven 360$^\circ$ Video Generation

将音乐带入视野：音乐驱动的360°视频生成

Kai Hsu Tsai, Yong Wei Fu, Hung I Yang, Yu-Chih Chen

机构 * National Yang Ming Chiao Tung University（国立阳明交通大学） 纠错

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV、eess.IV、cs.MM

AI总结 【一句话总结】该研究针对音乐可视化问题，提出情感感知的音乐驱动360°视频生成流程，先预估歌曲情感轨迹，再转化为视觉引导，经SEGA框架生成关键帧，最后合成视频，能反映歌曲情感与结构。

Comments 5 pages, 1 figure

Action Agent: Agentic Video Generation Meets Flow-Constrained Diffusion

动作代理：代理视频生成与流约束扩散的结合

Jeffrin Sam, Nguyen Khang, Yara Mahmoud, Miguel Altamirano Cabrera, Dzmitry Tsetserukou

机构 * Intelligent Space Robotics Laboratory, Skoltech（斯克里普切尔技术大学智能空间机器人实验室） 纠错

专题命中 视频生成 ：video generation(title,abstract)；video diffusion(abstract)

AI总结 本文提出Action Agent框架，结合代理导航视频生成与流约束扩散控制，通过两阶段方法提升多体机器人导航性能，实现从语言和图像输入生成物理合理的第一人称导航视频，并在真实和仿真环境中取得高成功率。

Comments 8 pages, 5 figures

LAViG-FLOW: Latent Autoregressive Video Generation for Fluid Flow Simulations

LAViG-FLOW：用于流体流动模拟的潜在自回归视频生成

Vittoria De Pellegrini, Tariq Alkhalifah

专题命中 视频生成 ：video generation(title,abstract)；video diffusion(abstract)

AI总结 LAViG-FLOW通过潜在自回归视频生成框架高效模拟流体流动，实现比传统方法快百倍的饱和度和压力场生成。

Communicative Agents for Slideshow Storytelling Video Generation based on LLMs

Jingxing Fan, Jinrong Shen, Yusheng Yao, Shuangqing Wang, Qian Wang, Yuling Wang

专题命中 视频生成 ：video generation(title,abstract)；text-to-video(abstract)

Comments 8 pages, 8 figures, 1 table

Probabilistic Adaptation of Text-to-Video Models

Mengjiao Yang, Yilun Du, Bo Dai, Dale Schuurmans, Joshua B. Tenenbaum, Pieter Abbeel

专题命中 视频生成 ：text-to-video(title,abstract)；video diffusion(abstract)

Comments Project website https://video-adapter.github.io/. First two authors contributed equally

SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix

Peng Dai, Feitong Tan, Qiangeng Xu, David Futschik, Ruofei Du, Sean Fanello, Xiaojuan Qi, Yinda Zhang

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV；video diffusion(comments)

Comments 3D stereoscopic video generation, video diffusion, inpainting

Delta Forcing: Trust Region Steering for Interactive Autoregressive Video Generation

Delta Forcing：交互式自回归视频生成中的信任区域引导

Yuheng Wu, Xiangbo Gao, Tianhao Chen, Xinghao Chen, Qing Yin, Zhengzhong Tu, Dongman Lee

机构 * Texas A\&M University（德克萨斯A&M大学） ； University of Washington（华盛顿大学） 纠错

专题命中 视频生成 ：video generation(title,abstract)；分类 cs.CV、cs.MM

AI总结 本文提出Delta Forcing方法，通过约束不可靠的教师监督在适应性信任区域中，以提高自回归视频生成的一致性并保持对新事件的响应性。

Comments preprint