多模态大模型

Training Video Foundation Models with NVIDIA NeMo

Zeeshan Patel, Ethan He, Parth Mannan, Xiaowei Ren, Ryan Wolf, Niket Agarwal, Jacob Huffman, Zhuoyao Wang, Carl Wang, Jack Chang, Yan Bai, Tommy Huang, Linnan Wang, Sahil Jain, Shanmugam Ramasamy, Joseph Jennings, Ekaterina Sirazitdinova, Oleg Sudakov, Mingyuan Ma, Bobby Chen, Forrest Lin, Hao Wang, Vasanth Rao Naik Sabavat, Sriharsha Niverty, Rong Ou, Pallab Bhattacharya, David Page, Nima Tajbakhsh, Ashwath Aithal

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

LHM: Large Animatable Human Reconstruction Model from a Single Image in Seconds

Lingteng Qiu, Xiaodong Gu, Peihao Li, Qi Zuo, Weichao Shen, Junfei Zhang, Kejie Qiu, Weihao Yuan, Guanying Chen, Zilong Dong, Liefeng Bo

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://lingtengqiu.github.io/LHM/

HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognition

Kun Yuan, Vinkle Srivastav, Nassir Navab, Nicolas Padoy

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by MICCAI2024

DeformPAM: Data-Efficient Learning for Long-horizon Deformable Object Manipulation via Preference-based Action Alignment

Wendi Chen, Han Xue, Fangyuan Zhou, Yuan Fang, Cewu Lu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICRA 2025. Project page: https://deform-pam.robotflow.ai

Video Action Differencing

James Burgess, Xiaohan Wang, Yuhui Zhang, Anita Rau, Alejandro Lozano, Lisa Dunlap, Trevor Darrell, Serena Yeung-Levy

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments ICLR 2025 (International Conference on Learning Representations) Project page: http://jmhb0.github.io/viddiff Benchmark: https://huggingface.co/datasets/jmhb/VidDiffBench

Deciphering the complaint aspects: Towards an aspect-based complaint identification model with video complaint dataset in finance

Sarmistha Das, Basha Mujavarsheik, R E Zera Lyngkhoi, Sriparna Saha, Alka Maurya

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Adaptive Keyframe Sampling for Long Video Understanding

Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, Qixiang Ye

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments CVPR2025

InternVQA: Advancing Compressed Video Quality Assessment with Distilling Large Foundation Model

Fengbin Guan, Zihao Yu, Yiting Lu, Xin Li, Zhibo Chen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ISCAS 2025(Lecture)

VidLBEval: Benchmarking and Mitigating Language Bias in Video-Involved LVLMs

Yiming Yang, Yangyang Guo, Hui Lu, Yan Wang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Do Language Models Understand Time?

Xi Ding, Lei Wang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication in the Companion Proceedings of the ACM Web Conference (WWW Companion 2025)

Unhackable Temporal Rewarding for Scalable Video MLLMs

En Yu, Kangheng Lin, Liang Zhao, Yana Wei, Zining Zhu, Haoran Wei, Jianjian Sun, Zheng Ge, Xiangyu Zhang, Jingyu Wang, Wenbing Tao

专题命中 视频多模态 ：MLLM(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ICLR2025. Project Page: https://ahnsun.github.io/UTR/

Knowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Grounding

Xin Gu, Yaojie Shen, Chenxi Luo, Tiejian Luo, Yan Huang, Yuewei Lin, Heng Fan, Libo Zhang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VideoQA-SC: Adaptive Semantic Communication for Video Question Answering

Jiangyuan Guo, Wei Chen, Yuxuan Sun, Jialong Xu, Bo Ai

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval

Reno Kriz, Kate Sanders, David Etter, Kenton Murray, Cameron Carpenter, Kelly Van Ochten, Hannah Recknor, Jimena Guallar-Blasco, Alexander Martin, Ronald Colaianni, Nolan King, Eugene Yang, Benjamin Van Durme

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Tonguescape: Exploring Language Models Understanding of Vowel Articulation

Haruki Sakajo, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2025

Task Me Anything

Jieyu Zhang, Weikai Huang, Zixian Ma, Oscar Michel, Dong He, Tanmay Gupta, Wei-Chiu Ma, Ali Farhadi, Aniruddha Kembhavi, Ranjay Krishna

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Track on Datasets and Benchmarks. Website: https://www.task-me-anything.org

Towards Safer Social Media Platforms: Scalable and Performant Few-Shot Harmful Content Moderation Using Large Language Models

Akash Bonagiri, Lucen Li, Rajvardhan Oak, Zeerak Babar, Magdalena Wojcieszak, Anshuman Chhabra

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments This paper is in submission and under peer review

ReasVQA: Advancing VideoQA with Imperfect Reasoning Process

Jianxin Liang, Xiaojun Meng, Huishuai Zhang, Yueqian Wang, Jiansheng Wei, Dongyan Zhao

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to main conference at NAACL 2025; 8 pages;

Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge

Haomiao Xiong, Zongxin Yang, Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, Huchuan Lu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICLR 2025. Code is available at https://github.com/hmxiong/StreamChat

Neptune: The Long Orbit to Benchmarking Long Video Understanding

Arsha Nagrani, Mingda Zhang, Ramin Mehran, Rachel Hornung, Nitesh Bharadwaj Gundavarapu, Nilpa Jha, Austin Myers, Xingyi Zhou, Boqing Gong, Cordelia Schmid, Mikhail Sirotenko, Yukun Zhu, Tobias Weyand

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Facial Dynamics in Video: Instruction Tuning for Improved Facial Expression Perception and Contextual Awareness

Jiaxing Zhao, Boyuan Sun, Xiang Chen, Xihan Wei

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

A Text-to-Game Engine for UGC-Based Role-Playing Games

Lei Zhang, Xuezheng Peng, Shuyi Yang, Feiyang Wang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 9 figures

VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition

Lan Chen, Haoxiang Yang, Pengpeng Shao, Haoyu Song, Xiao Wang, Zhicheng Zhao, Yaowei Wang, Yonghong Tian

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments In Peer Review

Modality-Aware Shot Relating and Comparing for Video Scene Detection

Jiawei Tan, Hongxing Wang, Kang Dang, Jiaxin Li, Zhilong Ou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Exploring Scalability of Self-Training for Open-Vocabulary Temporal Action Localization

Jeongseok Hyun, Su Ho Han, Hyolim Kang, Joon-Young Lee, Seon Joo Kim

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to WACV 2025

Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation

Xiaoqi An, Lin Zhao, Chen Gong, Jun Li, Jian Yang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to AAAI 2025

Text Proxy: Decomposing Retrieval from a 1-to-N Relationship into N 1-to-1 Relationships for Text-Video Retrieval

Jian Xiao, Zhenzhen Hu, Jia Li, Richang Hong

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Visual Object Tracking across Diverse Data Modalities: A Review

Mengmeng Wang, Teli Ma, Shuo Xin, Xiaojun Hou, Jiazheng Xing, Guang Dai, Jingdong Wang, Yong Liu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, Song Han

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Code and models are available at https://github.com/NVlabs/VILA/tree/main/longvila

Doe-1: Closed-Loop Autonomous Driving with Large World Model

Wenzhao Zheng, Zetian Xia, Yuanhui Huang, Sicheng Zuo, Jie Zhou, Jiwen Lu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Code is available at: https://github.com/wzzheng/Doe