多模态大模型

Just Dance with $π$! A Poly-modal Inductor for Weakly-supervised Video Anomaly Detection

Snehashis Majhi, Giacomo D'Amicantonio, Antitza Dantcheva, Quan Kong, Lorenzo Garattoni, Gianpiero Francesca, Egor Bondarev, Francois Bremond

机构 * INRIA ； Côte d’Azur University（科西嘉-阿兹尔大学） ； Woven by Toyota（丰田编织公司） ； Toyota Motor Europe（丰田欧洲公司） ； Eindhoven University of Technology（埃因霍温理工大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification

Jiaxing Xu, Kai He, Yue Tang, Wei Li, Mengcheng Lan, Xia Dong, Yiping Ke, Mengling Feng

机构 * College of Computing and Data Science, Nanyang Technological University, Singapore（南洋理工大学计算与数据科学学院） ； Saw Swee Hock School of Public Health at National University of Singapore, Singapore（新加坡国立大学 Saw Swee Hock 公共卫生学院） ； Xi’an Jiaotong University, Xi’an, Shaanxi（西安交通大学） ； S-Lab, Nanyang Technological University, Singapore（南洋理工大学 S 实验室） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Early accepted by MICCAI 2025

HLV-1K: A Large-scale Hour-Long Video Benchmark for Time-Specific Long Video Understanding

Heqing Zou, Tianze Luo, Guiyang Xie, Victor Xiao Jie Zhang, Fengmao Lv, Guangcong Wang, Junyang Chen, Zhuochen Wang, Hansheng Zhang, Huaijian Zhang

机构 * ByteDance（字节跳动） ； Nanyang Technological University（南洋理工大学） ； Southwest Jiaotong University（西南交通大学） ； Great Bay University（大湾大学） ； Shenzhen University（深圳大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICME 2025

When SAM2 Meets Video Camouflaged Object Segmentation: A Comprehensive Evaluation and Adaptation

Yuli Zhou, Guolei Sun, Yawei Li, Guo-Sen Xie, Luca Benini, Ender Konukoglu

机构 * Computer Vision Laboratory, ETH Zürich（苏黎世联邦理工学院计算机视觉实验室） ； Integrated System Laboratory, ETH Zürich（苏黎世联邦理工学院集成系统实验室） ； School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Technical report. Accepted by Visual Intelligence. Code is released at https://github.com/zhoustan/SAM2-VCOS

About Time: Advances, Challenges, and Outlooks of Action Understanding

Alexandros Stergiou, Ronald Poppe

机构 * University of Twente（特文特大学） ； Utrecht University（乌特雷赫大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at the International Journal of Computer Vision (IJCV)

Uncertainty-Guided Self-Questioning and Answering for Video-Language Alignment

Jin Chen, Kaijing Ma, Haojian Huang, Han Fang, Hao Sun, Mehdi Hosseinzadeh, Zhe Liu

机构 * School of Computer Science, Duy Tan University（计算机科学学院，杜益坦大学） ； School of Computer Sciences, Universiti Sains Malaysia（计算机科学学院，马来亚大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

RAVU: Retrieval Augmented Video Understanding with Compositional Reasoning over Graph

Sameer Malik, Moyuru Yamada, Ayush Singh, Dishank Aggarwal

机构 * Fujitsu Research of India Private Limited（日本富士通印度研究私有限公司） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition

Xiao Wang, Yao Rong, Zongzhen Wu, Lin Zhu, Bo Jiang, Jin Tang, Yonghong Tian

机构 * School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院） ； Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院） ； Beijing Institute Of Technology（北京理工大学） ； Peng Cheng Laboratory（鹏城实验室） ； National Engineering Laboratory for Video Technology, School of Electronics Engineering and Computer Science, Peking University（视频技术国家工程实验室，北京大学电子工程与计算机科学学院） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by IEEE Transactions on Cognitive and Developmental Systems (TCDS) 2025

Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark

Enxin Song, Wenhao Chai, Weili Xu, Jianwen Xie, Yuxuan Liu, Gaoang Wang

机构 * Zhejiang University（浙江大学） ； University of Washington（华盛顿大学） ； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Lambda, Inc.（Lambda公司） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Code, docs, and benchmark are all avaliable at https://enxinsong.com/Video-MMLU-web/

GENMO: A GENeralist Model for Human MOtion

Jiefeng Li, Jinkun Cao, Haotian Zhang, Davis Rempe, Jan Kautz, Umar Iqbal, Ye Yuan

机构 * NVIDIA 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Project page: https://research.nvidia.com/labs/dair/genmo/

P2P-Insole: Human Pose Estimation Using Foot Pressure Distribution and Motion Sensors

Atsuya Watanabe, Ratna Aisuwarya, Lei Jing

机构 * Department of Computer Science and Engineering, University of Aizu（计算机科学与工程系，大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

HierSum: A Global and Local Attention Mechanism for Video Summarization

Apoorva Beedu, Irfan Essa

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Google DeepMind（谷歌DeepMind） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing

Gaoxiang Cong, Jiadong Pan, Liang Li, Yuankai Qi, Yuxin Peng, Anton van den Hengel, Jian Yang, Qingming Huang

机构 * Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所） ； Macquarie University（麦觉里大学） ； University of Chinese Academy of Sciences（中国科学院大学） ； Peking University（北京大学） ； University of Adelaide（阿德莱德大学） 纠错

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.MM、eess.AS

Comments Accepted to CVPR 2025

FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding

De-An Huang, Subhashree Radhakrishnan, Zhiding Yu, Jan Kautz

机构 * NVIDIA（英伟达） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering

Xingrui Wang, Wufei Ma, Angtian Wang, Shuo Chen, Adam Kortylewski, Alan Yuille

机构 * Johns Hopkins University（约翰霍普金斯大学） ； Tsinghua University（清华大学） ； Max Planck Institute for Informatics（马克斯·普朗克研究所（信息学）） ； University of Freiburg（弗赖堡大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments ICLR 2025 accepted paper. Project url: https://xingruiwang.github.io/projects/DynSuperCLEVR/

An LMM for Efficient Video Understanding via Reinforced Compression of Video Cubes

Ji Qi, Yuan Yao, Yushi Bai, Bin Xu, Juanzi Li, Zhiyuan Liu, Tat-Seng Chua

机构 * Tsinghua University（清华大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Phoenix: A Motion-based Self-Reflection Framework for Fine-grained Robotic Action Correction

Wenke Xia, Ruoxuan Feng, Dong Wang, Di Hu

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京耿丽人工智能学院） ； Shanghai AI Laboratory（上海人工智能实验室） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR2025

Prototypes are Balanced Units for Efficient and Effective Partially Relevant Video Retrieval

WonJun Moon, Cheol-Ho Cho, Woojin Jun, Minho Shim, Taeoh Kim, Inwoong Lee, Dongyoon Wee, Jae-Pil Heo

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

F$^3$Set: Towards Analyzing Fast, Frequent, and Fine-grained Events from Videos

Zhaoyu Liu, Kan Jiang, Murong Ma, Zhe Hou, Yun Lin, Jin Song Dong

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICLR 2025; Website URL: https://lzyandy.github.io/f3set-website/

Improved Visual-Spatial Reasoning via R1-Zero-Like Training

Zhenyi Liao, Qingsong Xie, Yanhao Zhang, Zijian Kong, Haonan Lu, Zhenyu Yang, Zhijie Deng

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding

Yangliu Hu, Zikai Song, Na Feng, Yawei Luo, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR2025

Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level

Andong Deng, Tongjia Chen, Shoubin Yu, Taojiannan Yang, Lincoln Spencer, Yapeng Tian, Ajmal Saeed Mian, Mohit Bansal, Chen Chen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025

Comparative Analysis of Image, Video, and Audio Classifiers for Automated News Video Segmentation

Jonathan Attard, Dylan Seychell

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Preprint for paper in CAI 2025, 7 pages, 5 tables, 3 tables

Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding

Duo Zheng, Shijia Huang, Liwei Wang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by CVPR 2025

M-LLM Based Video Frame Selection for Efficient Video Understanding

Kai Hu, Feng Gao, Xiaohan Nie, Peng Zhou, Son Tran, Tal Neiman, Lingyun Wang, Mubarak Shah, Raffay Hamid, Bing Yin, Trishul Chilimbi

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

PAVE: Patching and Adapting Video Large Language Models

Zhuoming Liu, Yiquan Li, Khoi Duc Nguyen, Yiwu Zhong, Yin Li

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.AI

Comments CVPR2025 Camera Ready

STA-V2A: Video-to-Audio Generation with Semantic and Temporal Alignment

Yong Ren, Chenxing Li, Manjie Xu, Wei Liang, Yu Gu, Rilin Chen, Dong Yu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering

Sheng Zhou, Junbin Xiao, Qingyun Li, Yicong Li, Xun Yang, Dan Guo, Meng Wang, Tat-Seng Chua, Angela Yao

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by CVPR 2025

SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysis

Junho Kim, Hyunjun Kim, Hosu Lee, Yong Man Ro

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Project page: https://ivy-lvlm.github.io/SALOVA/

ROCKET-1: Mastering Open-World Interaction with Visual-Temporal Context Prompting

Shaofei Cai, Zihao Wang, Kewei Lian, Zhancun Mu, Xiaojian Ma, Anji Liu, Yitao Liang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI