多模态大模型

GAZE:Governance-Aware pre-annotation for Zero-shot World Model Environments

Leela Krishna, Mengyang Zhao, Saicharithreddy Pasula, Harshit Rajgarhia, Abhishek Mukherji

机构 * Centific Global Solutions Inc.（Centific全球解决方案公司） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Injecting Explainability and Lightweight Design into Weakly Supervised Video Anomaly Detection Systems

Wen-Dong Jiang, Chih-Yung Chang, Hsiang-Chuan Chang, Ji-Yuan Chen, Diptendu Sinha Roy

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

QuadKAN: KAN-Enhanced Quadruped Motion Control via End-to-End Reinforcement Learning

Yinuo Wang, Gavin Tao

机构 * Allen Wang Gavin Tao 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 14pages, 9 figures, Journal paper

TriPSS: A Tri-Modal Keyframe Extraction Framework Using Perceptual, Structural, and Semantic Representations

Mert Can Cakmak, Nitin Agarwal, Diwash Poudel

机构 * Computer and Information Science, University of Arkansas - Little Rock（计算机与信息科学，亚拉荷马州立大学） ； ICSI, University of California, Berkeley（ICSI，加州大学伯克利分校） ； COSMOS Research Center, University of Arkansas - Little Rock（COSMOS研究中心，亚拉荷马州立大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.MM

Consistent and Invariant Generalization Learning for Short-video Misinformation Detection

Hanghui Guo, Weijie Shi, Mengze Li, Juncheng Li, Hao Chen, Yue Cui, Jiajie Xu, Jia Zhu, Jiawei Shen, Zhangze Chen, Sirui Han

机构 * Zhejiang Normal University（浙江师范大学） ； Hong Kong University of Science and Technology（香港理工大学） ； Zhejiang University（浙江大学） ； Tencent（腾讯） ； Soochow University（苏州大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted to ACM MM 2025

EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understanding

Ashish Seth, Utkarsh Tyagi, Ramaneswaran Selvakumar, Nishit Anand, Sonal Kumar, Sreyan Ghosh, Ramani Duraiswami, Chirag Agarwal, Dinesh Manocha

机构 * University of Maryland, College Park（马里兰大学学院公园分校） ； University of Virginia（弗吉尼亚大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

LeAdQA: LLM-Driven Context-Aware Temporal Grounding for Video Question Answering

Xinxin Dong, Baoyun Peng, Haokai Ma, Yufei Wang, Zixuan Dong, Fei Hu, Xiaodong Wang

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Towards Universal Modal Tracking with Online Dense Temporal Token Learning

Yaozong Zheng, Bineng Zhong, Qihua Liang, Shengping Zhang, Guorong Li, Xianxian Li, Rongrong Ji

机构 * Key Laboratory of Education Blockchain and Intelligent Technology, Ministry of Education（教育区块链与智能技术重点实验室，教育部） ； Guangxi Key Laboratory of Multi-Source Information Mining and Security, Guangxi Normal University（广西多源信息挖掘与安全重点实验室，广西师范大学） ； School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院） ； University of Chinese Academy of Sciences（中国科学院大学） ； Media Analytics and Computing Lab, Department of Artificial Intelligence, School of Informatics, Xiamen University（媒体分析与计算实验室，人工智能系，厦门大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments arXiv admin note: text overlap with arXiv:2401.01686

Tell Me What to Track: Infusing Robust Language Guidance for Enhanced Referring Multi-Object Tracking

Wenjun Huang, Yang Ni, Hanning Chen, Yirui He, Ian Bryant, Yezi Liu, Mohsen Imani

机构 * University of California, Irvine（加州大学尔湾分校） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

RealGeneral: Unifying Visual Generation via Temporal In-Context Learning with Video Models

Yijing Lin, Mengqi Huang, Shuhan Zhuang, Zhendong Mao

机构 * University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Iterative Zoom-In: Temporal Interval Exploration for Long Video Understanding

Chenglin Li, Qianglong Chen, fengtao, Yin Zhang

机构 * Zhejiang University（浙江大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

CAVALRY-V: A Large-Scale Generator Framework for Adversarial Attacks on Video MLLMs

Jiaming Zhang, Rui Hu, Qing Guo, Wei Yang Bryan Lim

机构 * Nanyang Technological University（南洋理工大学） ； Beijing Jiaotong University（北京交通大学） ； A*STAR Project Page（A*STAR项目页面） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking

Shiao Wang, Ju Huang, Qingchuan Ma, Jinfeng Gao, Chunyi Xu, Xiao Wang, Lan Chen, Bo Jiang

机构 * School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院） ； School of Electronic and Information Engineering, Anhui University（安徽大学电子与信息工程学院） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Journal extension of Mamba-FETrack which was published on Pattern Recognition and Computer Vision (PRCV) 2024

Task-Aware KV Compression For Cost-Effective Long Video Understanding

Minghao Qin, Yan Shu, Peitian Zhang, Kun Lun, Huaying Yuan, Juenjie Zhou, Shitao Xiao, Bo Zhao, Zheng Liu

机构 * Beijing Academy of Artificial Intelligence（北京人工智能研究院） ； Shanghai Jiao Tong University（上海交通大学） ； University of Trento（特伦特大学） ； Renmin University of China（中国人民大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） ； Hong Kong Polytechnic University（香港理工大学） ； Institute of Automation CAS Beijing China（中国科学院自动化研究所） 纠错

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 3 figures, 6 tables

Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsification

Minghao Qin, Xiangrui Liu, Zhengyang Liang, Yan Shu, Huaying Yuan, Juenjie Zhou, Shitao Xiao, Bo Zhao, Zheng Liu

机构 * Beijing Academy of Artificial Intelligence（北京人工智能研究院） ； Shanghai Jiao Tong University（上海交通大学） ； University of Trento（特伦特大学） ； Renmin University of China（中国人民大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） ； Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 5 Figure, 3 Table

ScaleLong: A Multi-Timescale Benchmark for Long Video Understanding

David Ma, Huaqing Yuan, Xingjian Wang, Qianbo Zang, Tianci Liu, Xinyang He, Yanbin Wei, Jiawei Guo, Ni Jiahui, Zhenzhu Yang, Meng Cao, Shanghaoran Quan, Yizhi Li, Wangchunshu Zhou, Jiaheng Liu, Wenhao Huang, Ge Zhang, Shiwen Ni, Xiaojie Jin

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Universal Visuo-Tactile Video Understanding for Embodied Interaction

Yifan Xie, Mingyang Li, Shoujie Li, Xingting Li, Guangyu Chen, Fei Ma, Fei Richard Yu, Wenbo Ding

机构 * Tsinghua University（清华大学） ； Sun Yat-sen University（中山大学） ； Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济实验室（SZ）） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 13 pages, 5 figures

Predicting Implicit Arguments in Procedural Video Instructions

Anil Batra, Laura Sevilla-Lara, Marcus Rohrbach, Frank Keller

机构 * University of Edinburgh（爱丁堡大学） ； TU Darmstadt（德累斯顿理工大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments ACL 2025 Main

CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework

Wentao Wu, Xiao Wang, Chenglong Li, Bo Jiang, Jin Tang, Bin Luo, Qi Liu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

From Trial to Triumph: Advancing Long Video Understanding via Visual Context Sample Scaling and Self-reward Alignment

Yucheng Suo, Fan Ma, Linchao Zhu, Tianyi Wang, Fengyun Rao, Yi Yang

专题命中 视频多模态 ：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation

Kun Yuan, Vinkle Srivastav, Nassir Navab, Nicolas Padoy

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at the 38th Conference on Neural Information Processing Systems (NeurIPS 2024 Spolight)

The Devil is in Temporal Token: High Quality Video Reasoning Segmentation

Sitong Gong, Yunzhi Zhuge, Lu Zhang, Zongxin Yang, Pingping Zhang, Huchuan Lu

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Journal ref CVPR 2025

Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos

Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, Ziwei Liu

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition

Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, Wynne Hsu

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICML 2024

Personality Analysis from Online Short Video Platforms with Multi-domain Adaptation

Sixu An, Xiangguo Sun, Yicong Li, Yu Yang, Guandong Xu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Show and Guide: Instructional-Plan Grounded Vision and Language Model

Diogo Glória-Silva, David Semedo, João Magalhães

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at EMNLP 2024 Main Track

Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering

Ting Yu, Kunhao Fu, Shuhui Wang, Qingming Huang, Jun Yu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments IEEE Transactions on Circuits and Systems for Video Technology

Journal ref IEEE Transactions on Circuits and Systems for Video Technology, 2024

Encoding and Controlling Global Semantics for Long-form Video Question Answering

Thong Thanh Nguyen, Zhiyuan Hu, Xiaobao Wu, Cong-Duy T Nguyen, See-Kiong Ng, Anh Tuan Luu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to the main EMNLP 2024 conference

EAGLE: Egocentric AGgregated Language-video Engine

Jing Bi, Yunlong Tang, Luchuan Song, Ali Vosoughi, Nguyen Nguyen, Chenliang Xu

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACMMM 24

TC-LLaVA: Rethinking the Transfer from Image to Video Understanding with Temporal Considerations

Mingze Gao, Jingyu Liu, Mingda Li, Jiangtao Xie, Qingbin Liu, Bo Zhao, Xi Chen, Hui Xiong

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI