多模态大模型

MDMMT: Multidomain Multimodal Transformer for Video Retrieval

Maksim Dzabraev, Maksim Kalashnikov, Stepan Komkov, Aleksandr Petiushko

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV

Journal ref CVPR Workshops 2021: 3354-3363

Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning

Mandela Patrick, Yuki M. Asano, Bernie Huang, Ishan Misra, Florian Metze, Joao Henriques, Andrea Vedaldi

专题命中 视频多模态 ：cross-modal(title)；分类 cs.CV

Comments Accepted to ICCV 2021. Code at https://github.com/facebookresearch/GDT

Leveraging Local Temporal Information for Multimodal Scene Classification

Saurabh Sahu, Palash Goyal

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV

Modelling Urban Dynamics with Multi-Modal Graph Convolutional Networks

Krittika D'Silva, Jordan Cambe, Anastasios Noulas, Cecilia Mascolo, Adam Waksman

专题命中 视频多模态 ：multi-modal(title)；分类 cs.AI

Comments 10 pages, 4 figures. arXiv admin note: substantial text overlap with arXiv:2104.13981

Unified Multi-Modal Landmark Tracking for Tightly Coupled Lidar-Visual-Inertial Odometry

David Wisth, Marco Camurri, Sandipan Das, Maurice Fallon

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV

Comments Video: https://youtu.be/MjXYAHurWe8

Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging

He Sun, Katherine L. Bouman

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV

Comments This paper has been accepted to AAAI 2021. Keywords: Computational Imaging, Normalizing Flow, Uncertainty Quantification, Interferometry, MRI

Multimodal Memorability: Modeling Effects of Semantics and Decay on Video Memorability

Anelise Newman, Camilo Fosco, Vincent Casser, Allen Lee, Barry McNamara, Aude Oliva

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV

Comments European Conference on Computer Vision

A Modular Approach for Synchronized Wireless Multimodal Multisensor Data Acquisition in Highly Dynamic Social Settings

Chirag Raman, Stephanie Tan, Hayley Hung

专题命中 视频多模态 ：multimodal(title)；分类 cs.MM

Comments 9 pages, 8 figures, Proceedings of the 28th ACM International Conference on Multimedia (MM '20), October 12--16, 2020, Seattle, WA, USA. First two authors contributed equally

Multimodal and multiview distillation for real-time player detection on a football field

Anthony Cioppa, Adrien Deliège, Noor Ul Huda, Rikke Gade, Marc Van Droogenbroeck, Thomas B. Moeslund

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV

Comments Accepted for the CVSports workshop of CVPR 2020 ; 8 pages + references

Speech2Action: Cross-modal Supervision for Action Recognition

Arsha Nagrani, Chen Sun, David Ross, Rahul Sukthankar, Cordelia Schmid, Andrew Zisserman

专题命中 视频多模态 ：cross-modal(title)；分类 cs.CV

Comments Accepted to CVPR 2020

Scattering Features for Multimodal Gait Recognition

Srđan Kitić, Gilles Puy, Patrick Pérez, Philippe Gilberton

专题命中 视频多模态 ：multimodal(title)；分类 eess.AS

Comments Published at IEEE GlobalSIP 2017

Situational Grounding within Multimodal Simulations

James Pustejovsky, Nikhil Krishnaswamy

专题命中 视频多模态 ：multimodal(title)；分类 cs.AI

Comments AAAI-19 Workshop on Games and Simulations for Artificial Intelligence

Multi-modal Capsule Routing for Actor and Action Video Segmentation Conditioned on Natural Language Queries

Bruce McIntosh, Kevin Duarte, Yogesh S Rawat, Mubarak Shah

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV

Cross-Modal and Hierarchical Modeling of Video and Text

Bowen Zhang, Hexiang Hu, Fei Sha

专题命中 视频多模态 ：cross-modal(title)；分类 cs.CV

Comments Accepted by ECCV 2018

Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

Zhenzhou Shao, Hongfa Zhao, Jiexin Xie, Ying Qu, Yong Guan, Jindong Tan

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV

Comments 7 pages, 6 figures

Encoding Longer-term Contextual Multi-modal Information in a Predictive Coding Model

Junpei Zhong, Tetsuya Ogata, Angelo Cangelosi

专题命中 视频多模态 ：multi-modal(title)；分类 cs.AI

Comments Submitted to ICDL/EpiRob 2018 (8th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics )

Multi-modal Aggregation for Video Classification

Chen Chen, Xiaowei Zhao, Yang Liu

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV

A Computational Framework for Multi-Modal Social Action Identification

Jason Anastasopoulos, Jake Ryland Williams

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CL

A Nonparametric Model for Multimodal Collaborative Activities Summarization

Guy Rosman, John W. Fisher, Daniela Rus

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV

Summarization of ICU Patient Motion from Multimodal Multiview Videos

Carlos Torres, Kenneth Rose, Jeffrey C. Fried, B. S. Manjunath

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV

Comments Total of 17 pages: 1-12 (body), 13-16 (result figures), and 17(references) Number of figures: 21

PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding

Chunhui Liu, Yueyu Hu, Yanghao Li, Sijie Song, Jiaying Liu

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV

Comments 10 pages

Recognizing and Presenting the Storytelling Video Structure with Deep Multimodal Networks

Lorenzo Baraldi, Costantino Grana, Rita Cucchiara

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV

EmoNets: Multimodal deep learning approaches for emotion recognition in video

Samira Ebrahimi Kahou, Xavier Bouthillier, Pascal Lamblin, Caglar Gulcehre, Vincent Michalski, Kishore Konda, Sébastien Jean, Pierre Froumenty, Yann Dauphin, Nicolas Boulanger-Lewandowski, Raul Chandias Ferrari, Mehdi Mirza, David Warde-Farley, Aaron Courville, Pascal Vincent, Roland Memisevic, Christopher Pal, Yoshua Bengio

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV

Learning Multi-modal Similarity

Brian McFee, Gert Lanckriet

专题命中 视频多模态 ：multi-modal(title)；分类 cs.AI

DIMOS: Disentangling Instance-level Moving Object Segmentation

DIMOS: 解耦实例级运动目标分割

Hongxiang Huang, Hongwei Ren, Xiaopeng Lin, Yulong Huang, Zeke Xie, Bojun Cheng

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

AI总结 提出双解耦特征提取框架分离图像与事件模态的外观和运动信息，并通过多粒度跨模态对齐实现有效融合，在运动实例分割任务中尤其对快速运动和低光下的小目标取得最优性能。

I Seek You in Videos: Identity-Conditioned Queries for Person-Centric Video Reasoning

我在视频中寻找你：面向以人为中心的视频推理的身份条件查询

Shibo Gao, Chongxiao Wang, Chenglong Huang, Jie Ma, Haolin Shi, Fei Ding, Jing Li, Qiang Lyu, Yangyang Liu, Yang Liu, Jun Liu, Linlin Huang, Peipei Yang

机构 * Beijing Jiaotong University（北京交通大学） ； HUJING Digital Media & Entertainment Group（汇晶数字媒体与娱乐集团） ； MAIS Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所MAIS） 纠错

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract_cn)；分类 cs.CV、cs.AI

AI总结 该研究提出了身份条件查询（ICQ）任务，构建了ISYV基准、训练集与框架，实验显示主流多模态大语言模型在该任务上表现不佳，ISYV模型性能优于强基线且接近闭源模型。

Comments Accepted to ACM Multimedia 2026 (MM '26). 6 figures, 5 tables

ViSAGE: Constructing Self-Correcting Memories for Long-Form Video Understanding

ViSAGE：为长视频理解构建自修正记忆

Xinkui Zhao, Enbo Chen, Yifan Zhang, Chang Liu, Guanjie Cheng, Naibo Wang, Yueshen Xu

机构 * Zhejiang University（浙江大学） ； Xidian University（西安电子科技大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

AI总结 ViSAGE是一种多模态智能体记忆框架，通过跨模态绑定、双向记忆精调及多智能体交叉验证解决长视频理解中的实体混淆等问题，较最强基线准确率提升5.9%。

Comments Accept by ACMMM 2026

Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoning

Ouroboros-Spatial：闭环数据-模型循环的空间推理

Enhan Zhao, Wei Wu, Yuanrui Zhang, Xueliang Zhao, Di He

机构 * Peking University（北京大学） ； Ant International（蚂蚁国际） ； The University of Hong Kong（香港大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract_cn)；分类 cs.CV、cs.AI

AI总结 提出Ouroboros-Spatial自演化框架，通过提议器与求解器闭环交互，动态生成与模型能力匹配的训练样本，在六个空间推理基准上以十分之一数据量显著提升Qwen3-VL性能。

VideoAgent: All-in-One Framework for Video Understanding and Editing

VideoAgent: 视频理解与编辑的全能框架

Hengji Zhou, Lingxuan Huang, Jian Wang, Bing Zhou, Si Wu, Lianghao Xia, Chao Huang

机构 * South China University of Technology（华南理工大学） ； The University of Hong Kong（香港大学） ； Snap Inc（Snap公司） ； Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

AI总结 提出VideoAgent，一种基于多智能体编排的全能框架，通过自动镜头创建和30多个专业编辑智能体，解决现有系统在长视频理解和多样化编辑操作上的局限，在VideoEdit基准上实现87-95%编排成功率并降低60% API成本。

Comments Preprint. Code available at https://github.com/HKUDS/VideoAgent

Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task

增强工具的时空推理用于视频问答任务的优化

Sunqi Fan, Jiashuo Cui, Meng-Hao Guo, Shuojin Yang

机构 * BNRist, Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系，北京信息科学与技术国家研究中心） 纠错

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

AI总结 本文提出STAR框架和视频工具包，提升多模态大语言模型的时空推理能力，在VideoMME和LongVideoBench上分别提升8.2%和4.6%。

Comments Accepted by NeurIPS 2025 main track