多模态大模型

SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models

Lee Hyun, Kim Sung-Bin, Seungju Han, Youngjae Yu, Tae-Hyun Oh

专题命中 视频多模态 ：multimodal(title)；分类 cs.CL、cs.AI

Comments 19 pages, 14 figures

VidCoM: Fast Video Comprehension through Large Language Models with Multimodal Tools

Ji Qi, Kaixuan Ji, Jifan Yu, Duokang Wang, Bin Xu, Lei Hou, Juanzi Li

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.CL

Multimodal Attack Detection for Action Recognition Models

Furkan Mumcu, Yasin Yilmaz

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.AI

GSGFormer: Generative Social Graph Transformer for Multimodal Pedestrian Trajectory Prediction

Zhongchang Luo, Marion Robin, Pavan Vasishta

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Localizing Moments in Long Video Via Multimodal Guidance

Wayner Barrios, Mattia Soldan, Alberto Mario Ceballos-Arroyo, Fabian Caba Heilbron, Bernard Ghanem

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Journal ref Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2023

Video Pre-trained Transformer: A Multimodal Mixture of Pre-trained Experts

Kastan Day, Daniel Christl, Rohan Salvi, Pranav Sriram

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation

Sayontan Ghosh, Tanvi Aggarwal, Minh Hoai, Niranjan Balasubramanian

专题命中 视频多模态 ：cross-modal(title)；分类 cs.CV、cs.CL

Multimodal active speaker detection and virtual cinematography for video conferencing

Ross Cutler, Ramin Mehran, Sam Johnson, Cha Zhang, Adam Kirk, Oliver Whyte, Adarsh Kowdle

专题命中 视频多模态 ：multimodal(title)；分类 cs.MM、eess.AS

Cross-modal Learning of Graph Representations using Radar Point Cloud for Long-Range Gesture Recognition

Souvik Hazra, Hao Feng, Gamze Naz Kiprit, Michael Stephan, Lorenzo Servadei, Robert Wille, Robert Weigel, Avik Santra

专题命中 视频多模态 ：cross-modal(title)；分类 cs.CV、cs.AI

Comments Accepted by IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM 2022)

Induce, Edit, Retrieve: Language Grounded Multimodal Schema for Instructional Video Retrieval

Yue Yang, Joongwon Kim, Artemis Panagopoulou, Mark Yatskar, Chris Callison-Burch

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.CL

Multi-modal Ensemble Models for Predicting Video Memorability

Tony Zhao, Irving Fang, Jeffrey Kim, Gerald Friedland

专题命中 视频多模态 ：multi-modal(title)；分类 cs.AI、cs.MM

Multimodal Gait Recognition for Neurodegenerative Diseases

Aite Zhao, Jianbo Li, Junyu Dong, Lin Qi, Qianni Zhang, Ning Li, Xin Wang, Huiyu Zhou

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Deep Learning based Multi-Modal Sensing for Tracking and State Extraction of Small Quadcopters

Zhibo Zhang, Chen Zeng, Maulikkumar Dhameliya, Souma Chowdhury, Rahul Rai

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV、cs.AI

Towards Improved Human Action Recognition Using Convolutional Neural Networks and Multimodal Fusion of Depth and Inertial Sensor Data

Zeeshan Ahmad, Naimul Khan

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.AI

AI in Pursuit of Happiness, Finding Only Sadness: Multi-Modal Facial Emotion Recognition Challenge

Carl Norman

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV、eess.AS

Procedural Reasoning Networks for Understanding Multimodal Procedures

Mustafa Sercan Amac, Semih Yagcioglu, Aykut Erdem, Erkut Erdem

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.CL

Comments Accepted to CoNLL 2019. The project website with code and demo is available at https://hucvl.github.io/prn/

Multi-Modal Three-Stream Network for Action Recognition

Muhammad Usman Khalid, Jie Yu

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CV、cs.AI

Comments Presented in IEEE ICPR 2018

MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics

Xinchen Yan, Akash Rastogi, Ruben Villegas, Kalyan Sunkavalli, Eli Shechtman, Sunil Hadap, Ersin Yumer, Honglak Lee

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.AI

Comments Published at ECCV 2018

Topic Modeling Based Multi-modal Depression Detection

Yuan Gong, Christian Poellabauer

专题命中 视频多模态 ：multi-modal(title)；分类 cs.CL、eess.AS

Comments Proceedings of the 7th Audio/Visual Emotion Challenge and Workshop (AVEC). (Official Depression Challenge Winner)

Multimodal Content Analysis for Effective Advertisements on YouTube

Nikhita Vedula, Wei Sun, Hyunhwan Lee, Harsh Gupta, Mitsunori Ogihara, Joseph Johnson, Gang Ren, Srinivasan Parthasarathy

专题命中 视频多模态 ：multimodal(title)；分类 cs.AI、cs.MM

Comments 11 pages, 5 figures, ICDM 2017

Automatic Curation of Golf Highlights using Multimodal Excitement Features

Michele Merler, Dhiraj Joshi, Quoc-Bao Nguyen, Stephen Hammer, John Kent, John R. Smith, Rogerio S. Feris

专题命中 视频多模态 ：multimodal(title)；分类 cs.CV、cs.MM

Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding

保持简洁：用于超长视频理解的多键情景记忆检索

Yeeun Choi, Youngbeom Yoo, Joon-Young Lee, Hyolim Kang, Seon Joo Kim

机构 * Yonsei University（延世大学） ； Adobe Research（奥多比研究院） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；MLLM(abstract_cn)；分类 cs.CV、cs.CL、cs.AI

AI总结 针对超长视频理解的两阶段范式需求，提出MERIT框架，通过多键表示与按需时间扩展实现精准检索，在三个长视频基准数据集上取得最优性能。

Comments Accepted to ECCV 2026 (Oral). Project Page: https://choi-yeeun.github.io/MERIT/

Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding

主动视频感知：用于代理长视频理解的迭代证据寻求

Ziyang Wang, Honglu Zhou, Shijie Wang, Junnan Li, Caiming Xiong, Silvio Savarese, Mohit Bansal, Michael S. Ryoo, Juan Carlos Niebles

机构 * Salesforce AI Research（Salesforce AI研究院） ； University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校） 纠错

专题命中 视频多模态 ：MLLM(abstract,abstract_cn)；分类 cs.CV、cs.CL、cs.AI

AI总结 本文提出了一种主动视频感知框架AVP，通过迭代计划-观察-反思过程，主动决定视频内容的观察目标和时间，以提高长视频理解的准确性和效率。

Comments Website: https://activevideoperception.github.io/

Continual Visual and Verbal Learning Through a Child's Egocentric Input

通过儿童自我中心输入进行持续的视觉与语言学习

Xiaoyang Jiang, Yanlai Yang, Kenneth A. Norman, Brenden Lake, Mengye Ren

机构 * Agentic Learning AI Lab, New York University（代理学习人工智能实验室，纽约大学） ； Department of Psychology, Princeton University（心理学系，普林斯顿大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 提出BabyCL持续多模态学习框架，在单一时间顺序处理SAYCam数据集，通过流式视觉表示学习和图像-文本对比目标，在SAYCam Labeled-S 4AFC基准上优于流式学习基线，缩小了与离线训练上限的差距。

Comments 15 pages, 4 figures

PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warning

PaSBench-Video: 面向主动安全预警的流式视频基准

Yusong Zhao, Yuejin Xie, Youliang Yuan, Junjie Hu, Jitian Guo, Yujiu Yang, Pinjia He

机构 * The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Tsinghua University（清华大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract_cn)；分类 cs.CV、cs.CL、cs.AI

AI总结 提出PaSBench-Video基准，包含740个视频，评估多模态大模型在危险发生前及时发出预警的能力，发现现有模型在时序精度和低误报率上表现不佳。

Hierarchical Local-Global Transformer for Temporal Sentence Grounding

层次化局部-全局Transformer用于时间语句定位

Xiang Fang, Daizong Liu, Pan Zhou, Zichuan Xu, Ruixuan Li

机构 * Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology（大数据安全湖北工程研究中心，华中科技大学网络安全科学与工程学院） ； Wangxuan Institute of Computer fTechnology, Peking University（王宣计算机技术研究院，北京大学） ； School of software, Dalian University of Technology（软件学院，大连理工大学） ； School of Computer Science, and Technology, Huazhong University of Science, and Technology（计算机科学与技术学院，华中科技大学） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL、cs.MM

AI总结 提出层次化局部-全局Transformer（HLGT），通过建模视频和查询的不同粒度层次及跨模态交互，实现更细粒度的多模态表示，并在三个数据集上取得最先进性能。

Comments Publish in IEEE Transactions on Multimedia

DisImpact: Quantifying the Physi-Social Impact of Natural Disasters Through Social Media

DisImpact：通过社交媒体量化自然灾害的生理-社会影响

Ruichen Yao, Tejna Dasari, Xuanyu Meng, Elliot Cao, Zelin Li, Yifan Liu, Yaokun Liu, Lanyu Shang, Dong Wang

专题命中 视频多模态 ：MLLM(abstract,abstract_cn)；multimodal(abstract)

AI总结 本文提出DisImpact框架，利用多模态大语言模型系统量化自然灾害的生理和社会影响，通过社交媒体内容分类和构建灾害影响指数，实现对灾害影响的统一表示和分析。

Comments Accepted by ICWSM 2026

LPM 1.0: Video-based Character Performance Model

LPM 1.0：基于视频的角色表现模型

Ailing Zeng, Casper Yang, Chauncey Ge, Eddie Zhang, Garvey Xu, Gavin Lin, Gilbert Gu, Jeremy Pi, Leo Li, Mingyi Shi, Shawn Wang, Sheng Bi, Steven Tang, Thorn Hang, Tobey Guo, Vincent Li, Xin Tong, Yikang Li, Yuchen Sun, Yue Zhao, Yuhan Lu, Yuwei Li, Zane Zhang, Zeshi Yang, Zi Ye

专题命中 视频多模态 ：multimodal(abstract)；audio-visual(abstract)；分类 cs.CV、cs.AI、cs.MM

AI总结 本文提出LPM 1.0模型，通过构建多模态数据集和训练扩散变换器，实现高可控性和身份一致性的角色表现生成，支持实时交互和无限长度生成。

Comments 43 pages, 15 figures, 2 tables. Project page: https://large-performance-model.github.io

Small Vision-Language Models are Smart Compressors for Long Video Understanding

小视觉-语言模型是长视频理解的智能压缩器

Junjie Fei, Jun Chen, Zechun Liu, Yunyang Xiong, Chong Zhou, Wei Wen, Junlin Han, Mingchen Zhuge, Saksham Suri, Qi Qian, Shuming Liu, Lemeng Wu, Raghuraman Krishnamoorthi, Vikas Chandra, Mohamed Elhoseiny, Chenchen Zhu

机构 * Meta AI ； King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 本文提出Tempo框架，利用小视觉-语言模型压缩长视频，通过自适应令牌分配实现高效压缩，实验显示其在极端长视频任务中表现优异。

Comments Project page and demo are available at https://FeiElysia.github.io/tempo-page/

EVA: Efficient Reinforcement Learning for End-to-End Video Agent

EVA: 为端到端视频代理的高效强化学习

Yaolun Zhang, Ruohui Wang, Jiahao Wang, Yepeng Tang, Xuanyu Zheng, Haonan Duan, Hao Lu, Hanming Deng, Lewei Lu

机构 * SenseTime Research（商汤科技研究院） 纠错

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 EVA通过迭代总结-计划-行动-反思推理实现先规划后感知，提升长视频理解效率，采用三阶段学习流程和高质量数据集，在六个视频理解基准上取得显著提升。

Comments CVPR2026