多模态大模型

CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysis

CrashChat: 一种多模态大语言模型用于多任务交通事故视频分析

Kaidi Liang, Ke Li, Xianbiao Hu, Ruwen Qin

机构 * Stony Brook University（石溪大学） ； The Pennsylvania State University（宾夕法尼亚州立大学） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

AI总结 CrashChat是一种多模态大语言模型，用于多任务交通事故视频分析，通过任务解耦和分组策略提升事故识别、定位等任务的性能。

EDVD-LLaMA: Explainable Deepfake Video Detection via Multimodal Large Language Model Reasoning

EDVD-LLaMA: 通过多模态大语言模型推理进行可解释的深度伪造视频检测

Haoran Sun, Chen Cai, Huiping Zhuang, Kong Aik Lee, Lap-Pui Chau, Yi Wang

机构 * Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University（电子与电气工程系，香港理工大学） ； School of Electrical and Electronic Engineering, Nanyang Technological University（电子与电气工程学院，南洋理工大学） ； Shien-Ming Wu School of Intelligent Engineering, South China University of Technology（智能工程学院，华南理工大学） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

AI总结 EDVD-LLaMA通过多模态大语言模型推理实现深度伪造视频检测的可解释性，结合时空特征提取和细粒度推理机制，提升检测准确性与解释可靠性。

TEn-CATG:Text-Enriched Audio-Visual Video Parsing with Multi-Scale Category-Aware Temporal Graph

Yaru Chen, Faegheh Sardari, Peiliang Zhang, Ruohao Guo, Yang Xiang, Zhenbo Li, Wenwu Wang

机构 * Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey（视觉、语音和信号处理中心（CVSSP），萨里大学） ； School of Computer Science and Artificial Intelligence, Wuhan University of Technology（计算机科学与人工智能学院，武汉理工大学） ； National Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（通用人工智能国家重点实验室，北京大学智能科学与技术学院） ； College of Information and Electrical Engineering, China Agricultural University（信息与电子工程学院，中国农业大学） 纠错

专题命中 视频多模态 ：audio-visual(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

LongInsightBench: A Comprehensive Benchmark for Evaluating Omni-Modal Models on Human-Centric Long-Video Understanding

ZhaoYang Han, Qihan Lin, Hao Liang, Bowen Chen, Zhou Liu, Wentao Zhang

机构 * Huazhong University of Science and Technology（华中科技大学） ； Peking University（北京大学） 纠错

专题命中 视频多模态 ：omni-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Submitted to ARR Rolling Review

Enrich and Detect: Video Temporal Grounding with Multimodal LLMs

Shraman Pramanick, Effrosyni Mavroudi, Yale Song, Rama Chellappa, Lorenzo Torresani, Triantafyllos Afouras

机构 * FAIR, Meta（FAIR、Meta） ； Johns Hopkins University（约翰霍普金斯大学） ； Northeastern University（东北大学） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.MM

Comments ICCV 2025 (Highlights)

V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning

Hang Hua, Yolo Yunlong Tang, Chenliang Xu, Jiebo Luo

专题命中 视频多模态 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to AAAI 2025

Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of Images

Boyang Deng, Songyou Peng, Kyle Genova, Gordon Wetzstein, Noah Snavely, Leonidas Guibas, Thomas Funkhouser

机构 * Stanford University（斯坦福大学） ； Google DeepMind（谷歌DeepMind） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2025, Project page: https://boyangdeng.com/visual-chronicles , second and third listed authors have equal contributions

EventVL: Understand Event Streams via Multimodal Large Language Model

Pengteng Li, Yunfan Lu, Pinghao Song, Wuyang Li, Huizai Yao, Hui Xiong

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； KU Leuven（根特大学） ； École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院） ； Carleton University（卡尔顿大学） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Model

Pengteng Li, Pinhao Song, Wuyang Li, Weiyu Guo, Huizai Yao, Yijie Xu, Dugang Liu, Hui Xiong

机构 * HKUST(GZ)（香港科技大学（广州）） ； KU Leuven（比利时鲁文大学） ； EPFL（苏黎世联邦理工学院） ； SZU（深圳大学） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2025

E-CaTCH: Event-Centric Cross-Modal Attention with Temporal Consistency and Class-Imbalance Handling for Misinformation Detection

Ahmad Mousavi, Yeganeh Abdollahinejad, Roberto Corizzo, Nathalie Japkowicz, Zois Boukouvalas

机构 * Department of Mathematics and Statistics, American University, Washington, DC, USA（数学与统计学系，美国大学，华盛顿特区，美国） ； Department of Computer Science and Mathematics, Pennsylvania State University, Harrisburg, PA, USA（计算机科学与数学系，宾夕法尼亚州立大学，哈里斯堡，宾夕法尼亚州，美国） ； Department of Computer Science, American University, Washington, DC, USA（计算机科学系，美国大学，华盛顿特区，美国） 纠错

专题命中 视频多模态 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CL、cs.AI

StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification

Yichen He, Yuan Lin, Jianchao Wu, Hanchong Zhang, Yuchen Zhang, Ruicheng Le

专题命中 视频多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Tempo-R0: A Video-MLLM for Temporal Video Grounding through Efficient Temporal Sensing Reinforcement Learning

Feng Yue, Zhaoxing Zhang, Junming Jiao, Zhengyu Liang, Shiwen Cao, Feifei Zhang, Rong Shen

专题命中 视频多模态 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

M$^3$-Med: A Benchmark for Multi-lingual, Multi-modal, and Multi-hop Reasoning in Medical Instructional Video Understanding

Shenxi Liu, Kan Li, Mingyang Zhao, Yuhang Tian, Bin Li, Shoujun Zhou, Hongliang Li, Fuxia Yang

机构 * Beijing Institute of Technology（北京理工大学） ； The Hong Kong Polytechnic University（香港理工大学） ； Chinese Academy of Sciences（中国科学院） ； Air Force Hospital of Southern Theater Command of PLA（中国人民解放军南部战区空军医院） ； Shenzhen Traditional Chinese Medicine Hospital（深圳中医药医院） 纠错

专题命中 视频多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 19 pages, 8 figures, 7 tables

Multimodal Alignment with Cross-Attentive GRUs for Fine-Grained Video Understanding

Namho Kim, Junhwa Kim

机构 * Korean Broadcasting System(KBS)（韩国广播系统） ； Department of Artificial Intelligence（人工智能系） ； Konyang University（全南大学） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

COEF-VQ: Cost-Efficient Video Quality Understanding through a Cascaded Multimodal LLM Framework

Xin Dong, Sen Jia, Ming Rui Wang, Yan Li, Zhenheng Yang, Bingfeng Deng, Hongyu Xiong

机构 * ByteDance Inc.（字节跳动公司） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

S4-Driver: Scalable Self-Supervised Driving Multimodal Large Language Modelwith Spatio-Temporal Visual Representation

Yichen Xie, Runsheng Xu, Tong He, Jyh-Jing Hwang, Katie Luo, Jingwei Ji, Hubert Lin, Letian Chen, Yiren Lu, Zhaoqi Leng, Dragomir Anguelov, Mingxing Tan

机构 * UC Berkeley（加州大学伯克利分校） ； Waymo LLC（Waymo公司） ； Cornell University（康奈尔大学） ； Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR2025; Project website: s4-driver.github.io

HaploOmni: Unified Single Transformer for Multimodal Video Understanding and Generation

Yicheng Xiao, Lin Song, Rui Yang, Cheng Cheng, Zunnan Xu, Zhaoyang Zhang, Yixiao Ge, Xiu Li, Ying Shan

机构 * Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学） ； ARC Lab, Tencent PCG（腾讯PCG ARC实验室） ； The University of Hong Kong（香港大学） ； Xi’an JiaoTong University（西安交通大学） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Art

Yiming Lei, Chenkai Zhang, Zeming Liu, Haitao Leng, Shaoguo Liu, Tingting Gao, Qingjie Liu, Yunhong Wang

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments 69 pages, 66 figures, accepted by ACL 2025

TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation

Ling You, Wenxuan Huang, Xinni Xie, Xiangyi Wei, Bangyan Li, Shaohui Lin, Yang Li, Changbo Wang

机构 * East China Normal University（东华大学） 纠错

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA

Hanrong Ye, Haotian Zhang, Erik Daxberger, Lin Chen, Zongyu Lin, Yanghao Li, Bowen Zhang, Haoxuan You, Dan Xu, Zhe Gan, Jiasen Lu, Yinfei Yang

专题命中 视频多模态 ：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICLR 2025

Cross-modal Causal Relation Alignment for Video Question Grounding

Weixing Chen, Yang Liu, Binglin Chen, Jiandong Su, Yongsen Zheng, Liang Lin

专题命中 视频多模态 ：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by CVPR 2025

Multimodal Action Quality Assessment

Ling-An Zeng, Wei-Shi Zheng

专题命中 视频多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments IEEE Transactions on Image Processing 2024

When Video Coding Meets Multimodal Large Language Models: A Unified Paradigm for Video Coding

Pingping Zhang, Jinlong Li, Kecheng Chen, Meng Wang, Long Xu, Haoliang Li, Nicu Sebe, Sam Kwong, Shiqi Wang

专题命中 视频多模态 ：multimodal(title,abstract)；image-text(abstract)；分类 cs.CV、cs.MM

Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models

Yifang Xu, Yunzhuo Sun, Benxiang Zhai, Ming Li, Wenxin Liang, Yang Li, Sidan Du

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.MM

Comments Accepted by AAAI 2025

H-MBA: Hierarchical MamBa Adaptation for Multi-Modal Video Understanding in Autonomous Driving

Siran Chen, Yuxiao Luo, Yue Ma, Yu Qiao, Yali Wang

专题命中 视频多模态 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 4 figures

Assessing Modality Bias in Video Question Answering Benchmarks with Multimodal Large Language Models

Jean Park, Kuk Jin Jang, Basam Alasaly, Sriharsha Mopidevi, Andrew Zolensky, Eric Eaton, Insup Lee, Kevin Johnson

专题命中 视频多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Experts

Chenyang Zhu, Bin Xiao, Lin Shi, Shoukun Xu, Xu Zheng

专题命中 视频多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Dreaming User Multimodal Representation Guided by The Platonic Representation Hypothesis for Micro-Video Recommendation

Chengzhi Lin, Hezheng Lin, Shuchang Liu, Cangguang Ruan, LingJing Xu, Dezhao Yang, Chuyuan Wang, Yongqi Liu

专题命中 视频多模态 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 4 Figure; 2 Table

LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models

Qihang Ge, Wei Sun, Yu Zhang, Yunhao Li, Zhongpeng Ji, Fengyu Sun, Shangling Jui, Xiongkuo Min, Guangtao Zhai

专题命中 视频多模态 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

A Multimodal Transformer for Live Streaming Highlight Prediction

Jiaxin Deng, Shiyao Wang, Dong Shen, Liqin Zhao, Fan Yang, Guorui Zhou, Gaofeng Meng

专题命中 视频多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted at ICME 2024 as poster presentation. arXiv admin note: text overlap with arXiv:2306.14392