多模态大模型

Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Grounding

Yang Jin, Yongzhi Li, Zehuan Yuan, Yadong Mu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 18 pages, 7 figures, Accepted by Neurips 2022, Spotlight Presentation

AVE-CLIP: AudioCLIP-based Multi-window Temporal Transformer for Audio Visual Event Localization

Tanvir Mahmud, Diana Marculescu

专题命中 视频多模态 ：multi-modal(abstract)；audio-visual(abstract)；分类 cs.CV

Comments Accepted in IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023 (10 Pages, 5 Figures)

AOE-Net: Entities Interactions Modeling with Adaptive Attention Mechanism for Temporal Action Proposals Generation

Khoa Vo, Sang Truong, Kashu Yamazaki, Bhiksha Raj, Minh-Triet Tran, Ngan Le

专题命中 视频多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted for publication in International Journal of Computer Vision

Boosting Video-Text Retrieval with Explicit High-Level Semantics

Haoran Wang, Di Xu, Dongliang He, Fu Li, Zhong Ji, Jungong Han, Errui Ding

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by ACMMM 2022

Reducing the Vision and Language Bias for Temporal Sentence Grounding

Daizong Liu, Xiaoye Qu, Wei Hu

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by ACM MM 2022

SVGraph: Learning Semantic Graphs from Instructional Videos

Madeline C. Schiappa, Yogesh S. Rawat

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 20 pages, 27 figures

Online Video Instance Segmentation via Robust Context Fusion

Xiang Li, Jinglu Wang, Xiaohao Xu, Bhiksha Raj, Yan Lu

专题命中 视频多模态 ：multi-modal(abstract)；audio-visual(abstract)；分类 cs.CV

Depth-Cooperated Trimodal Network for Video Salient Object Detection

Yukang Lu, Dingyao Min, Keren Fu, Qijun Zhao

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 5 pages, 3 figures, Accepted at ICIP-2022

LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling

Linjie Li, Zhe Gan, Kevin Lin, Chung-Ching Lin, Zicheng Liu, Ce Liu, Lijuan Wang

专题命中 视频多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Temporal Aggregation for Adaptive RGBT Tracking

Zhangyong Tang, Tianyang Xu, Xiao-Jun Wu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 12 pages, 10 figures

Memory-Guided Semantic Learning Network for Temporal Sentence Grounding

Daizong Liu, Xiaoye Qu, Xing Di, Yu Cheng, Zichuan Xu, Pan Zhou

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by AAAI2022

Align and Prompt: Video-and-Language Pre-training with Entity Prompts

Dongxu Li, Junnan Li, Hongdong Li, Juan Carlos Niebles, Steven C. H. Hoi

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

ActionCLIP: A New Paradigm for Video Action Recognition

Mengmeng Wang, Jiazheng Xing, Yong Liu

专题命中 视频多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Deconfounded Video Moment Retrieval with Causal Intervention

Xun Yang, Fuli Feng, Wei Ji, Meng Wang, Tat-Seng Chua

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments This work has been accepted by SIGIR 2021

CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval

Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, Tianrui Li

专题命中 视频多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Siamese Network for RGB-D Salient Object Detection and Beyond

Keren Fu, Deng-Ping Fan, Ge-Peng Ji, Qijun Zhao, Jianbing Shen, Ce Zhu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2004.08515

Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection

Duygu Sarikaya, Jason J. Corso, Khurshid A. Guru

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Journal ref IEEE Transactions on Medical Imaging 36 (2017) 1542-1549

Learning Modality Interaction for Temporal Sentence Localization and Event Captioning in Videos

Shaoxiang Chen, Wenhao Jiang, Wei Liu, Yu-Gang Jiang

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

OVC-Net: Object-Oriented Video Captioning with Temporal Graph and Detail Enhancement

Fangyi Zhu, Jenq-Neng Hwang, Zhanyu Ma, Guang Chen, Jun Guo

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Video Highlight Prediction Using Audience Chat Reactions

Cheng-Yang Fu, Joon Lee, Mohit Bansal, Alexander C. Berg

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2017

Compact Tensor Pooling for Visual Question Answering

Yang Shi, Tommaso Furlanello, Anima Anandkumar

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Integrating Large Language Models into a Tri-Modal Architecture for Automated Depression Classification on the DAIC-WOZ

Santosh V. Patapati

专题命中 视频多模态 ：multi-modal(abstract,comments)；分类 cs.CV、cs.AI、cs.MM

Comments Keywords: Multi-Modal Neural Networks, Deep Learning, Large Language Models, Depression Diagnosis, Biomedical Informatics, DAIC-WOZ

EEG2MOTION: Towards Open-Vocabulary Human Motion Synthesis from Non-invasive Brain Signals

EEG2MOTION：面向基于无创脑信号的开放词汇人体动作合成

Yulong Peng, Yijian Pan, Yuqi Yang, Nenggan Zheng, Weidong Chen, Xiaoling Hu, Shaomin Zhang

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)

AI总结 研究人员推出首个EEG-动作-文本数据集EEG2MOTION，提出EEG条件掩码动作模型EMMM，实现从无创脑信号生成多样连贯的全身体人体动作，为生成式开放词汇运动BCI开辟新方向。

Deep Thought Alignment: Trajectory-Level Latent Distillation for Video Reasoning

深度思维对齐：面向视频推理的轨迹级潜在知识蒸馏

Ao Shen, Yongheng Zhang, Yinghui Li, Manning Wang, Di Yin, Xing Sun

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 针对视频推理LMMs的高计算成本问题，提出Latent-OPD方法，通过轨迹级潜在知识蒸馏与渐进式教师前瞻策略提升轻量模型性能，在6个基准上优于仅输出监督的OPD。

MobileMem: Learning from a Year of Mobile Experiences

MobileMem：从一年的移动体验中学习

Xinle Deng, Yida Xue, Xiangyuan Ru, Yijun Chen, Buqiang Xu, Mingjun Mao, Xinjie Liu, Haoming Xu, Shuofei Qiao, Mengru Wang, Chen Jiang, Yuchen Eleanor Jiang, Lizhong Wang, Jason Wang, Li Zeng, Haofen Wang, Guilin Qi, Huajun Chen, Ningyu Zhang

机构 * OPPO ； OpenKG 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI、cs.MM

AI总结 该研究推出MobileMem基准与框架，基于一年移动体验构建，用于设备端长期记忆研究，支持多类推理，推动智能体从信息检索向体验智能发展。

Comments Technical Report; Project Page: http://mobilemem.openkg.cn/

From Prompts to Pavement Through Time: Temporal Grounding in Agentic Scene-to-Plan Reasoning

从提示到路面通过时间：代理场景到计划推理中的时间定位

Ahmed Y. Gado, Omar Y. Goba, Alaa Hassanein, Catherine M. Elias, Ahmed Hussein

机构 * Computer Science & Engineering Department, German University in Cairo (GUC), Egypt（德国亚历山大大学（GUC）计算机科学与工程系，埃及） ； C-DRiVeS Lab: Cognitive Driving Research in Vehicular Systems, Cairo, Egypt（认知驾驶系统实验室，埃及开罗，C-DRiVeS） ； M.Eng. Robotics Candidate at Deggendorf Institute of Technology, Germany（德国德格多夫技术学院机器人硕士候选人） ； IAV GmbH, Berlin, Germany（德国柏林IAV GmbH公司） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 本研究探讨了在代理间通信中引入时间条件是否能保持或增强推理的一致性，而不会降低语义或逻辑一致性，并通过BDD-X数据集的curated子集评估了三种具有递增时间整合的规划器架构。结果表明，时间条件改变了推理风格，但并未在标准NLP正确性指标上产生统计显著改进，但定性分析揭示了预测危险推理、稳定纠正行为和战略分歧。

Towards Expert-level Medical AI for Real-time Video Consultations

面向专家级的实时视频问诊医疗AI

Mahvish Nagda, Jihyeon Lee, Matthew Thompson, Chunjong Park, Tim Strother, Valentin Liévin, Roma Ruparel, Akshay Goel, Teya Bergamaschi, Suhana Bedi, Meet Shah, Pavel Dubov, Liviu Panait, Toshiyuki Fukuzawa, Sam Schmidgall, Craig Schiff, Joseph Xu, Aliya Rysbek, Yana Lunts, Jan Freyberg, Rebecca Hemengway, Sunny Virmani, David Racz, Carey Radebaugh, Joëlle Barral, Kavi Goel, Dale R. Webster, Katherine Chou, Avinatan Hassidim, Yossi Matias, James Manyika, Gregory Wayne, Tao Tu, Yun Liu, Ethan Goh, Christina Chen, Ryutaro Tanno, Po-Hsuan Cameron Chen, Mike Schaekermann, Anil Palepu

机构 * Google Research（谷歌研究院） ； Google DeepMind（谷歌DeepMind） 纠错

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 本研究开发了基于Gemini的多智能体系统AMIE（视频版），在随机OSCE研究中其临床问诊表现与初级保健医生相当或更优，为专家级实时视频问诊医疗AI的发展奠定了重要基础。

BioKD: Selective Physiology-to-Video Knowledge Distillation via Reliability Gate for Emotion Recognition

BioKD：通过可靠性门实现面向情感识别的选择性生理信号到视频的知识蒸馏

Bojing Hou, Ruohao Li, Yitong Zhu, Hongjun Liu, Luwen Yu, Yuyang Wang

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； New York University（纽约大学） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)

AI总结 本文提出BioKD框架，以生理信号为训练特权信息指导视频学生模型，通过可靠性门控抑制负迁移，在DEAP、AMIGOS数据集上的情感识别任务中优于基线，且推理无额外开销。

A Hybrid Deterministic Framework for Named Entity Extraction in Broadcast News Video

一种混合确定性框架用于广播新闻视频中的命名实体提取

Andrea Filiberto Lucas, Dylan Seychell

机构 * Dept. of Artificial Intelligence University of Malta Msida, Malta（人工智能系 马耳他大学 Msida） 纠错

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

AI总结 本文提出一种混合确定性框架，用于自动提取广播新闻视频中的个人姓名，通过可解释的模块化管道实现高精度和可追溯性，适用于新闻媒体信息提取任务。

Comments 7 pages, 5 figures. Accepted for publication at the 2026 IEEE Conference on Artificial Intelligence (CAI)

Journal ref 2026 IEEE Conference on Artificial Intelligence (CAI), 2026, pp. 1134-1139

VisTR: Visualizations as Representations for Time-series Table Reasoning

VisTR：将可视化作为时间序列表推理的表示

Jianing Hao, Zhuowen Liang, Chunting Li, Yuyu Luo, Jie Li, Wei Zeng

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)

AI总结 针对时间序列表推理难题，VisTR框架以可视化核心，利用其连接数据与认知，通过多模态大语言模型对齐输入，集成机制处理大数据，实现交互式可视化，经评估和案例验证了其在时间序列推理任务中的有效性和适用性。

Comments 15 pages, 10 figures