多模态大模型

Multimodal Smart Glove for Sign Language Recognition Using Deep Learning

基于深度学习的用于手语识别的多模态智能手套

Anh Thu Nguyen Ngoc, Tam Phong Truong, Thai Anh Nguyen Duong, Vu Linh Nguyen, Manh Duong Phung

专题命中 视频多模态 ：multimodal(title)

AI总结 研究旨在解决手语识别系统实际部署难题，提出集成可穿戴传感与深度学习的智能手套系统，利用柔性传感器、IMU及摄像头采集数据，经LSTM网络处理，准确率约95%，模型转换后可实时推理，证明系统用于手语翻译的可行性。

Comments In Proceedings of IFToMM International Symposium on Robotics and Mechatronics

M-EPDet: Real-Time Real-Bogus Classification and Transient Candidate Judgement for the EP-WXT Pipeline via Multi-Modal Data

M-EPDet：基于多模态数据的EP-WXT流水线实时真实-虚假分类与瞬变候选判断

Lang Chen, Yunfei Xu, Zhen Zhang, Dongyue Li, Hui Sun, Yuan Liu, Chenzhou Cui, Jinhui Xie, XiaoXiong Zuo, Shirui Wei, Wujun Shao

专题命中 视频多模态 ：multi-modal(title)

AI总结 提出M-EPDet三阶段框架，结合ResNet臂滤波器、双分支时-谱宇宙射线滤波器和背景感知贝叶斯块模块，实现EP-WXT数据实时候选筛选，真实源召回率98.31%，候选量减少99.25%。

Comments 20 pages, 13 figures, accepted by ApJS

TibetCPR: A Multimodal Tactile Feedback System to Enhance Cardiopulmonary Resuscitation Training in High-Altitude Regions of Tibet

TibetCPR：一种增强高海拔地区心肺复苏训练的多模态触觉反馈系统

Yibo Meng, Ruiqi Chen, Zhiming Liu, Xiaolan Ding

专题命中 视频多模态 ：multimodal(title)

AI总结 针对西藏地区心肺复苏训练资源匮乏的问题，提出低成本自引导训练系统TibetCPR，结合深度电触觉与节奏视觉反馈，实验表明能显著稳定按压节奏与深度，并提炼出自引导训练的设计原则。

Comments Accept to MobileHCI 2026

Cross-modal characterization of infant cry: validation of a chest-surface accelerometer in extracting acoustic vocal function measures

婴儿哭声的跨模态表征：胸表加速度计在提取声学发声功能测量中的验证

Winko W. An, Saketh Sundar, Lisa Yankowitz, Daryush D. Mehta, Carol L. Wilkinson

机构 * Division of Developmental Medicine, Boston Children’s Hospital（发育医学部，波士顿儿童医院） ； Harvard Medical School（哈佛医学院） ； Harvard University（哈佛大学） ； Children’s Hospital of Philadelphia（费城儿童医院） ； Center for Laryngeal Surgery and Voice Rehabilitation, Massachusetts General Hospital（嗓音康复中心，麻省总医院） 纠错

专题命中 视频多模态 ：cross-modal(title)

AI总结 本研究验证了胸表加速度计在婴儿哭声分析中的有效性，发现其能可靠捕获基频和抖动等声学特征，为噪声鲁棒且保护隐私的临床研究提供替代方案。

STQuant: Spatio-Temporal Adaptive Framework for Optimizer Quantization in Large Multimodal Model Training

STQuant：一种用于大规模多模态模型训练中优化器量化的空间-时间自适应框架

Minglu Liu, Cunchen Hu, Liangliang Xu, Fengming Tang, Ruijia Wang, Fu Yu

机构 * Xidian University（西安电子科技大学） ； China Telecom Cloud Computing Research Institute（中国电信云计算研究院） 纠错

专题命中 视频多模态 ：multimodal(title)

AI总结 STQuant通过动态分配精度降低优化器状态内存，保持模型质量，实验显示内存减少84.4%，平均比特宽低至5.1。

The First Mass Protest on Threads: Multimodal Mobilization and AI-Generated Visuals in Taiwan's Bluebird Movement

台湾蓝鸟运动中的首次大规模抗议：多模态动员与AI生成视觉内容

Tracy Weener, Ho-Chun Herbert Chang

专题命中 视频多模态 ：multimodal(title)

AI总结 本文分析台湾2024年蓝鸟运动中文本与视觉信息的传播动态，揭示算法曝光与用户支持的不对称性及AI生成图像在抗议中的作用。

Harnessing Rich Multi-Modal Data for Spatial-Temporal Homophily-Embedded Graph Learning Across Domains and Localities

利用丰富的多模态数据进行跨领域和地域的时空同质性嵌入图学习

Takuya Kurihana, Xiaojian Zhang, Wing Yee Au, Hon Yung Wong

机构 * Fujitsu Research of America（富士通美国研究机构） ； Department of Civil and Coastal Engineering University of Florida（佛罗里达大学土木与海岸工程系） 纠错

专题命中 视频多模态 ：multi-modal(title)

AI总结 本文提出一种跨领域和地域的时空同质性嵌入图学习框架，通过整合多模态数据提升城市预测性能，解决智能城市分析中的关键挑战。

Comments 18 pages, 8 figures, Presented in part at the 2025 INFORMS Annual Meeting

Sleep Apnea Detection on a Wireless Multimodal Wearable Device Without Oxygen Flow Using a Mamba-based Deep Learning Approach

基于Mamba深度学习方法的无线多模态可穿戴设备无氧流睡眠呼吸暂停检测

Dominik Luszczynski, Richard Fei Yin, Nicholas Afonin, Andrew S. P. Lim

机构 * University of Toronto Department of Medicine（多伦多大学医学系） ； Sunnybrook Research Institute Department of Medicine Neurology Div.（圣玛丽医院研究学院医学系神经病学部） 纠错

专题命中 视频多模态 ：multimodal(title)

AI总结 本文提出基于Mamba架构的深度学习模型，利用无线多模态可穿戴设备无氧流信号，实现睡眠呼吸暂停的准确检测与事件级表征。

Comments 29 pages, 14 figures. Authors Dominik Luszczynski, Richard Fei Yin and Nicholas Afonin contributed equally

Spatio-Temporal Graph Convolutional Networks for EV Charging Demand Forecasting Using Real-World Multi-Modal Data Integration

Jose Tupayachi, Mustafa C. Camur, Kevin Heaslip, Xueping Li

机构 * University of Tennessee（田纳西大学） 纠错

专题命中 视频多模态 ：multi-modal(title)

Continental-scale habitat distribution modelling with multimodal earth observation foundation models

Sara Si-Moussi, Stephan Hennekens, Sander Mucher, Stan Los, Yoann Cartier, Borja Jiménez-Alfaro, Fabio Attorre, Jens-Christian Svenning, Wilfried Thuiller

专题命中 视频多模态 ：multimodal(title)

DeepTx: Real-Time Transaction Risk Analysis via Multi-Modal Features and LLM Reasoning

Yixuan Liu, Xinlei Li, Yi Li

专题命中 视频多模态 ：multi-modal(title)

Comments Accepted to ASE'25

Research on Short-Video Platform User Decision-Making via Multimodal Temporal Modeling and Reinforcement Learning

Jinmeiyang Wang, Jing Dong, Li Zhou

专题命中 视频多模态 ：multimodal(title)

Comments 26 pages

Task-Aware Tuning of Time Constants in Spiking Neural Networks for Multimodal Classification

Chiu-Chang Cheng, Kapil Bhardwaj, Ya-Ning Chang, Sayani Majumdar, Chao-Hung Wang

专题命中 视频多模态 ：multimodal(title)

Comments 25 Pages and 5 Figures and a supplementary discussion as well

Make Silence Speak for Itself: a multi-modal learning analytic approach with neurophysiological data

Mingxuan Gao, Jingjing Chen, Yun Long, Xiaomeng Xu, Yu Zhang

专题命中 视频多模态 ：multi-modal(title)

Comments 25 pages, 6 figures

Personalized Predictions of Glioblastoma Infiltration: Mathematical Models, Physics-Informed Neural Networks and Multimodal Scans

Ray Zirui Zhang, Ivan Ezhov, Michal Balcerak, Andy Zhu, Benedikt Wiestler, Bjoern Menze, John S. Lowengrub

专题命中 视频多模态 ：multimodal(title)

Force-Constrained Visual Policy: Safe Robot-Assisted Dressing via Multi-Modal Sensing

Zhanyi Sun, Yufei Wang, David Held, Zackory Erickson

专题命中 视频多模态 ：multi-modal(title)

Cross-temporal Detection of Novel Ransomware Campaigns: A Multi-Modal Alert Approach

Sathvik Murli, Dhruv Nandakumar, Prabhat Kumar Kushwaha, Cheng Wang, Christopher Redino, Abdul Rahman, Shalini Israni, Tarun Singh, Edward Bowen

专题命中 视频多模态 ：multi-modal(title)

Comments Preprint. Under Review

CAESURA: Language Models as Multi-Modal Query Planners

Matthias Urban, Carsten Binnig

专题命中 视频多模态 ：multi-modal(title)

Comments 6 pages, 4 figures

Adaptive Transit Design: Optimizing Fixed and Demand Responsive Multi-Modal Transportation via Continuous Approximation

Giovanni Calabro', Andrea Araldo, Simon Oh, Ravi Seshadri, Giuseppe Inturri, Moshe Ben-Akiva

专题命中 视频多模态 ：multi-modal(title)

Comments 57 pages, 13 figures

Smart-Badge: A wearable badge with multi-modal sensors for kitchen activity recognition

Mengxi Liu, Sungho Suh, Bo Zhou, Agnes Gruenerbl, Paul Lukowicz

专题命中 视频多模态 ：multi-modal(title)

Comments Presented at HASCA workshop of Ubicomp2022

Robust mode-locking in a hybrid ultrafast laser based on nonlinear multimodal interference

Xuanyi Liu, Maolin Dai, Denghui Pan, Kaibin Lin, Boris A. Malomed, Qian Li, H. Y. Fu

专题命中 视频多模态 ：multimodal(title)

Comments to be published in Optics and Laser Technology

Modelling Patient Trajectories Using Multimodal Information

João Figueira Silva, Sérgio Matos

专题命中 视频多模态 ：multimodal(title)

Comments To be published in Journal of Biomedical Informatics (Elsevier)

Multimodal Indoor Localisation for Measuring Mobility in Parkinson's Disease using Transformers

Ferdian Jovan, Ryan McConville, Catherine Morgan, Emma Tonkin, Alan Whone, Ian Craddock

专题命中 视频多模态 ：multimodal(title)

Comments 17 pages, 1 figure, 3 tables

Exoskeleton-Based Multimodal Action and Movement Recognition: Identifying and Developing the Optimal Boosted Learning Approach

Nirmalya Thakur, Chia Y. Han

专题命中 视频多模态 ：multimodal(title)

Journal ref Journal of Advances in Artificial Intelligence and Machine Learning. 2021; Volume 1, Issue 1, Article 4

Multi-modal Networks Reveal Patterns of Operational Similarity of Terrorist Organizations

Gian Maria Campedelli, Iain J. Cruickshank, Kathleen M. Carley

专题命中 视频多模态 ：multi-modal(title)

Comments 42 pages, 19 figures

Journal ref Terrorism and Political Violence, 0(0), 1-20 (2021)

Forecasting Thermoacoustic Instabilities in Liquid Propellant Rocket Engines Using Multimodal Bayesian Deep Learning

Ushnish Sengupta, Günther Waxenegger-Wilfing, Jan Martin, Justin Hardi, Matthew P. Juniper

专题命中 视频多模态 ：multimodal(title)

Cooperative Multi-Modal Localization in Connected and Autonomous Vehicles

Nikos Piperigkos, Aris S. Lalos, Kostas Berberidis, Christos Anagnostopoulos

专题命中 视频多模态 ：multi-modal(title)

Rough-Terrain Locomotion and Unilateral Contact Force Regulations With a Multi-Modal Legged Robot

Kaier Liang, Eric Sihite, Pravin Dangol, Andrew Lessieur, Alireza Ramezani

专题命中 视频多模态 ：multi-modal(title)

Comments 8 pages, 5 figure, submitted to accepted in American Control Conference (ACC)

Multi-Modal Data Collection for Measuring Health, Behavior, and Living Environment of Large-Scale Participant Cohorts: Conceptual Framework and Findings from Deployments

Congyu Wu, Hagen Fritz, Zoltan Nagy, Juan P. Maestre, Edison Thomaz, Christine Julien, Darla M. Castelli, Kaya de Barbaro, Gabriella M. Harari, R. Cameron Craddock, Kerry A. Kinney, Samuel D. Gosling, David M. Schnyer

专题命中 视频多模态 ：multi-modal(title)

Towards a Flexible Deep Learning Method for Automatic Detection of Clinically Relevant Multi-Modal Events in the Polysomnogram

Alexander Neergaard Olesen, Stanislas Chambon, Valentin Thorey, Poul Jennum, Emmanuel Mignot, Helge B. D. Sorensen

专题命中 视频多模态 ：multi-modal(title)

Comments Accepted for publication in 41st International Engineering in Medicine and Biology Conference (EMBC), July 23-27, 2019

Journal ref 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany, 2019, pp. 556-561