多模态信息融合

A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospects

Shulan Ruan, Rongwei Wang, Xuchen Shen, Huijie Liu, Baihui Xiao, Jun Shi, Kun Zhang, Zhenya Huang, Yu Liu, Enhong Chen, You He

机构 * Tsinghua University（清华大学） ； University of Science and Technology of China（中国科学技术大学） ； Hefei University of Technology（合肥工业大学） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；multi-modal fusion(abstract)；分类 cs.MM

ADM-Fusion: Adaptive Deep Multi-Sensor Fusion for Robust Ego-Motion Estimation in Diverse Conditions

ADM-Fusion：自适应深度多传感器融合用于多样化条件下的鲁棒自我运动估计

Hasan Moughnieh, Ibrahim Ghaddar, Hadi Elham, Imad H. Elhajj, Daniel Asmar

机构 * American University of Beirut（贝鲁特美国大学） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV、cs.RO

AI总结 提出ADM-Fusion，一种基于深度学习的端到端多传感器融合方法，通过自适应专家混合框架和跨任务注意力机制动态平衡传感器权重，在退化条件下保持鲁棒性。

Comments 8 pages, 4 figures

TFusionOcc: T-Primitive Based Object-Centric Multi-Sensor Fusion Framework for 3D Occupancy Prediction

TFusionOcc：基于T-原语的对象中心多传感器融合框架用于3D占用预测

Zhenxing Ming, Yaoqi Huang, Julie Stephany Berrio, Mao Shan, Stewart Worrall

机构 * Australian Centre for Robotics（澳大利亚机器人中心） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV、cs.RO

AI总结 本文提出TFusionOcc框架，通过T-原语实现3D语义占用预测，结合概率模型和多阶段融合架构，实验显示其在nuScenes数据集上表现优异。

An Intelligent Water-Saving Irrigation System Based on Multi-Sensor Fusion and Visual Servoing Control

ZhengKai Huang, YiKun Wang, ChenYu Hui, XiaoCheng

机构 * Xi'an Jiaotong University Faculty of Microelectronics Science（西安交通大学微电子科学与工程学院） ； Xi'an Jiaotong University School of Electrical Engineering（西安交通大学电气工程学院） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV、cs.RO

Robust and Precise Vehicle Localization based on Multi-sensor Fusion in Diverse City Scenes

Guowei Wan, Xiaolong Yang, Renlan Cai, Hao Li, Hao Wang, Shiyu Song

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV、cs.RO

Comments 8 pages, 6 figures, 2 tables, Accepted by ICRA 2018

Journal ref IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 4670-4677

i2Nav-Robot: A Large-Scale Indoor-Outdoor Robot Dataset for Multi-Sensor Fusion Navigation

i2Nav-Robot：用于多传感器融合导航的大规模室内外机器人数据集

Hailiang Tang, Tisheng Zhang, Liqiang Wang, Xin Ding, Man Yuan, Zhiyu Xiang, Jujin Chen, Yuhan Bian, Shuangyan Liu, Yuqing Wang, Guan Wang, Xiaoji Niu

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

AI总结 针对无人地面车辆导航数据集不足，提出i2Nav-Robot数据集，集成多模态传感器，经在线硬件同步和离线校准获取精确时间戳，含十个大规模序列，经评估数据质量高，对推进车辆导航研究实用。

Comments 13 pages, 15 figures

NavEYE: Vision-Centered Multi-Sensor Fusion-Based Situational Awareness System for Intelligent Surface Vehicles

NavEYE：用于智能水面舰艇的基于视觉的多传感器融合态势感知系统

Ryan Wen Liu, Junxiong Liang, Haoyu Wang, Mengwei Bao

机构 * Sanya Science and Education Innovation Park, Wuhan University of Technology（武汉理工大学三亚科教创新园） ； School of Navigation, Wuhan University of Technology（武汉理工大学航运学院） ； State Key Laboratory of Maritime Technology and Safety（船舶技术与安全国家重点实验室） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV

AI总结 为实现智能水面舰艇在复杂环境中的高质量感知，开发以视觉为中心的多传感器融合系统NavEYE，提出多约束门控数据关联等方法，经实验验证其融合方法在多方面的优越性及对舰艇航行安全的促进作用。

Journal ref Ocean Engineering, Volume 363, Part 4, 2026, Article 126781

EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation

EPMF: 高效感知感知多传感器融合用于3D语义分割

Mingkui Tan, Zhuangwei Zhuang, Sitao Chen, Rong Li, Kui Jia, Qicheng Wang, Yuanqing Li

机构 * School of Software Engineering, South China University of Technology（南方科技大学软件工程学院） ； Pazhou Laboratory, Guangzhou, China（广州帕佐实验室） ； School of Electronic and Information Engineering, South China University of Technology（南方科技大学电子与信息工程学院） ； Minieye, Shenzhen, Guangdong, China（深圳Minieye公司） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV

AI总结 提出一种高效感知多传感器融合方法EPMF，通过透视投影对齐点云与RGB图像，利用残差融合模块和感知损失提升3D语义分割性能，在nuScenes上mIoU超越RangeFormer 0.9%。

Comments 16 pages, 12 figures, 14 tables, IEEE TPAMI 2024, extended version of the ICCV2021 paper

Ultra-Fusion: A Resilient Tightly-Coupled Multi-Sensor Fusion SLAM Framework under Sensor Degradation and Spatiotemporal Perturbation for Intelligent Transportation Systems

Ultra-Fusion：面向传感器退化与时空扰动下智能交通系统的鲁棒紧耦合多传感器融合SLAM框架

Yihong Tian, Junjie Zhang, Liuyang Li, Deteng Zhang, Yunfei Zuo, Jie Yin

机构 * Beijing Institute of Technology（北京理工大学） ； Chongqing University（重庆大学） ； Sichuan University（四川大学） ； Northwestern Polytechnical University（西北工业大学） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

AI总结 提出Ultra-Fusion紧耦合多传感器融合SLAM框架，通过可观测性感知初始化、因子级可靠性调度和在线LiDAR-IMU时空标定，在传感器退化与时空扰动下实现鲁棒定位，在多种平台和基准上验证了高精度与可用性。

Comments 19 pages

Intelligent Multimodal Multi-Sensor Fusion-Based UAV Identification, Localization, and Countermeasures for Safeguarding Low-Altitude Economy

智能多模多传感器融合-based无人机识别、定位与防护系统用于保障低空经济安全

Yi Tao, Zhen Gao, Fangquan Ye, Jingbo Xu, Tao Song, Weidong Li, Yu Su, Lu Peng, Xiaomei Wu, Tong Qin, Zhongxiang Li, Dezhi Zheng

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 eess.SP

AI总结 本文提出基于深度学习的多模多传感器融合系统，用于实现无人机的识别、定位与防护，提升低空经济安全防护效率。

GNSS/Multi-Sensor Fusion Using Continuous-Time Factor Graph Optimization for Robust Localization

Haoming Zhang, Chih-Chun Chen, Heike Vallery, Timothy D. Barfoot

机构 * Institute of Automatic Control, Faculty of Mechanical Engineering, RWTH Aachen University（自动控制研究所，机械工程学院，亚琛RWTH大学） ； Department of BioMechanical Engineering, Delft University of Technology（生物机械工程系，代尔夫特技术大学） ； University of Toronto Robotics Institute（多伦多大学机器人研究所） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Comments Accepted for publication in the IEEE Transactions on Robotics

Journal ref IEEE Transactions on Robotics, vol. 40, pp. 4003-4023, 2024

SmartPNT-MSF: A Multi-Sensor Fusion Dataset for Positioning and Navigation Research

Feng Zhu, Zihang Zhang, Kangcheng Teng, Abduhelil Yakup, Xiaohong Zhang

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

RadarRGBD A Multi-Sensor Fusion Dataset for Perception with RGB-D and mmWave Radar

Tieshuai Song, Jiandong Ye, Ao Guo, Guidong He, Bin Yang

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 eess.IV

Comments 6 pages, 7 figures. Contains a new RGBD dataset for depth completion. Code and dataset will be released

Multi-Sensor Fusion-Based Mobile Manipulator Remote Control for Intelligent Smart Home Assistance

Xiao Jin, Bo Xiao, Huijiang Wang, Wendong Wang, Zhenhua Yu

机构 * Dyson School of Design Engineering, Imperial College London（帝国理工学院伦敦设计工程学院） ； School of Robotics, Xi’an Jiaotong-Liverpool University (XJTLU)（西安交通大学利物浦大学机器人学院） ； Department of Engineering, University of Cambridge（剑桥大学工程学院） ； School of Mechanical Engineering, Northwestern Polytechnical University（西北工业大学机械工程学院） ； Department of Computer Science, University of Aberdeen（阿伯丁大学计算机科学学院） 纠错

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

An End-to-End Learning-Based Multi-Sensor Fusion for Autonomous Vehicle Localization

Changhong Lin, Jiarong Lin, Zhiqiang Sui, XiaoZhi Qu, Rui Wang, Kehua Sheng, Bo Zhang

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Comments 7 pages, 8 figures, to be published in ICRA2025

A2DO: Adaptive Anti-Degradation Odometry with Deep Multi-Sensor Fusion for Autonomous Navigation

Hui Lai, Qi Chen, Junping Zhang, Jian Pu

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Comments 6+1pages, 6 figures, accept by ICRA

UniMSF: A Unified Multi-Sensor Fusion Framework for Intelligent Transportation System Global Localization

Wei Liu, Jiaqi Zhu, Guirong Zhuo, Wufei Fu, Zonglin Meng, Yishi Lu, Min Hua, Feng Qiao, You Li, Yi He, Lu Xiong

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation

Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela Rus, Song Han

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV

Comments ICRA 2023. The first two authors contributed equally to this work. Project page: https://bevfusion.mit.edu

AVM-SLAM: Semantic Visual SLAM with Multi-Sensor Fusion in a Bird's Eye View for Automated Valet Parking

Ye Li, Wenchao Yang, Dekun Lin, Qianlei Wang, Zhe Cui, Xiaolin Qin

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Comments Accepted by IROS 2024

ContextualFusion: Context-Based Multi-Sensor Fusion for 3D Object Detection in Adverse Operating Conditions

Shounak Sural, Nishad Sahu, Ragunathan Rajkumar

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV

Comments 8 pages, 8 figures

COMMIT: Certifying Robustness of Multi-Sensor Fusion Systems against Semantic Attacks

Zijian Huang, Wenda Chu, Linyi Li, Chejian Xu, Bo Li

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV

On State Estimation in Multi-Sensor Fusion Navigation: Optimization and Filtering

Feng Zhu, Zhuo Xu, Xveqing Zhang, Yuantai Zhang, Weijie Chen, Xiaohong Zhang

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusion

Siyu Teng, Luxi Li, Yuchen Li, Xuemin Hu, Lingxi Li, Yunfeng Ai, Long Chen

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Comments 20 Pages, 10 figures

Journal ref MSSP, vol. 208, p. 111051, 2024

To Fuse or Not to Fuse: Measuring Consistency in Multi-Sensor Fusion for Aerial Robots

Christian Lanegger, Helen Oleynikova, Michael Pantic, Lionel Ott, Roland Siegwart

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Comments Accepted and presented at the 18th International Symposium on Experimental Robotics (ISER 2023)

Benchmarking Robustness of AI-Enabled Multi-sensor Fusion Systems: Challenges and Opportunities

Xinyu Gao, Zhijie Wang, Yang Feng, Lei Ma, Zhenyu Chen, Baowen Xu

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.CV

Comments To appear in ESEC/FSE 2023

Detecting Multi-Sensor Fusion Errors in Advanced Driver-Assistance Systems

Ziyuan Zhong, Zhisheng Hu, Shengjian Guo, Xinyang Zhang, Zhenyu Zhong, Baishakhi Ray

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Lvio-Fusion: A Self-adaptive Multi-sensor Fusion SLAM Framework Using Actor-critic Method

Yupeng Jia, Haiyong Luo, Fang Zhao, Guanlin Jiang, Yuhang Li, Jiaquan Yan, Zhuqing Jiang, Zitian Wang

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Comments IROS 2021

Error State Extended Kalman Filter Multi-Sensor Fusion for Unmanned Aerial Vehicle Localization in GPS and Magnetometer Denied Indoor Environments

Lovro Markovic, Marin Kovac, Robert Milijas, Marko Car, Stjepan Bogdan

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Drift with Devil: Security of Multi-Sensor Fusion based Localization in High-Level Autonomous Driving under GPS Spoofing (Extended Version)

Junjie Shen, Jun Yeon Won, Zeyuan Chen, Qi Alfred Chen

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 cs.RO

Comments This is an extended version of our paper, which appears in USENIX Security 2020. For attack demos, see our project website: https://sites.google.com/view/cav-sec/fusionripper

Migrating Monarch Butterfly Localization Using Multi-Sensor Fusion Neural Networks

Mingyu Yang, Roger Hsiao, Gordy Carichner, Katherine Ernst, Jaechan Lim, Delbert A. Green, Inhee Lee, David Blaauw, Hun-Seok Kim

专题命中 机器人多传感器融合 ：sensor fusion(title,abstract)；multi-sensor fusion(title,abstract)；分类 eess.SP

Comments under review for ICASSP 2020