基于多模态融合的室内人体跟踪技术研究

Research on Indoor Human Tracking Technology Based on Multi-modal Fusion

传统的室内人体跟踪一般基于单独的相机或者二维激光雷达进行,基于二维点云的人腿检测在面对桌/椅腿较多的环境时准确度明显不足,而基于图像的人体检测在室内光线较暗时也会出现鲁棒性不足的问题,这些单模态的方法往往难以在干扰嘈杂的环境工作。因此,该文提出了一种基于多模态融合的室内人体跟踪方法,训练一个Adaboost分类器将点云段分为人腿/非人腿,使用成熟的YOLOv3网络对图像中的人体进行检测,结合点云段分类结果和视觉人体边界框完成基于概率数据关联的模态融合,最后使用扩展卡尔曼滤波器完成人体跟踪。对该方法在嘈杂的实验室环境下和晦暗的走廊环境下进行测试,结果表明多模态融合的室内人体检测取得了优于单模态的效果,验证了该系统的鲁棒性和有效性,适用于移动机器人的室内人体跟踪任务。

Traditional indoor human tracking is usually based on a separate camera or two-dimensional LiDAR.The accuracy of two-dimensional point clouds-based human leg detection is significantly inadequate when facing the environments having more table/chair legs.Image-based human detection also has the problem of insufficient robustness when the light is dim.These single-modal methods are often difficult to work in noisy environments.Therefore,we present an indoor human tracking method based on multi-modal fusion.An Adaboost classifier is trained to divide the segments of point clouds into human legs and non-human legs.The mature YOLOv3 network is used to detect human body in the image.The classification results of segments and the visual human bounding boxes are combined to complete the modal fusion based on probability data association.Finally,the extended Kalman filter is used to complete human tracking.The experiments in noisy laboratory environment and dark corridor environment show that the proposed algorithm achieves better results than three single-modal methods.The robustness and effectiveness of the proposed system are verified and it is suitable for indoor human tracking tasks of mobile robots.