State Estimation Method for GNSS/INS/Visual Multi-sensor Fusion Based on Factor Graph Optimization for Unmanned System

With the development of unmanned driving technology,intelligent robots and drones,high-precision localization,navigation and state estimation technologies have also made great progress.Traditional global navigation satellite system/inertial navigation system(GNSS/INS)integrated navigation systems can provide high-precision navigation information continuously.However,when this system is applied to indoor or GNSS-denied environments,such as outdoor substations with strong electromagnetic interference and complex dense spaces,it is often unable to obtain high-precision GNSS positioning data.The positioning and orientation errors will diverge and accumulate rapidly,which cannot meet the high-precision localization requirements in large-scale and long-distance navigation scenarios.This paper proposes a method of high-precision state estimation with fusion of GNSS/INS/Vision using a nonlinear optimizer factor graph optimization as the basis for multi-source optimization.Through the collected experimental data and simulation results,this system shows good performance in the indoor environment and the environment with partial GNSS signal loss.

Applications of state estimation in multi-sensor information fusion for the monitoring of open pit mine slope deformation

The traditional open pit mine slope deformation monitoring system can not use the monitoring information coming from many monitoring points at the same time, can only using the monitoring data coming from a key monitoring point,and that is to say it can only handle one-dimensional time series.Given this shortage in the monitoring, the multi-sensor information fusion in the state estimation techniques would be intro- duced to the slope deformation monitoring system,and by the dynamic characteristics of deformation slope,the open pit slope would be regarded as a dynamic goal,the condi- tion monitoring of which would be regarded as a dynamic target tracking.Distributed In- formation fusion technology with feedback was used to process the monitoring data and on this basis Klman filtering algorithms was introduced,and the simulation examples was used to prove its effectivenes.

Deep Weighted Fusion Learning(DWFL)-based multi-sensor fusion model for accurate building occupancy detection

With the advancement of artificial intelligence,the dominance of deep learning(DL)models over ordinary machine learning(ML)algorithms has become a reality in recent years due to its capability of handling complex pattern recognition without manual feature pre-definition.With the growing demands for power savings,building energy loss reduction could benefit from DL techniques.For buildings/rooms with the varying number of occupants,heating,ventilation,and air conditioning(HVAC)systems are often found in operations without much necessity.To reduce the building’s energy loss,accurate occupancy detection/prediction(ODP)results could be used to control the proper operations of HVACs.However,ODP is a challenging issue due to multiple reasons,such as improper selection/deployment of sensors,inefficient learning algorithms for pattern recognition,varying room conditions,etc.To overcome the above challenges,we propose a DL-based framework,i.e.,Deep Weighted Fusion Learning(DWFL),to detect and predict occupancy counts with optimal multi-sensor fusion structure.DWFL fuses the extracted features from multiple types of sensors with the priority/weight assignment to each sensor.Such weight assignment considers different room conditions and the pros/cons of each type of sensor.To evaluate DWFL model in terms of occupancy prediction accuracy,we have set up an experimental testbed with low-cost cameras,carbon dioxide(CO_(2)),and passive infrared(PIR)sensors.Among the recently proposed occupancy detection models,DeepFusion utilized deep learning model on heterogeneous sensor data and achieved 88%accuracy in occupancy count estimation(Xue et al.,2019).Another deep learning-based model MI-PIR achieved 91%accuracy on raw analog data from PIR sensors(Andrews et al.,2020).Our research outcome is 94%.Therefore,the experiment results show that our DWFL scheme outperforms the state-of-the-art ODP methods by 3%.

Integrated Navigation Based on Robust Estimation Outputs of Multi-sensor Measurements and Adaptive Weights of Dynamic Model Information

An integrated navlgation based on the kinematic or dynamic state model and the raw measurements has the advantages of high redundancy, high reliability, as well as high ability of fault tolerance and simplicity in calculation. In order to control the influences of measurements outliers and the kinematic model errors on the integrated navigation results, a robust estimation method and an adaptive data fusion method are applied. An integrated navigation example using simulated data is performed and analyzed.

A survey on multi-sensor fusion based obstacle detection for intelligent ground vehicles in off-road environments

With the development of sensor fusion technologies, there has been a lot of research on intelligent ground vehicles, where obstacle detection is one of the key aspects of vehicle driving. Obstacle detection is a complicated task, which involves the diversity of obstacles, sensor characteristics, and environmental conditions. While the on-road driver assistance system or autonomous driving system has been well researched, the methods developed for the structured road of city scenes may fail in an off-road environment because of its uncertainty and diversity.A single type of sensor finds it hard to satisfy the needs of obstacle detection because of the sensing limitations in range, signal features, and working conditions of detection, and this motivates researchers and engineers to develop multi-sensor fusion and system integration methodology. This survey aims at summarizing the main considerations for the onboard multi-sensor configuration of intelligent ground vehicles in the off-road environments and providing users with a guideline for selecting sensors based on their performance requirements and application environments.State-of-the-art multi-sensor fusion methods and system prototypes are reviewed and associated to the corresponding heterogeneous sensor configurations. Finally, emerging technologies and challenges are discussed for future study.

Optimal fusion state estimator for a multi-sensor system subject to multiple packet dropouts

In this note,we study the state estimation problem for a multi-sensor system subject to multiple packet dropouts.A novel optimal distributed fusion estimator is derived by applying a resending mechanism and a parallel information filtering structure.It is shown that the proposed distributed fusion estimator has smaller estimation error covariance and less computation complexity when compared with the centralised Kalman like estimator with multiple intermittent measurements.

基于激光雷达探测的飞机尾流融合预测方法

为提高基于激光雷达的飞机尾流探测反演精度,根据扫描出的径向风速数据,建立了基于Kolmogorov结构函数的大气背景湍流耗散率(EDR)估计方法;然后基于两阶段消散预测模型,计入湍流对尾流消散过程的影响,实现基于历史探测数据的尾涡强度环量和涡核运动趋势的预测;通过环境探测数据与预测模型的结合,提高尾涡特性参数的反演精度。研究表明,相较于反演算法,采用本文模型预测的尾涡轨迹在径向距离和角度的精度上分别提高了59.5%、64.8%。

ADAS系统视觉与毫米波雷达分布式抗差卡尔曼滤波融合算法

自动驾驶车辆常常利用多传感器对周围目标进行检测和跟踪,但受限于传感器多源异构特性和复杂多变的驾驶环境,准确的多目标检测和跟踪仍是实现自动驾驶的一大困难和挑战。本文面向高级驾驶辅助系统(ADAS)的多目标检测与跟踪任务,采用了基于1个视觉传感器和5个毫米波雷达(1V5R)的传感器配置方案,且设计了基于分布式抗差卡尔曼滤波器的多传感器信息融合算法以实现对周围目标的准确感知。首先,针对不同传感器数据特征,采用不同的线性卡尔曼滤波器和扩展卡尔曼滤波器进行数据融合,并基于分布式卡尔曼滤波建立了1V5R多传感器信息融合框架。其次,为降低传感器动态误差对于融合精度的影响,在卡尔曼加权观测融合的基础上,引入抗差估计方法,实现了对传感器动态误差的实时估计和修正。最后,通过离线仿真和实车道路试验对所提出的基于分布式抗差卡尔曼滤波的多传感器融合算法进行了验证。试验结果表明,与单一传感器的测量值相比,所提出的算法能有效融合多个传感器的信息以提升目标的检测与跟踪精度,且鲁棒性较好。

基于多关键点检测加权融合的无人机相对位姿估计算法

针对无人机降落阶段中无人船受水面波浪影响导致图像产生运动模糊以及获取无人机相对位姿精度低且鲁棒性差的问题,提出一种基于多模型关键点加权融合的6D目标位姿估计算法,以提高位姿估计的精度和鲁棒性.首先,基于无人船陀螺仪得到的运动信息设计帧间抖动模型,通过还原图像信息达到降低图像噪声的目的;然后,设计一种多模型的级联回归特征提取算法,通过多模型检测舰载视觉系统获取的图像,以增强特征空间的多样性;同时,将检测过程中关键点定位形状增量集作为融合权重对模型进行加权融合,以提高特征空间的鲁棒性;紧接着,利用EPnP(Efficient perspective-n-point)计算关键点相机坐标系坐标,将PnP(Perspective-n-point)问题转化为ICP(Iterative closest point)问题;最终,基于关键点解集的离散度为关键点赋权,使用ICP算法求解位姿以削弱深度信息对位姿的影响.仿真结果表明,该算法能够建立一个精度更高的特征空间,使得位姿解算时特征映射的损失降低,最终提高位姿解算的精度.

面向无人机自主着陆的视觉感知与位姿估计方法综述

自主着陆技术是制约无人机(Unmanned aerial vehicle,UAV)自主性等级提升中极具挑战性的一项技术.立足于未来基于视觉的无人机自主着陆技术的发展需求,围绕其中的核心问题——着陆场检测与位姿估计,对近十年来国内外无人机自主着陆领域中基于视觉的着陆场检测与位姿估计方法研究进展进行总结.首先,在分析无人机自主着陆应用需求的基础上,指出机器视觉在无人机自主着陆领域的应用优势,并凝练出存在的科学问题;其次,按不同应用场景划分对着陆场检测算法进行梳理;然后,分别对纯视觉、多源信息融合的位姿估计技术研究成果进行归纳;最后,总结该领域有待进一步解决的难点,并对未来的技术发展趋势进行展望.

一种三星探测数据融合的弹道估计算法

针对天基红外系统(Space Based Infrared System,SBIRS)三星探测弹道估计问题,提出GEO卫星与HEO卫星探测数据融合的估计算法。根据星座构成和探测体制,利用STK分析SBIRS对某一区域的覆盖能力,约43%的时间可以实现三星以上完全覆盖;建立三星探测数据融合算法模型,对导弹目标的运动状态进行实时估计,导弹运动建模采用当前统计模型,数据融合采用集中式结构,滤波算法采用无迹卡尔曼滤波。试验表明,与双星探测弹道估计误差相比,三星探测弹道估计误差显著减小。

基于改进SPRT的机载欺骗式干扰检测方法

针对传统机载GNSS欺骗式干扰检测方法在多星受欺骗、或伪距欺骗量较小时检测性能差的问题,提出一种基于改进SPRT的欺骗式干扰检测方法。利用惯性导航系统(INS)和测距机(DME)机载多源数据,构造欺骗检测量。基于Bayes参数估计理论,提出一种改进SPRT算法,利用欺骗检测量计算自适应补偿值,构成改进的欺骗检测统计量。利用改进的欺骗检测统计量进行欺骗式干扰检测,加快算法的检测速度。巡航阶段的仿真结果表明,在满足机载设备相关要求的同时,在多星受欺骗情况和伪距欺骗量较小情况下均具有较好的检测性能。

复杂背景下改进的ViBe运动目标检测算法

针对传统ViBe算法在复杂背景下检测运动目标时会出现鬼影、阴影、误检等问题,提出了一种改进的ViBe运动目标检测算法,称为GS-ViBe算法。在GS-ViBe背景模型初始化阶段,利用最大后验估计法确定每个像素点的最佳高斯分布数目,使其形成多帧融合背景来代替ViBe的单帧背景初始化方法,从而消除鬼影;在GS-ViBe前景检测阶段,增加多特征融合阴影检测过程,并将其检测结果和ViBe前景目标融合,得到消除阴影后的前景目标;最后,在GS-ViBe背景模型更新阶段,引入动态更新因子代替固定更新因子,使得背景可以自适应更新,从而降低目标的误检率。在多种复杂背景下与传统ViBe算法对比发现,GS-ViBe算法召回率提高了37.74%,准确率平均提高了19.83%,误检率平均降低了52.57%,表明GS-ViBe算法可以有效消除鬼影、阴影、误检的干扰,获取到完整的前景目标。

基于自适应阈值的多特征经验融合疲劳检测

在安防领域,疲劳是导致安防工作人员注意力下降,诱发各类问题的重要原因.现有的疲劳检测方法存在各种问题,比如生理指标检测侵入性强且价格昂贵,人脸疲劳检测结果受个体差异和头部姿态的影响以及疲劳预警时长较短等,因此本文提出一种基于自适应阈值和面部多特征经验融合的疲劳早后期检测算法,使用轻量级的SCRFD模型进行人脸检测,使用MobileNetV2模型进行人脸关键点定位,使用梯度提升树学习头部姿态信息与眼睛纵横比(EAR)阈值的映射关系,通过眼睑闭合时间百分比(PERCLOS)、嘴巴张开时间百分比(FOM)和头部姿态6个自由度分别实现眨眼、哈欠、点头前后倾动作的识别.在疲劳估计阶段,为了将多种疲劳行为融合映射成与疲劳相关的KSS值,先根据专家经验预先构建好多种人脸行为的疲劳因果图,接着使用自定义的singleton,mutual和activate/inhibit特征算子,结合因果图从人脸行为检测序列中计算疲劳早期和疲劳后期KSS值,最后使用双尺度KNN实现疲劳早后期估计.实验结果表明所提算法在YawDD数据集上哈欠检测准确率达到93.81%,在UTA-RLDD和Drozy数据集上疲劳识别准确率分别达到67.72%,87.88%,仅通过CPU,推理实时性可达到17.96每秒传输帧数(FPS).

基于多模态特征融合的井下人员不安全行为识别

采用人工智能技术对井下人员的行为进行实时识别,对保证矿井安全生产具有重要意义。针对基于RGB模态的行为识别方法易受视频图像背景噪声影响、基于骨骼模态的行为识别方法缺乏人与物体的外观特征信息的问题,将2种方法进行融合,提出了一种基于多模态特征融合的井下人员不安全行为识别方法。通过SlowOnly网络对RGB模态特征进行提取;使用YOLOX与Lite-HRNet网络获取骨骼模态数据,采用PoseC3D网络对骨骼模态特征进行提取;对RGB模态特征与骨骼模态特征进行早期融合与晚期融合,最后得到井下人员不安全行为识别结果。在X-Sub标准下的NTU60 RGB+D公开数据集上的实验结果表明:在基于单一骨骼模态的行为识别模型中,PoseC3D拥有比GCN(图卷积网络)类方法更高的识别准确率,达到93.1%;基于多模态特征融合的行为识别模型对比基于单一骨骼模态的识别模型拥有更高的识别准确率,达到95.4%。在自制井下不安全行为数据集上的实验结果表明:基于多模态特征融合的行为识别模型在井下复杂环境下识别准确率仍最高,达到93.3%,对相似不安全行为与多人不安全行为均能准确识别。

基于人脸姿态估计的虚拟眼镜试戴技术

为解决用户线上眼镜的最优选购,提出了一种基于人脸姿态估计的虚拟眼镜试戴技术。首先采用肤色模型与形状模型结合的算法对场景中的人脸区域进行检测,然后根据人眼在人脸中的几何位置关系实现人眼的精确定位;进一步利用人眼对称性先验知识来估计脸在三维空间中的姿态信息,即人脸与正面的偏移角度;最后,依据人眼位置和人脸姿态将眼镜图像融合到眼睛区域,即完成眼镜的虚拟试戴。该方法为3D环境下客户与商品之间虚拟视觉化的实现提供了一种可靠的技术支撑和应用思路。

多传感信息融合与自动化

综述多传感信息融合技术的概念、发展的起因、主要研究内容、现有的理论和方法 ,同时还就其在工业机器人、汽车自动导航与驾驶、身份识别、多目标跟踪、人工智能等自动化领域的应用进行了讨论 .作者对于这一新兴研究方向未来的发展趋势作了展望 ,并就信息融合的一般理论、主要方法论、存在的问题等进行了探讨 .

基于多传感器融合的紧耦合SLAM系统

同时建图与定位(SLAM)是自动驾驶功能重要的组成部分,现有算法以激光或视觉惯性里程计为主,未充分利用多模态传感器各自的优势,对特征缺失的场景鲁棒性不足。针对此问题,本文中提出了一种采用激光雷达、摄像头和惯性测量单元(IMU)的多传感器紧耦合SLAM系统。首先它改善了激光雷达点云特征提取和平面拟合的方案,提升了利用点云对视觉特征点深度信息优化的效率和精度。其次提出的紧耦合状态估计框架通过在视觉惯性系统中直接添加激光雷达里程计约束,在不增加算法复杂度的前提下提升了系统的稳定性和精度。最后由粗到精的视觉-激光雷达耦合回环框架进一步降低了系统的长时累计漂移。在开源数据集KITTI上进行大量测试验证的结果表明,与其它常用的算法相比,所提出的算法具有较高的精度和环境适应能力。另外在基于自主搭建的自动驾驶汽车测试平台进行的实车试验还证明本算法可适应长时间大场景的工作环境。

动态场景中的改进混合高斯背景模型

提出一种应用于运动目标检测的改进混合高斯背景模型。在背景模型更新过程中,通过调整阈值,降低单模态背景的误检率。在运动目标检测时,融合统计差分法和时域差分法,降低多模态背景像素的误检率。实验结果表明,改进模型能有效解决由复杂动态背景引起的误检问题,具有较好的检测性能。

基于多检测器最大熵融合的多通道光谱图像异常检测

提出了一种多检测器最大熵融合的多通道光谱图像异常检测算法.选择多个不同的异常检测器,并利用自适应窗宽非参核密度估计方法估计其各自的输出分布,保留了多通道光谱图像数据的"长尾"特性,且避免了先验模型假设带来的模型误差.将各原始检测器的输出投影到具有标准正态边缘分布的变换空间中,利用变换空间中模型化的最大熵融合规则实现多检测器的决策级最优概率融合.在原数据空间通过似然函数的检验完成多通道光谱图像的目标检测.利用机载EPS-A航拍多通道光谱图像进行了实验,实验结果表明了算法的有效性.