Performance of GPS and IMU sensor fusion using unscented Kalman filter for precise i-Boat navigation in infinite wide waters

The Unmanned Surface Vehicle(USV)navigation system needs an accurate,firm,and reliable performance to avoid obstacles,as well as carry out automatic movements during missions.The Global Positioning System(GPS)is often used in these systems to provide absolute position information.However,the GPS measurements are affected by external conditions such as atmospheric bias and multipath effects.This leads to the inability of the stand-alone GPS to provide accurate positioning for the USV systems.One of the solutions to correct the errors of this sensor is by conducting GPS and Inertial Measurement Unit(IMU)fusion.The IMU sensor is complementary to the GPS and not affected by external conditions.However,it accumulates noise as time elapses.Therefore,this study aims to determine the fusion of the GPS and IMU sensors for the i-Boat navigation system,which is a USV developed by Institut Teknologi Sepuluh Nopember(ITS)Surabaya.Using the Unscented Kalman filter(UKF),sensor fusion was carried out based on the state equation defined by the dynamic and kinematic mathematical model of ship motion in 6 degrees of freedom.Then the performance of this model was tested through several simulations using different combinations of attitude measurement data.Two scenarios were conducted in the simulations:attitude measurement inclusion and exclusion(Scenarios I and II,respectively).The results showed that the position estimation in Scenario II was better than in Scenario I,with the Root Mean Square Error(RMSE)value of 0.062 m.Further simulations showed that the presence of attitude measurement data caused a decrease in the fusion accuracy.The UKF simulation with eight measurement parameters(Scenarios A,B and C)and seven measurement parameters(Scenarios D,E and F),as well as analytical attitude movement,indicated that yaw data had the largest noise accumulation compared to roll and pitch.

Multi-rate sensor fusion-based adaptive discrete finite-time synergetic control for flexible-joint mechanical systems

This paper proposes an adaptive discrete finite-time synergetic control （ADFTSC） scheme based on a multi-rate sensor fusion estimator for flexible-joint mechanical systems in the presence of unmeasured states and dynamic uncertainties. Multi-rate sensors are employed to observe the system states which cannot be directly obtained by encoders due to the existence of joint flexibilities. By using an extended Kalman filter （EKF）, the finite-time synergetic controller is designed based on a sensor fusion estimator which estimates states and parameters of the mechanical system with multi-rate measurements. The proposed controller can guarantee the finite-time convergence of tracking errors by the theoretical derivation. Simulation and experimental studies are included to validate the effectiveness of the proposed approach.

Obstacle detection using multi-sensor fusion

This paper presents an obstacle detection approach for blind pedestrians by fusing data from camera and laser sensor.For purely vision-based blind guidance system,it is difficult to discriminate low-level obstacles with cluttered road surface,while for purely laser-based system,it usually requires to scan the forward environment,which turns out to be very inconvenient.To overcome these inherent problems when using camera and laser sensor independently,a sensor-fusion model is proposed to associate range data from laser domain with edges from image domain.Based on this fusion model,obstacle's position,size and shape can be estimated.The proposed method is tested in several indoor scenes,and its efficiency is confirmed.

Sensor Fusion with Square-Root Cubature Information Filtering

This paper derives a square-root information-type filtering algorithm for nonlinear multi-sensor fusion problems using the cubature Kalman filter theory. The resulting filter is called the square-root cubature Information filter (SCIF). The SCIF propagates the square-root information matrices derived from numerically stable matrix operations and is therefore numerically robust. The SCIF is applied to a highly maneuvering target tracking problem in a distributed sensor network with feedback. The SCIF’s performance is finally compared with the regular cubature information filter and the traditional extended information filter. The results, presented herein, indicate that the SCIF is the most reliable of all three filters and yields a more accurate estimate than the extended information filter.

Identification of Pitch Dynamics of An Autonomous Underwater Vehicle Using Sensor Fusion

This paper presents a method for identification of the hydrodynamic coefficients of the dive plane of an autonomous underwater vehicle. The proposed identification method uses the governing equations of motion to estimate the coefficients of the linear damping, added mass and inertia, cross flow drag and control. Parts of data required by the proposed identification method are not measured by the onboard instruments. Hence, an optimal fusion algorithm is devised which estimates the required data accurately with a high sampling rate. To excite the dive plane dynamics and obtain the required measurements, diving maneuvers should be performed. Hence, a reliable controller with satisfactory performance and stability is needed. A cascaded controller is designed based on the coefficients obtained using a semi-empirical method and its robustness to the uncertainties is verified by the μ-analysis method. The performance and accuracy of the identification and fusion algorithms are investigated through 6-DOF numerical simulations of a realistic autonomous underwater vehicle.

Collaborative positioning for swarms:A brief survey of vision,LiDAR and wireless sensors based methods

As positioning sensors,edge computation power,and communication technologies continue to develop,a moving agent can now sense its surroundings and communicate with other agents.By receiving spatial information from both its environment and other agents,an agent can use various methods and sensor types to localize itself.With its high flexibility and robustness,collaborative positioning has become a widely used method in both military and civilian applications.This paper introduces the basic fundamental concepts and applications of collaborative positioning,and reviews recent progress in the field based on camera,LiDAR(Light Detection and Ranging),wireless sensor,and their integration.The paper compares the current methods with respect to their sensor type,summarizes their main paradigms,and analyzes their evaluation experiments.Finally,the paper discusses the main challenges and open issues that require further research.

Target Detection on Water Surfaces Using Fusion of Camera and LiDAR Based Information

To address the challenges of missed detections in water surface target detection using solely visual algorithms in unmanned surface vehicle(USV)perception,this paper proposes a method based on the fusion of visual and LiDAR point-cloud projection for water surface target detection.Firstly,the visual recognition component employs an improved YOLOv7 algorithmbased on a self-built dataset for the detection of water surface targets.This algorithm modifies the original YOLOv7 architecture to a Slim-Neck structure,addressing the problemof excessive redundant information during feature extraction in the original YOLOv7 network model.Simultaneously,this modification simplifies the computational burden of the detector,reduces inference time,and maintains accuracy.Secondly,to tackle the issue of sample imbalance in the self-built dataset,slide loss function is introduced.Finally,this paper replaces the original Complete Intersection over Union(CIoU)loss function with the Minimum Point Distance Intersection over Union(MPDIoU)loss function in the YOLOv7 algorithm,which accelerates model learning and enhances robustness.To mitigate the problem of missed recognitions caused by complex water surface conditions in purely visual algorithms,this paper further adopts the fusion of LiDAR and camera data,projecting the threedimensional point-cloud data from LiDAR onto a two-dimensional pixel plane.This significantly reduces the rate of missed detections for water surface targets.

基于改进Centerfusion的自动驾驶3D目标检测模型

针对自动驾驶路面上目标漏检和错检的问题,提出一种基于改进Centerfusion的自动驾驶3D目标检测模型。该模型通过将相机信息和雷达特征融合,构成多通道特征数据输入,从而增强目标检测网络的鲁棒性,减少漏检问题;为了能够得到更加准确丰富的3D目标检测信息,引入了改进的注意力机制,用于增强视锥网格中的雷达点云和视觉信息融合;使用改进的损失函数优化边框预测的准确度。在Nuscenes数据集上进行模型验证和对比,实验结果表明,相较于传统的Centerfusion模型,提出的模型平均检测精度均值(mean Average Precision,mAP)提高了1.3%,Nuscenes检测分数(Nuscenes Detection Scores,NDS)提高了1.2%。

Mobile robot localization algorithm based on multi-sensor information fusion

In order to effectively reduce the uncertainty error of mobile robot localization with a single sensor and improve the accuracy and robustness of robot localization and mapping,a mobile robot localization algorithm based on multi-sensor information fusion(MSIF)was proposed.In this paper,simultaneous localization and mapping(SLAM)was realized on the basis of laser Rao-Blackwellized particle filter(RBPF)-SLAM algorithm and graph-based optimization theory was used to constrain and optimize the pose estimation results of Monte Carlo localization.The feature point extraction and quadrilateral closed loop matching algorithm based on oriented FAST and rotated BRIEF(ORB)were improved aiming at the problems of generous calculation and low tracking accuracy in visual information processing by means of the three-dimensional(3D)point feature in binocular visual reconstruction environment.Factor graph model was used for the information fusion under the maximum posterior probability criterion for laser RBPF-SLAM localization and binocular visual localization.The results of simulation and experiment indicate that localization accuracy of the above-mentioned method is higher than that of traditional RBPF-SLAM algorithm and general improved algorithms,and the effectiveness and usefulness of the proposed method are verified.

Non-parametric belief propagation for mobile mapping sensor fusion

Many different forms of sensor fusion have been proposed each with its own niche.We propose a method of fusing multiple different sensor types.Our approach is built on the discrete belief propagation to fuse photogrammetry with GPS to generate three-dimensional(3D)point clouds.We propose using a non-parametric belief propagation similar to Sudderth et al’s work to fuse different sensors.This technique allows continuous variables to be used,is trivially parallel making it suitable for modern many-core processors,and easily accommodates varying types and combinations of sensors.By defining the relationships between common sensors,a graph containing sensor readings can be automatically generated from sensor data without knowing a priori the availability or reliability of the sensors.This allows the use of unreliable sensors which firstly,may start and stop providing data at any time and secondly,the integration of new sensor types simply by defining their relationship with existing sensors.These features allow a flexible framework to be developed which is suitable for many tasks.Using an abstract algorithm,we can instead focus on the relationships between sensors.Where possible we use the existing relationships between sensors rather than developing new ones.These relationships are used in a belief propagation algorithm to calculate the marginal probabilities of the network.In this paper,we present the initial results from this technique and the intended course for future work.

ANFIS-based Sensor Fusion System of Sit-to-stand for Elderly People Assistive Device Protocols

This paper describes the analysis and design of an assistive device for elderly people under development at the EgyptJapan University of Science and Technology（E-JUST） named E-JUST assistive device（EJAD）.Several experiments were carried out using a motion capture system（VICON） and inertial sensors to identify the human posture during the sit-to-stand motion.The EJAD uses only two inertial measurement units（IMUs） fused through an adaptive neuro-fuzzy inference systems（ANFIS） algorithm to imitate the real motion of the caregiver.The EJAD consists of two main parts,a robot arm and an active walker.The robot arm is a 2-degree-of-freedom（2-DOF） planar manipulator.In addition,a back support with a passive joint is used to support the patient s back.The IMUs on the leg and trunk of the patient are used to compensate for and adapt to the EJAD system motion depending on the obtained patient posture.The ANFIS algorithm is used to train the fuzzy system that converts the IMUs signals to the right posture of the patient.A control scheme is proposed to control the system motion based on practical measurements taken from the experiments.A computer simulation showed a relatively good performance of the EJAD in assisting the patient.

Development of temperature-robust damage factor based on sensor fusion for a wind turbine structure

Wind power systems have gained much attention due to the relatively high reliability, maturity in technology and cost competitiveness compared to other renewable alternatives. Advances have been made to increase the power efficiency of the wind turbines while less attention has been focused on structural integrity assessment of the structural systems. Vibration-based damage detection has widely been researched to identify damages on a structure based on change in d^mmic characteristics. Widely spread methods are natural frequency-based, mode shape-based, and curvature mode shape-based methods. The natural frequency-based methods are convenient but vulnerable to environmental temperature variation which degrades damage detection capability; mode shapes are less influenced by temperature variation and able to locate damage but requires extensive sensor instrumentation which is costly and vulnerable to signal noises. This study proposes novelty of damage factor based on sensor fusion to exclude effect of temperature variation. The combined use of an accelerometer and an inclinometer was considered and damage factor was defined as a change in relationship between those two measurements. The advantages of the proposed method are： 1） requirement of small number of sensor, 2） robusmess to change in temperature and signal noise and 3） ability to roughly locate damage. Validation of the proposed method is carried out through numerical simulation on a simplified 5 MW wind turbine model.

Vision‐audio fusion SLAM in dynamic environments

Moving humans,agents,and subjects bring many challenges to robot self‐localisation and environment perception.To adapt to dynamic environments,SLAM researchers typically apply several deep learning image segmentation models to eliminate these moving obstacles.However,these moving obstacle segmentation methods cost too much computation resource for the onboard processing of mobile robots.In the current industrial environment,mobile robot collaboration scenario,the noise of mobile robots could be easily found by on‐board audio‐sensing processors and the direction of sound sources can be effectively acquired by sound source estimation algorithms,but the distance estimation of sound sources is difficult.However,in the field of visual perception,the 3D structure information of the scene is relatively easy to obtain,but the recognition and segmentation of moving objects is more difficult.To address these problems,a novel vision‐audio fusion method that combines sound source localisation methods with a visual SLAM scheme is proposed,thereby eliminating the effect of dynamic obstacles on multi‐agent systems.Several heterogeneous robots experiments in different dynamic scenes indicate very stable self‐localisation and environment reconstruction performance of our method.

A multi-rate sensor fusion approach using information filters for estimating aero-engine performance degradation

Gas-path performance estimation plays an important role in aero-engine health management, and Kalman Filter(KF) is a well-known technique to estimate performance degradation. In previous studies, it is assumed that different kinds of sensors are with the same sampling rate, and they are used for state estimation by the KF simultaneously. However, it is hard to achieve state estimation using various kinds of sensor measurements at the same sampling rate due to a complex network and physical characteristic differences between sensors, especially in an advanced multisensor architecture. For this purpose, a multi-rate sensor fusion using the information filtering approach is proposed based on the square-root cubature rule, which is called Multi-rate Squareroot Cubature Information Filter(MSCIF) to track engine performance degradation. Soft measurement synchronization of the MSCIF is designed to provide a sensor fusion condition for multiple sampling rates of measurement, and a fault sensor is isolated by maximum likelihood validation before state estimation. The contribution of this paper is to supply a novel multi-rate informationfilter approach for sensor fault tolerant health estimation of an aero-engine in a multi-sensor system. Tests are conducted for aero-engine performance degradation estimation with multiple sampling rates of sensor measurement on both digital simulation and semi-physical experiment.Experimental results illustrate the superiority of the proposed algorithm in terms of degradation estimation accuracy and robustness to sensor failure in a multi-sensor system.

Sensor fusion-based approach for the field robot localization on Rovitis 4.0 vineyard robot

This study proposed an approach for robot localization using data from multiple low-cost sensors with two goals in mind,to produce accurate localization data and to keep the computation as simple as possible.The approach used data from wheel odometry,inertial-motion data from the Inertial Motion Unit(IMU),and a location fix from a Real-Time Kinematics Global Positioning System(RTK GPS).Each of the sensors is prone to errors in some situations,resulting in inaccurate localization.The odometry is affected by errors caused by slipping when turning the robot or putting it on slippery ground.The IMU produces drifts due to vibrations,and RTK GPS does not return to an accurate fix in(semi-)occluded areas.None of these sensors is accurate enough to produce a precise reading for a sound localization of the robot in an outdoor environment.To solve this challenge,sensor fusion was implemented on the robot to prevent possible localization errors.It worked by selecting the most accurate readings in a given moment to produce a precise pose estimation.To evaluate the approach,two different tests were performed,one with robot localization from the robot operating system(ROS)repository and the other with the presented Field Robot Localization.The first did not perform well,while the second did and was evaluated by comparing the location and orientation estimate with ground truth,captured by a hovering drone above the testing ground,which revealed an average error of 0.005 m±0.220 m in estimating the position,and 0.6°±3.5°when estimating orientation.The tests proved that the developed field robot localization is accurate and robust enough to be used on a ROVITIS 4.0 vineyard robot.

Enhanced Autonomous Exploration and Mapping of an Unknown Environment with the Fusion of Dual RGB-D Sensors

The autonomous exploration and mapping of an unknown environment is useful in a wide range of applications and thus holds great significance. Existing methods mostly use range sensors to generate twodimensional (2D) grid maps. Red/green/blue-depth (RGB-D) sensors provide both color and depth information on the environment, thereby enabling the generation of a three-dimensional (3D) point cloud map that is intuitive for human perception. In this paper, we present a systematic approach with dual RGB-D sensors to achieve the autonomous exploration and mapping of an unknown indoor environment. With the synchronized and processed RGB-D data, location points were generated and a 3D point cloud map and 2D grid map were incrementally built. Next, the exploration was modeled as a partially observable Markov decision process. Partial map simulation and global frontier search methods were combined for autonomous exploration, and dynamic action constraints were utilized in motion control. In this way, the local optimum can be avoided and the exploration efficacy can be ensured. Experiments with single connected and multi-branched regions demonstrated the high robustness, efficiency, and superiority of the developed system and methods.

SPOT WELDING QUALITY FUZZY CONTROL SYSTEM BASED ON MULTISENSOR INFORMATION FUSION

The multisensor information fusion technology is adopted for real time measuring the four parameters which are connected closely with the weld nugget size（welding current, electrode displacement, dynamic resistance, welding time）, thus much more original information is obtained. In this way, the difficulty caused by measuring indirectly weld nugget size can be decreased in spot welding quality control, and the stability of spot welding quality can be improved. According to this method, two-dimensional fuzzy controllers are designed with the information fusion result as input and the thyristor control signal as output. The spot welding experimental results indicate that the spot welding quality intelligent control method based on multiscnsor information fusion technology can compensate the influence caused by variable factors in welding process and ensure the stability of welding quality.

A novel intelligent vehicle risk assessment method combined with multi-sensor fusion in dense traffic environment

Purpose–The purpose of this paper is to accurately capture the risks which are caused by each road user in time.Design/methodology/approach–The authors proposed a novel risk assessment approach based on the multi-sensor fusion algorithm in the real traffic environment.Firstly,they proposed a novel detection-level fusion approach for multi-object perception in dense traffic environment based on evidence theory.This approach integrated four states of track life into a generic fusion framework to improve the performance of multi-object perception.The information of object type,position and velocity was accurately obtained.Then,they conducted several experiments in real dense traffic environment on highways and urban roads,which enabled them to propose a novel road traffic risk modeling approach based on the dynamic analysis of vehicles in a variety of driving scenarios.By analyzing the generation process of traffic risks between vehicles and the road environment,the equivalent forces of vehicle–vehicle and vehicle–road were presented and theoretically calculated.The prediction steering angle and trajectory were considered in the determination of traffic risk influence area.Findings–The results of multi-object perception in the experiments showed that the proposed fusion approach achieved low false and missing tracking,and the road traffic risk was described as afield of equivalent force.The results extend the understanding of the traffic risk,which supported that the traffic risk from the front and back of the vehicle can be perceived in advance.Originality/value–This approach integrated four states of track life into a generic fusion framework to improve the performance of multi-object perception.The information of object type,position and velocity was used to reduce erroneous data association between tracks and detections.Then,the authors conducted several experiments in real dense traffic environment on highways and urban roads,which enabled them to propose a novel road traffic risk modeling approach based on the dynamic analysis of vehicles in a variety of driving scenarios.By analyzing the generation process of traffic risks between vehicles and the road environment,the equivalent forces of vehicle–vehicle and vehicle–road were presented and theoretically calculated.

Weighted Multi-sensor Data Level Fusion Method of Vibration Signal Based on Correlation Function

As the differences of sensor＇s precision and some random factors are difficult to control,the actual measurement signals are far from the target signals that affect the reliability and precision of rotating machinery fault diagnosis.The traditional signal processing methods,such as classical inference and weighted averaging algorithm usually lack dynamic adaptability that is easy for trends to cause the faults to be misjudged or left out.To enhance the measuring veracity and precision of vibration signal in rotary machine multi-sensor vibration signal fault diagnosis,a novel data level fusion approach is presented on the basis of correlation function analysis to fast determine the weighted value of multi-sensor vibration signals.The approach doesn＇t require knowing the prior information about sensors,and the weighted value of sensors can be confirmed depending on the correlation measure of real-time data tested in the data level fusion process.It gives greater weighted value to the greater correlation measure of sensor signals,and vice versa.The approach can effectively suppress large errors and even can still fuse data in the case of sensor failures because it takes full advantage of sensor＇s own-information to determine the weighted value.Moreover,it has good performance of anti-jamming due to the correlation measures between noise and effective signals are usually small.Through the simulation of typical signal collected from multi-sensors,the comparative analysis of dynamic adaptability and fault tolerance between the proposed approach and traditional weighted averaging approach is taken.Finally,the rotor dynamics and integrated fault simulator is taken as an example to verify the feasibility and advantages of the proposed approach,it is shown that the multi-sensor data level fusion based on correlation function weighted approach is better than the traditional weighted average approach with respect to fusion precision and dynamic adaptability.Meantime,the approach is adaptable and easy to use,can be applied to other areas of vibration measurement.

A Novel Voronoi Based Particle Filter for Multi-Sensor Data Fusion

Seamless and reliable navigation for civilian/military application is possible by fusing prominent Global Positioning System (GPS) with Inertial Navigation System (INS). This integrated GPS/INS unit exhibits a continuous navigation solution with increased accuracy and reduced uncertainty or ambiguity. In this paper, we propose a novel approach of dynamically creating a Voronoi based Particle Filter (VPF) for integrating INS and GPS data. This filter is based on redistribution of the proposal distribution such that the redistributed particles lie in high likelihood region;thereby increasing the filter accuracy. The usual limitations like degeneracy, sample impoverishment that are seen in conventional particle filter are overcome using our VPF with minimum feasible particles. The small particle size in our methodology reduces the computational load of the filter and makes real-time implementation feasible. Our field test results clearly indicate that the proposed VPF algorithm effectively compensated and reduced positional inaccuracies when GPS data is available. We also present the preliminary results for cases with short GPS outages that occur for low-cost inertial sensors.