Collaborative positioning for swarms:A brief survey of vision,LiDAR and wireless sensors based methods

As positioning sensors,edge computation power,and communication technologies continue to develop,a moving agent can now sense its surroundings and communicate with other agents.By receiving spatial information from both its environment and other agents,an agent can use various methods and sensor types to localize itself.With its high flexibility and robustness,collaborative positioning has become a widely used method in both military and civilian applications.This paper introduces the basic fundamental concepts and applications of collaborative positioning,and reviews recent progress in the field based on camera,LiDAR(Light Detection and Ranging),wireless sensor,and their integration.The paper compares the current methods with respect to their sensor type,summarizes their main paradigms,and analyzes their evaluation experiments.Finally,the paper discusses the main challenges and open issues that require further research.

Target Detection on Water Surfaces Using Fusion of Camera and LiDAR Based Information

To address the challenges of missed detections in water surface target detection using solely visual algorithms in unmanned surface vehicle(USV)perception,this paper proposes a method based on the fusion of visual and LiDAR point-cloud projection for water surface target detection.Firstly,the visual recognition component employs an improved YOLOv7 algorithmbased on a self-built dataset for the detection of water surface targets.This algorithm modifies the original YOLOv7 architecture to a Slim-Neck structure,addressing the problemof excessive redundant information during feature extraction in the original YOLOv7 network model.Simultaneously,this modification simplifies the computational burden of the detector,reduces inference time,and maintains accuracy.Secondly,to tackle the issue of sample imbalance in the self-built dataset,slide loss function is introduced.Finally,this paper replaces the original Complete Intersection over Union(CIoU)loss function with the Minimum Point Distance Intersection over Union(MPDIoU)loss function in the YOLOv7 algorithm,which accelerates model learning and enhances robustness.To mitigate the problem of missed recognitions caused by complex water surface conditions in purely visual algorithms,this paper further adopts the fusion of LiDAR and camera data,projecting the threedimensional point-cloud data from LiDAR onto a two-dimensional pixel plane.This significantly reduces the rate of missed detections for water surface targets.

Intent Pattern Recognition of Lower-limb Motion Based on Mechanical Sensors

Based on the regularity nature of lower-limb motion,an intent pattern recognition approach for above-knee prosthesis is proposed in this paper. To remedy the defects of recognizer based on electromyogram(EMG), we develop a pure mechanical sensor architecture for intent pattern recognition of lower-limb motion. The sensor system is composed of an accelerometer, a gyroscope mounted on the prosthetic socket, and two pressure sensors mounted under the sole. To compensate the delay in the control of prosthesis, the signals in the stance phase are used to predict the terrain and speed in the swing phase. Specifically, the intent pattern recognizer utilizes intraclass correlation coefficient(ICC) according to the Cartesian product of walking speed and terrain. Moreover, the sensor data are fused via DempsterShafer's theory. And hidden Markov model(HMM) is used to recognize the realtime motion state with the reference of the prior step. The proposed method can infer the prosthesis user's intent of walking on different terrain, which includes level ground,stair ascent, stair descent, up and down ramp. The experiments demonstrate that the intent pattern recognizer is capable of identifying five typical terrain-modes with the rate of 95.8%. The outcome of this investigation is expected to substantially improve the control performance of powered above-knee prosthesis.

Mobile robot localization algorithm based on multi-sensor information fusion

In order to effectively reduce the uncertainty error of mobile robot localization with a single sensor and improve the accuracy and robustness of robot localization and mapping,a mobile robot localization algorithm based on multi-sensor information fusion(MSIF)was proposed.In this paper,simultaneous localization and mapping(SLAM)was realized on the basis of laser Rao-Blackwellized particle filter(RBPF)-SLAM algorithm and graph-based optimization theory was used to constrain and optimize the pose estimation results of Monte Carlo localization.The feature point extraction and quadrilateral closed loop matching algorithm based on oriented FAST and rotated BRIEF(ORB)were improved aiming at the problems of generous calculation and low tracking accuracy in visual information processing by means of the three-dimensional(3D)point feature in binocular visual reconstruction environment.Factor graph model was used for the information fusion under the maximum posterior probability criterion for laser RBPF-SLAM localization and binocular visual localization.The results of simulation and experiment indicate that localization accuracy of the above-mentioned method is higher than that of traditional RBPF-SLAM algorithm and general improved algorithms,and the effectiveness and usefulness of the proposed method are verified.

Fault diagnosis method of hydraulic system based on fusion of neural network and D-S evidence theory

According to fault type diversity and fault information uncertainty problem of the hydraulic driven rocket launcher servo system（HDRLSS） , the fault diagnosis method based on the evidence theory and neural network ensemble is proposed. In order to overcome the shortcomings of the single neural network, two improved neural network models are set up at the com-mon nodes to simplify the network structure. The initial fault diagnosis is based on the iron spectrum data and the pressure, flow and temperature（PFT） characteristic parameters as the input vectors of the two improved neural network models, and the diagnosis result is taken as the basic probability distribution of the evidence theory. Then the objectivity of assignment is real-ized. The initial diagnosis results of two improved neural networks are fused by D-S evidence theory. The experimental results show that this method can avoid the misdiagnosis of neural network recognition and improve the accuracy of the fault diagnosis of HDRLSS.

Multi-rate sensor fusion-based adaptive discrete finite-time synergetic control for flexible-joint mechanical systems

This paper proposes an adaptive discrete finite-time synergetic control （ADFTSC） scheme based on a multi-rate sensor fusion estimator for flexible-joint mechanical systems in the presence of unmeasured states and dynamic uncertainties. Multi-rate sensors are employed to observe the system states which cannot be directly obtained by encoders due to the existence of joint flexibilities. By using an extended Kalman filter （EKF）, the finite-time synergetic controller is designed based on a sensor fusion estimator which estimates states and parameters of the mechanical system with multi-rate measurements. The proposed controller can guarantee the finite-time convergence of tracking errors by the theoretical derivation. Simulation and experimental studies are included to validate the effectiveness of the proposed approach.

Vision‐audio fusion SLAM in dynamic environments

Moving humans,agents,and subjects bring many challenges to robot self‐localisation and environment perception.To adapt to dynamic environments,SLAM researchers typically apply several deep learning image segmentation models to eliminate these moving obstacles.However,these moving obstacle segmentation methods cost too much computation resource for the onboard processing of mobile robots.In the current industrial environment,mobile robot collaboration scenario,the noise of mobile robots could be easily found by on‐board audio‐sensing processors and the direction of sound sources can be effectively acquired by sound source estimation algorithms,but the distance estimation of sound sources is difficult.However,in the field of visual perception,the 3D structure information of the scene is relatively easy to obtain,but the recognition and segmentation of moving objects is more difficult.To address these problems,a novel vision‐audio fusion method that combines sound source localisation methods with a visual SLAM scheme is proposed,thereby eliminating the effect of dynamic obstacles on multi‐agent systems.Several heterogeneous robots experiments in different dynamic scenes indicate very stable self‐localisation and environment reconstruction performance of our method.

Performance of GPS and IMU sensor fusion using unscented Kalman filter for precise i-Boat navigation in infinite wide waters

The Unmanned Surface Vehicle(USV)navigation system needs an accurate,firm,and reliable performance to avoid obstacles,as well as carry out automatic movements during missions.The Global Positioning System(GPS)is often used in these systems to provide absolute position information.However,the GPS measurements are affected by external conditions such as atmospheric bias and multipath effects.This leads to the inability of the stand-alone GPS to provide accurate positioning for the USV systems.One of the solutions to correct the errors of this sensor is by conducting GPS and Inertial Measurement Unit(IMU)fusion.The IMU sensor is complementary to the GPS and not affected by external conditions.However,it accumulates noise as time elapses.Therefore,this study aims to determine the fusion of the GPS and IMU sensors for the i-Boat navigation system,which is a USV developed by Institut Teknologi Sepuluh Nopember(ITS)Surabaya.Using the Unscented Kalman filter(UKF),sensor fusion was carried out based on the state equation defined by the dynamic and kinematic mathematical model of ship motion in 6 degrees of freedom.Then the performance of this model was tested through several simulations using different combinations of attitude measurement data.Two scenarios were conducted in the simulations:attitude measurement inclusion and exclusion(Scenarios I and II,respectively).The results showed that the position estimation in Scenario II was better than in Scenario I,with the Root Mean Square Error(RMSE)value of 0.062 m.Further simulations showed that the presence of attitude measurement data caused a decrease in the fusion accuracy.The UKF simulation with eight measurement parameters(Scenarios A,B and C)and seven measurement parameters(Scenarios D,E and F),as well as analytical attitude movement,indicated that yaw data had the largest noise accumulation compared to roll and pitch.

Obstacle detection using multi-sensor fusion

This paper presents an obstacle detection approach for blind pedestrians by fusing data from camera and laser sensor.For purely vision-based blind guidance system,it is difficult to discriminate low-level obstacles with cluttered road surface,while for purely laser-based system,it usually requires to scan the forward environment,which turns out to be very inconvenient.To overcome these inherent problems when using camera and laser sensor independently,a sensor-fusion model is proposed to associate range data from laser domain with edges from image domain.Based on this fusion model,obstacle's position,size and shape can be estimated.The proposed method is tested in several indoor scenes,and its efficiency is confirmed.

Sensor Registration Based on Neural Network in Data Fusion

The contents of sensor registration in the multi-sensor data fusion system are introduced, and some existing methods are analyzed. Then, one approach to sensor registration based on BP neural network is proposed. Here the measurements from radar are transformed from the polar coordinate system to the Cartesian coordinate through a BP neural network. With this approach, the systematic errors are removed as well as the coordinate is transformed. The efficiency of this method is demonstrated by simulation, and the result show that this approach could remove the systematic errors effectively and the DAR are closer to real position than DBR.

Application of data fusion on multi-function earth drill

taking the bucket of multi function earth drill as an example, combining with the conception of multi sensor integration and data fusion, adopting the terrene column chart and digging torque formula as control dependence, the detecting method of the earth drill’s working state is introduced. Multi sensor data fusion is done with the aid of BP neural network in Matlab. The data to be interfused are pre processed and the program of simulation and “point checking” is given.

Road Friction Estimation under Complicated Maneuver Conditions for Active Yaw Control

Road friction coefficient is a key factor for the stability control of the vehicle dynamics in the critical conditions. Obviously the vehicle dynamics stability control systems, including the anti-lock brake system（ABS）, the traction control system（TCS）, and the active yaw control（AYC） system, need the accurate tire and road friction information. However, the simplified method based on the linear tire and vehicle model could not obtain the accurate road friction coefficient for the complicated maneuver of the vehicle. Because the active braking control mode of AYC is different from that of ABS, the road friction coefficient cannot be estimated only with the dynamics states of the tire. With the related dynamics states measured by the sensors of AYC, a comprehensive strategy of the road friction estimation for the active yaw control is brought forward with the sensor fusion technique. Firstly, the variations of the dynamics characteristics of vehicle and tire, and the stability control mode in the steering process are considered, and then the proper road friction estimation methods are brought forward according to the vehicle maneuver process. In the steering maneuver without braking, the comprehensive road friction from the four wheels may be estimated based on the multi-sensor signal fusion method. The estimated values of the road friction reflect the road friction characteristic. When the active brake involved, the road friction coefficient of the braked wheel may be estimated based on the brake pressure and tire forces, the estimated values reflect the road friction between the braked wheel and the road. So the optimal control of the wheel slip rate may be obtained according to the road friction coefficient. The methods proposed in the paper are integrated into the real time controller of AYC, which is matched onto the test vehicle. The ground tests validate the accuracy of the proposed method under the complicated maneuver conditions.

Application of a Novel Method for Machine Performance Degradation Assessment Based on Gaussian Mixture Model and Logistic Regression

The currently prevalent machine performance degradation assessment techniques involve estimating a machine＇s current condition based upon the recognition of indications of failure features,which entail complete data collected in different conditions.However,failure data are always hard to acquire,thus making those techniques hard to be applied.In this paper,a novel method which does not need failure history data is introduced.Wavelet packet decomposition（WPD） is used to extract features from raw signals,principal component analysis（PCA） is utilized to reduce feature dimensions,and Gaussian mixture model（GMM） is then applied to approximate the feature space distributions.Single-channel confidence value（SCV） is calculated by the overlap between GMM of the monitoring condition and that of the normal condition,which can indicate the performance of single-channel.Furthermore,multi-channel confidence value（MCV）,which can be deemed as the overall performance index of multi-channel,is calculated via logistic regression（LR） and that the task of decision-level sensor fusion is also completed.Both SCV and MCV can serve as the basis on which proactive maintenance measures can be taken,thus preventing machine breakdown.The method has been adopted to assess the performance of the turbine of a centrifugal compressor in a factory of Petro-China,and the result shows that it can effectively complete this task.The proposed method has engineering significance for machine performance degradation assessment.

A non-destructive method to measure the thermal properties of frozen soils during phase transition

Frozen soils cover about 40% of the land surface on the earth and are responsible for the global energybalances affecting the climate. Measurement of the thermal properties of frozen soils during phasetransition is important for analyzing the thermal transport process. Due to the involvement of phasetransition, the thermal properties of frozen soils are rather complex. This paper introduces the uses of amultifunctional instrument that integrates time domain reflectometry （TDR） sensor and thermal pulsetechnology （TPT） to measure the thermal properties of soil during phase transition. With this method,the extent of phase transition （freezing/thawing） was measured with the TDR module; and the correspondingthermal properties were measured with the TPT module. Therefore, the variation of thermalproperties with the extent of freezing/thawing can be obtained. Wet soils were used to demonstrate theperformance of this measurement method. The performance of individual modules was first validatedwith designed experiments. The new sensor was then used to monitor the properties of soils duringfreezingethawing process, from which the freezing/thawing degree and thermal properties weresimultaneously measured. The results are consistent with documented trends of thermal propertiesvariations. 2015 Institute of Rock and Soil Mechanics, Chinese Academy of Sciences. Production and hosting byElsevier B.V. All rights reserved.

Study on Federated Architecture for GPS/INS/TRN Integrated Navigation System

Based on the information fusion theory, a kind of integrated navigation system integration for cruise missile is presented in this paper. Besides, the way with which the system is integrated and the related data fusion technique are discussed. Information-fusion-based hybrid navigation system integration can fully utilize information provided by all kinds of navigation sensor subsystem and can improve the precision of the system effectively. Simultaneously, the reconstructing ability ensures the system of great reliability.

On-line Tool Wear Classification in Unmanned-machining Environments

One of the most important features of the modern ma ch ining system in an "unmanned" factory is to change tools that have been subjec ted to wear and damage. An integrated tool condition monitoring system composed of multi-sensors, signal processing devices and intelligent decision making pla ns is a necessary requirement for automatic manufacturing processes. An intellig ent tool wear monitoring system will be introduced in this paper. The system is equipped with power consumption, vibration, AE and cutting force sensors, signal transformation and collection apparatus and a microcomputer. Tool condition monitoring is a pattern recognition process in which the characte ristics of the tool to be monitored are compared with those of the standard mode ls. The tool wear classification process is composed of the following parts: fea ture extraction; determination of the fuzzy membership functions of the features ; calculation of the fuzzy similarity; learning and tool wear classification. Fe atures extracted from the time domain and frequency domain for the future patter n recognition are as follows. Power consumption signal: mean value; AE-RMS sign al: mean value, skew and kutorsis; Cutting force, AE and vibration signal: mean value, standard deviation and the mean power in 10 frequency ranges. These signa l features can reflect the tool wear states comprehensively. The fuzzy approachi ng degree and the fuzzy distance between corresponding features of different obj ects are combined to describe the closeness of two fuzzy sets more accurately. A unique fuzzy driven neural network based pattern recognition algorithm has bee n developed from this research. The combination of Artificial Neural Networks (A NNs) and fuzzy logic system integrates the strong learning and classification ab ility of the former and the superb flexibility of the latter to express the dist ribution characteristics of signal features with vague boundaries. This methodol ogy indirectly solves the automatic weight assignment problem of the conventiona l fuzzy pattern recognition system and let it have greater representative power, higher training speed and be more robust. The introduction of the two-dimensio nal weighted approaching degree can make the pattern recognition process more re liable. The fuzzy driven neural network can effectively fuse multi-sensor i nformation and successfully recognize the tool wear states. Armed with the advan ced pattern recognition methodology, the established intelligent tool condition monitoring system has the advantages of being suitable for different machini ng conditions, robust to noise and tolerant to faults. Cooperated with the contr ol system of the machine tool, the optimized machining processed can be achieved .

Real-Time Indoor Path Planning Using Object Detection for Autonomous Flying Robots

Unknown closed spaces are a big challenge for the navigation of robots since there are no global and pre-defined positioning options in the area.One of the simplest and most efficient algorithms,the artificial potential field algorithm(APF),may provide real-time navigation in those places but fall into local mini-mum in some cases.To overcome this problem and to present alternative escape routes for a robot,possible crossing points in buildings may be detected by using object detection and included in the path planning algorithm.This study utilized a proposed sensor fusion method and an improved object classification method for detecting windows,doors,and stairs in buildings and these objects were classified as valid or invalid for the path planning algorithm.The performance of the approach was evaluated in a simulated environment with a quadrotor that was equipped with camera and laser imaging detection and ranging(LIDAR)sensors to navigate through an unknown closed space and reach a desired goal point.Inclusion of crossing points allows the robot to escape from areas where it is con-gested.The navigation of the robot has been tested in different scenarios based on the proposed path planning algorithm and compared with other improved APF methods.The results showed that the improved APF methods and the methods rein-forced with other path planning algorithms were similar in performance with the proposed method for the same goals in the same room.For the goals outside the current room,traditional APF methods were quite unsuccessful in reaching the goals.Even though improved methods were able to reach some outside targets,the proposed method gave approximately 17%better results than the most success-ful example in achieving targets outside the current room.The proposed method can also work in real-time to discover a building and navigate between rooms.

A Survey of Intelligent Sensing Technologies in Autonomous Driving

Intelligent perception technology of sensors in autonomous vehicles has been deeply integrated with the algorithm of autonomous driving.This paper provides a survey of the impact of sensing technologies on autonomous driving,including the intelligent perception reshaping the car architecture from distributed to centralized processing and the common perception algorithms being explored in autonomous driving vehicles,such as visual perception,3D perception and sensor fusion.The pure visual sensing solutions have shown the powerful capabilities in 3D perception leveraging the latest self-supervised learning progress,compared with light detection and ranging(LiDAR)-based solutions.Moreover,we discuss the trends on end-to-end policy decision models of high-level autonomous driving technologies.

Tracking Pedestrians Under Occlusion in Parking Space

Many traffic accidents occur in parking lots.One of the serious safety risks is vehicle-pedestrian conflict.Moreover,with the increasing development of automatic driving and parking technology,parking safety has received significant attention from vehicle safety analysts.However,pedestrian protection in parking lots still faces many challenges.For example,the physical structure of a parking lot may be complex,and dead corners would occur when the vehicle density is high.These lead to pedestrians’sudden appearance in the vehicle’s path from an unexpected position,resulting in collision accidents in the parking lot.We advocate that besides vehicular sensing data,high-precision digital map of the parking lot,pedestrians’smart device’s sensing data,and attribute information of pedestrians can be used to detect the position of pedestrians in the parking lot.However,this subject has not been studied and explored in existing studies.Tofill this void,this paper proposes a pedestrian tracking framework integrating multiple information sources to provide pedestrian position and status information for vehicles and protect pedestrians in parking spaces.We also evaluate the proposed method through real-world experiments.The experimental results show that the proposed framework has its advantage in pedestrian attribute information extraction and positioning accuracy.It can also be used for pedestrian tracking in parking spaces.

SIGNATURE DRIVEN MULTIPLE TARGET TRACKING

Tracking multiple maneuvering targets remains a challenge due to the existence of clutter and the disturbance of spurious targets. Traditional tracking algorithms treat target measurements as points which results in the loss of information. We have propose a Signature Driven multiple-target Tracking (SDT) method which uses target signature in spectral,spatial and temporary spaces as well as the Markov property of target motion,and the data association process in SDT is very effective. The experimental results have shown its outstanding performance.