Collaborative positioning for swarms:A brief survey of vision,LiDAR and wireless sensors based methods

As positioning sensors,edge computation power,and communication technologies continue to develop,a moving agent can now sense its surroundings and communicate with other agents.By receiving spatial information from both its environment and other agents,an agent can use various methods and sensor types to localize itself.With its high flexibility and robustness,collaborative positioning has become a widely used method in both military and civilian applications.This paper introduces the basic fundamental concepts and applications of collaborative positioning,and reviews recent progress in the field based on camera,LiDAR(Light Detection and Ranging),wireless sensor,and their integration.The paper compares the current methods with respect to their sensor type,summarizes their main paradigms,and analyzes their evaluation experiments.Finally,the paper discusses the main challenges and open issues that require further research.

Vision‐audio fusion SLAM in dynamic environments

Moving humans,agents,and subjects bring many challenges to robot self‐localisation and environment perception.To adapt to dynamic environments,SLAM researchers typically apply several deep learning image segmentation models to eliminate these moving obstacles.However,these moving obstacle segmentation methods cost too much computation resource for the onboard processing of mobile robots.In the current industrial environment,mobile robot collaboration scenario,the noise of mobile robots could be easily found by on‐board audio‐sensing processors and the direction of sound sources can be effectively acquired by sound source estimation algorithms,but the distance estimation of sound sources is difficult.However,in the field of visual perception,the 3D structure information of the scene is relatively easy to obtain,but the recognition and segmentation of moving objects is more difficult.To address these problems,a novel vision‐audio fusion method that combines sound source localisation methods with a visual SLAM scheme is proposed,thereby eliminating the effect of dynamic obstacles on multi‐agent systems.Several heterogeneous robots experiments in different dynamic scenes indicate very stable self‐localisation and environment reconstruction performance of our method.

Performance of GPS and IMU sensor fusion using unscented Kalman filter for precise i-Boat navigation in infinite wide waters

The Unmanned Surface Vehicle(USV)navigation system needs an accurate,firm,and reliable performance to avoid obstacles,as well as carry out automatic movements during missions.The Global Positioning System(GPS)is often used in these systems to provide absolute position information.However,the GPS measurements are affected by external conditions such as atmospheric bias and multipath effects.This leads to the inability of the stand-alone GPS to provide accurate positioning for the USV systems.One of the solutions to correct the errors of this sensor is by conducting GPS and Inertial Measurement Unit(IMU)fusion.The IMU sensor is complementary to the GPS and not affected by external conditions.However,it accumulates noise as time elapses.Therefore,this study aims to determine the fusion of the GPS and IMU sensors for the i-Boat navigation system,which is a USV developed by Institut Teknologi Sepuluh Nopember(ITS)Surabaya.Using the Unscented Kalman filter(UKF),sensor fusion was carried out based on the state equation defined by the dynamic and kinematic mathematical model of ship motion in 6 degrees of freedom.Then the performance of this model was tested through several simulations using different combinations of attitude measurement data.Two scenarios were conducted in the simulations:attitude measurement inclusion and exclusion(Scenarios I and II,respectively).The results showed that the position estimation in Scenario II was better than in Scenario I,with the Root Mean Square Error(RMSE)value of 0.062 m.Further simulations showed that the presence of attitude measurement data caused a decrease in the fusion accuracy.The UKF simulation with eight measurement parameters(Scenarios A,B and C)and seven measurement parameters(Scenarios D,E and F),as well as analytical attitude movement,indicated that yaw data had the largest noise accumulation compared to roll and pitch.

Intent Pattern Recognition of Lower-limb Motion Based on Mechanical Sensors

Based on the regularity nature of lower-limb motion,an intent pattern recognition approach for above-knee prosthesis is proposed in this paper. To remedy the defects of recognizer based on electromyogram(EMG), we develop a pure mechanical sensor architecture for intent pattern recognition of lower-limb motion. The sensor system is composed of an accelerometer, a gyroscope mounted on the prosthetic socket, and two pressure sensors mounted under the sole. To compensate the delay in the control of prosthesis, the signals in the stance phase are used to predict the terrain and speed in the swing phase. Specifically, the intent pattern recognizer utilizes intraclass correlation coefficient(ICC) according to the Cartesian product of walking speed and terrain. Moreover, the sensor data are fused via DempsterShafer's theory. And hidden Markov model(HMM) is used to recognize the realtime motion state with the reference of the prior step. The proposed method can infer the prosthesis user's intent of walking on different terrain, which includes level ground,stair ascent, stair descent, up and down ramp. The experiments demonstrate that the intent pattern recognizer is capable of identifying five typical terrain-modes with the rate of 95.8%. The outcome of this investigation is expected to substantially improve the control performance of powered above-knee prosthesis.

Multi-rate sensor fusion-based adaptive discrete finite-time synergetic control for flexible-joint mechanical systems

This paper proposes an adaptive discrete finite-time synergetic control （ADFTSC） scheme based on a multi-rate sensor fusion estimator for flexible-joint mechanical systems in the presence of unmeasured states and dynamic uncertainties. Multi-rate sensors are employed to observe the system states which cannot be directly obtained by encoders due to the existence of joint flexibilities. By using an extended Kalman filter （EKF）, the finite-time synergetic controller is designed based on a sensor fusion estimator which estimates states and parameters of the mechanical system with multi-rate measurements. The proposed controller can guarantee the finite-time convergence of tracking errors by the theoretical derivation. Simulation and experimental studies are included to validate the effectiveness of the proposed approach.

Obstacle detection using multi-sensor fusion

This paper presents an obstacle detection approach for blind pedestrians by fusing data from camera and laser sensor.For purely vision-based blind guidance system,it is difficult to discriminate low-level obstacles with cluttered road surface,while for purely laser-based system,it usually requires to scan the forward environment,which turns out to be very inconvenient.To overcome these inherent problems when using camera and laser sensor independently,a sensor-fusion model is proposed to associate range data from laser domain with edges from image domain.Based on this fusion model,obstacle's position,size and shape can be estimated.The proposed method is tested in several indoor scenes,and its efficiency is confirmed.

Sensor Registration Based on Neural Network in Data Fusion

The contents of sensor registration in the multi-sensor data fusion system are introduced, and some existing methods are analyzed. Then, one approach to sensor registration based on BP neural network is proposed. Here the measurements from radar are transformed from the polar coordinate system to the Cartesian coordinate through a BP neural network. With this approach, the systematic errors are removed as well as the coordinate is transformed. The efficiency of this method is demonstrated by simulation, and the result show that this approach could remove the systematic errors effectively and the DAR are closer to real position than DBR.

Real-Time Indoor Path Planning Using Object Detection for Autonomous Flying Robots

Unknown closed spaces are a big challenge for the navigation of robots since there are no global and pre-defined positioning options in the area.One of the simplest and most efficient algorithms,the artificial potential field algorithm(APF),may provide real-time navigation in those places but fall into local mini-mum in some cases.To overcome this problem and to present alternative escape routes for a robot,possible crossing points in buildings may be detected by using object detection and included in the path planning algorithm.This study utilized a proposed sensor fusion method and an improved object classification method for detecting windows,doors,and stairs in buildings and these objects were classified as valid or invalid for the path planning algorithm.The performance of the approach was evaluated in a simulated environment with a quadrotor that was equipped with camera and laser imaging detection and ranging(LIDAR)sensors to navigate through an unknown closed space and reach a desired goal point.Inclusion of crossing points allows the robot to escape from areas where it is con-gested.The navigation of the robot has been tested in different scenarios based on the proposed path planning algorithm and compared with other improved APF methods.The results showed that the improved APF methods and the methods rein-forced with other path planning algorithms were similar in performance with the proposed method for the same goals in the same room.For the goals outside the current room,traditional APF methods were quite unsuccessful in reaching the goals.Even though improved methods were able to reach some outside targets,the proposed method gave approximately 17%better results than the most success-ful example in achieving targets outside the current room.The proposed method can also work in real-time to discover a building and navigate between rooms.

Tracking Pedestrians Under Occlusion in Parking Space

Many traffic accidents occur in parking lots.One of the serious safety risks is vehicle-pedestrian conflict.Moreover,with the increasing development of automatic driving and parking technology,parking safety has received significant attention from vehicle safety analysts.However,pedestrian protection in parking lots still faces many challenges.For example,the physical structure of a parking lot may be complex,and dead corners would occur when the vehicle density is high.These lead to pedestrians’sudden appearance in the vehicle’s path from an unexpected position,resulting in collision accidents in the parking lot.We advocate that besides vehicular sensing data,high-precision digital map of the parking lot,pedestrians’smart device’s sensing data,and attribute information of pedestrians can be used to detect the position of pedestrians in the parking lot.However,this subject has not been studied and explored in existing studies.Tofill this void,this paper proposes a pedestrian tracking framework integrating multiple information sources to provide pedestrian position and status information for vehicles and protect pedestrians in parking spaces.We also evaluate the proposed method through real-world experiments.The experimental results show that the proposed framework has its advantage in pedestrian attribute information extraction and positioning accuracy.It can also be used for pedestrian tracking in parking spaces.

A Monitoring Method for Transmission Tower Foots Displacement Based on Wind-Induced Vibration Response

The displacement of transmission tower feet can seriously affect the safe operation of the tower,and the accuracy of structural health monitoring methods is limited at the present stage.The application of deep learning method provides new ideas for structural health monitoring of towers,but the current amount of tower vibration fault data is restricted to provide adequate training data for Deep Learning(DL).In this paper,we propose a DT-DL based tower foot displacement monitoring method,which firstly simulates the wind-induced vibration response data of the tower under each fault condition by finite element method.Then the vibration signal visualization and Data Transfer(DT)are used to add tower fault data samples to solve the problem of insufficient actual data quantity.Subsequently,the dynamic response test is carried out under different tower fault states,and the tower fault monitoring is carried out by the DL method.Finally,the proposed method is compared with the traditional online monitoring method,and it is found that this method can significantly improve the rate of convergence and recognition accuracy in the recognition process.The results show that the method can effectively identify the tower foot displacement state,which can greatly reduce the accidents that occurred due to the tower foot displacement.

Distributed computations for large-scale networked systems using belief propagation

This paper introduces several related distributed algorithms,generalised from the celebrated belief propagation algorithm for statistical learning.These algorithms are suitable for a class of computational problems in largescale networked systems,ranging from average consensus,sensor fusion,distributed estimation,distributed optimisation,distributed control,and distributed learning.By expressing the underlying computational problem as a sparse linear system,each algorithm operates at each node of the network graph and computes iteratively the desired solution.The behaviours of these algorithms are discussed in terms of the network graph topology and parameters of the corresponding computational problem.A number of examples are presented to illustrate their applications.Also introduced is a message-passing algorithm for distributed convex optimisation.

Study on key technologies of GNSS-based train state perception for traincentric railway signaling

The application of Global Navigation Satellite Systems(GNSSs)in the intelligent railway systems is rapidly developing all over the world.With the GNSs-based train positioning and moving state perception,the autonomy and flexibility of a novel train control system can be greatly enhanced over the existing solutions relying on the track-side facilities.Considering the safety critical features of the railway signaling applications,the GNSS stand-alone mode may not be sufficient to satisfy the practical requirements.In this paper,the key technologies for applying GNSS in novel train-centric railway signaling systems are investigated,including the multi-sensor data fusion,Virtual Balise(VB)capturing and messaging,train integrity monitoring and system performance evaluation.According to the practical characteristics of the novel train control system under the moving block mode,the details of the key technologies are introduced.Field demonstration results of a novel train control system using the presented technologies under the practical railway operation conditions are presented to illustrate the achievable performance feature of autonomous train state perception using BeiDou Navigation Satellite System(BDS)and related solutions.It reveals the great potentials of these key technologies in the next generation train control system and other GNSS-based railway implementations.

Target Detection on Water Surfaces Using Fusion of Camera and LiDAR Based Information

To address the challenges of missed detections in water surface target detection using solely visual algorithms in unmanned surface vehicle(USV)perception,this paper proposes a method based on the fusion of visual and LiDAR point-cloud projection for water surface target detection.Firstly,the visual recognition component employs an improved YOLOv7 algorithmbased on a self-built dataset for the detection of water surface targets.This algorithm modifies the original YOLOv7 architecture to a Slim-Neck structure,addressing the problemof excessive redundant information during feature extraction in the original YOLOv7 network model.Simultaneously,this modification simplifies the computational burden of the detector,reduces inference time,and maintains accuracy.Secondly,to tackle the issue of sample imbalance in the self-built dataset,slide loss function is introduced.Finally,this paper replaces the original Complete Intersection over Union(CIoU)loss function with the Minimum Point Distance Intersection over Union(MPDIoU)loss function in the YOLOv7 algorithm,which accelerates model learning and enhances robustness.To mitigate the problem of missed recognitions caused by complex water surface conditions in purely visual algorithms,this paper further adopts the fusion of LiDAR and camera data,projecting the threedimensional point-cloud data from LiDAR onto a two-dimensional pixel plane.This significantly reduces the rate of missed detections for water surface targets.

Road Friction Estimation under Complicated Maneuver Conditions for Active Yaw Control

Road friction coefficient is a key factor for the stability control of the vehicle dynamics in the critical conditions. Obviously the vehicle dynamics stability control systems, including the anti-lock brake system（ABS）, the traction control system（TCS）, and the active yaw control（AYC） system, need the accurate tire and road friction information. However, the simplified method based on the linear tire and vehicle model could not obtain the accurate road friction coefficient for the complicated maneuver of the vehicle. Because the active braking control mode of AYC is different from that of ABS, the road friction coefficient cannot be estimated only with the dynamics states of the tire. With the related dynamics states measured by the sensors of AYC, a comprehensive strategy of the road friction estimation for the active yaw control is brought forward with the sensor fusion technique. Firstly, the variations of the dynamics characteristics of vehicle and tire, and the stability control mode in the steering process are considered, and then the proper road friction estimation methods are brought forward according to the vehicle maneuver process. In the steering maneuver without braking, the comprehensive road friction from the four wheels may be estimated based on the multi-sensor signal fusion method. The estimated values of the road friction reflect the road friction characteristic. When the active brake involved, the road friction coefficient of the braked wheel may be estimated based on the brake pressure and tire forces, the estimated values reflect the road friction between the braked wheel and the road. So the optimal control of the wheel slip rate may be obtained according to the road friction coefficient. The methods proposed in the paper are integrated into the real time controller of AYC, which is matched onto the test vehicle. The ground tests validate the accuracy of the proposed method under the complicated maneuver conditions.

Application of a Novel Method for Machine Performance Degradation Assessment Based on Gaussian Mixture Model and Logistic Regression

The currently prevalent machine performance degradation assessment techniques involve estimating a machine＇s current condition based upon the recognition of indications of failure features,which entail complete data collected in different conditions.However,failure data are always hard to acquire,thus making those techniques hard to be applied.In this paper,a novel method which does not need failure history data is introduced.Wavelet packet decomposition（WPD） is used to extract features from raw signals,principal component analysis（PCA） is utilized to reduce feature dimensions,and Gaussian mixture model（GMM） is then applied to approximate the feature space distributions.Single-channel confidence value（SCV） is calculated by the overlap between GMM of the monitoring condition and that of the normal condition,which can indicate the performance of single-channel.Furthermore,multi-channel confidence value（MCV）,which can be deemed as the overall performance index of multi-channel,is calculated via logistic regression（LR） and that the task of decision-level sensor fusion is also completed.Both SCV and MCV can serve as the basis on which proactive maintenance measures can be taken,thus preventing machine breakdown.The method has been adopted to assess the performance of the turbine of a centrifugal compressor in a factory of Petro-China,and the result shows that it can effectively complete this task.The proposed method has engineering significance for machine performance degradation assessment.

A non-destructive method to measure the thermal properties of frozen soils during phase transition

Frozen soils cover about 40% of the land surface on the earth and are responsible for the global energybalances affecting the climate. Measurement of the thermal properties of frozen soils during phasetransition is important for analyzing the thermal transport process. Due to the involvement of phasetransition, the thermal properties of frozen soils are rather complex. This paper introduces the uses of amultifunctional instrument that integrates time domain reflectometry （TDR） sensor and thermal pulsetechnology （TPT） to measure the thermal properties of soil during phase transition. With this method,the extent of phase transition （freezing/thawing） was measured with the TDR module; and the correspondingthermal properties were measured with the TPT module. Therefore, the variation of thermalproperties with the extent of freezing/thawing can be obtained. Wet soils were used to demonstrate theperformance of this measurement method. The performance of individual modules was first validatedwith designed experiments. The new sensor was then used to monitor the properties of soils duringfreezingethawing process, from which the freezing/thawing degree and thermal properties weresimultaneously measured. The results are consistent with documented trends of thermal propertiesvariations. 2015 Institute of Rock and Soil Mechanics, Chinese Academy of Sciences. Production and hosting byElsevier B.V. All rights reserved.

Study on Federated Architecture for GPS/INS/TRN Integrated Navigation System

Based on the information fusion theory, a kind of integrated navigation system integration for cruise missile is presented in this paper. Besides, the way with which the system is integrated and the related data fusion technique are discussed. Information-fusion-based hybrid navigation system integration can fully utilize information provided by all kinds of navigation sensor subsystem and can improve the precision of the system effectively. Simultaneously, the reconstructing ability ensures the system of great reliability.

On-line Tool Wear Classification in Unmanned-machining Environments

One of the most important features of the modern ma ch ining system in an "unmanned" factory is to change tools that have been subjec ted to wear and damage. An integrated tool condition monitoring system composed of multi-sensors, signal processing devices and intelligent decision making pla ns is a necessary requirement for automatic manufacturing processes. An intellig ent tool wear monitoring system will be introduced in this paper. The system is equipped with power consumption, vibration, AE and cutting force sensors, signal transformation and collection apparatus and a microcomputer. Tool condition monitoring is a pattern recognition process in which the characte ristics of the tool to be monitored are compared with those of the standard mode ls. The tool wear classification process is composed of the following parts: fea ture extraction; determination of the fuzzy membership functions of the features ; calculation of the fuzzy similarity; learning and tool wear classification. Fe atures extracted from the time domain and frequency domain for the future patter n recognition are as follows. Power consumption signal: mean value; AE-RMS sign al: mean value, skew and kutorsis; Cutting force, AE and vibration signal: mean value, standard deviation and the mean power in 10 frequency ranges. These signa l features can reflect the tool wear states comprehensively. The fuzzy approachi ng degree and the fuzzy distance between corresponding features of different obj ects are combined to describe the closeness of two fuzzy sets more accurately. A unique fuzzy driven neural network based pattern recognition algorithm has bee n developed from this research. The combination of Artificial Neural Networks (A NNs) and fuzzy logic system integrates the strong learning and classification ab ility of the former and the superb flexibility of the latter to express the dist ribution characteristics of signal features with vague boundaries. This methodol ogy indirectly solves the automatic weight assignment problem of the conventiona l fuzzy pattern recognition system and let it have greater representative power, higher training speed and be more robust. The introduction of the two-dimensio nal weighted approaching degree can make the pattern recognition process more re liable. The fuzzy driven neural network can effectively fuse multi-sensor i nformation and successfully recognize the tool wear states. Armed with the advan ced pattern recognition methodology, the established intelligent tool condition monitoring system has the advantages of being suitable for different machini ng conditions, robust to noise and tolerant to faults. Cooperated with the contr ol system of the machine tool, the optimized machining processed can be achieved .

A Survey of Intelligent Sensing Technologies in Autonomous Driving

Intelligent perception technology of sensors in autonomous vehicles has been deeply integrated with the algorithm of autonomous driving.This paper provides a survey of the impact of sensing technologies on autonomous driving,including the intelligent perception reshaping the car architecture from distributed to centralized processing and the common perception algorithms being explored in autonomous driving vehicles,such as visual perception,3D perception and sensor fusion.The pure visual sensing solutions have shown the powerful capabilities in 3D perception leveraging the latest self-supervised learning progress,compared with light detection and ranging(LiDAR)-based solutions.Moreover,we discuss the trends on end-to-end policy decision models of high-level autonomous driving technologies.

Multi-layer perception approach to identification of compound information

Presents a novel approach of multi layer sensing for perception of high level environmental information related to many conventional physical quantities, such as temperature, humidity and brightness, which focuses on the processing of multi functional variables in a multi layer framework, and consists of multi functional sensing and multi layer fusion. Concerning the first aspect, a CdS and Fe 3O 4 materials based multi function sensor has been developed to measure the three quantities, and provides a possible solution to the sensor multi functional measurement equations, especially when the sensor processes more than three quantities, and proposes ways to evaluate the concerned environment as degree of comfort, Quantity Creditability Tactics (QCT) of multi layer data fusion.