Sim-to-Real: A Performance Comparison of PPO, TD3, and SAC Reinforcement Learning Algorithms for Quadruped Walking Gait Generation

The performance of the state-of-the-art Deep Reinforcement algorithms such as Proximal Policy Optimization, Twin Delayed Deep Deterministic Policy Gradient, and Soft Actor-Critic for generating a quadruped walking gait in a virtual environment was presented in previous research work titled “A Comparison of PPO, TD3, and SAC Reinforcement Algorithms for Quadruped Walking Gait Generation”. We demonstrated that the Soft Actor-Critic Reinforcement algorithm had the best performance generating the walking gait for a quadruped in certain instances of sensor configurations in the virtual environment. In this work, we present the performance analysis of the state-of-the-art Deep Reinforcement algorithms above for quadruped walking gait generation in a physical environment. The performance is determined in the physical environment by transfer learning augmented by real-time reinforcement learning for gait generation on a physical quadruped. The performance is analyzed on a quadruped equipped with a range of sensors such as position tracking using a stereo camera, contact sensing of each of the robot legs through force resistive sensors, and proprioceptive information of the robot body and legs using nine inertial measurement units. The performance comparison is presented using the metrics associated with the walking gait: average forward velocity (m/s), average forward velocity variance, average lateral velocity (m/s), average lateral velocity variance, and quaternion root mean square deviation. The strengths and weaknesses of each algorithm for the given task on the physical quadruped are discussed.

Robust Controller Design for Satellite Attitude Tracking System

The attitude tracking control problem for a satellite with parameter uncertainties and external disturbances is considered in this paper. For this class of multi-input multi-output uncertain nonlinear systems, a design method of robust output tracking controllers is proposed based on the upper-bounds of the uncertainties. Using the input/output feedback linearization approach and Lyapunov method, a control law is designed, which guarantees that the system output exponentially tracks the given desired output. The proposed controller is easy to compute and complement. Simulation results show that, in the closed-loop system, precise attitude control is accomplished in spite of the uncertainties in the system.

UAV target tracking algorithm based on task allocation consensus

In order to adapt to the changes in the states of moving targets and meet the requirements of quality of service in communication among unmanned aerial vehicles (UAVs), the UAVs formation needs to dynamically adjust tracking tasks and tracking paths, when they carry out a mission of tracking multiple moving targets. A multi-targets tracking algorithm based on task allocation consensus is proposed for UAVs to track multiple moving targets under the distributed control architecture in the limited communication range. This algorithm creates a distributed dynamic task allocation model using intermittent asynchronous communication principle to realize the sharing of observation information gathered by each UAV with fewer communications. In addition, this algorithm also makes it possible that multi-UAVs can plan tracking paths for multiple moving targets. We implement the algorithm on the formation with three UAVs to track three moving targets through simulation. Simulation results show the effectiveness of the proposed algorithm. © 2016 Beijing Institute of Aerospace Information.

The linear position tracking servo systemusing a linear voice-coil motor

More and more linear servo systems have been used in servo applications. Direct drive technology can greatly increase the bandwidth and the tracking accuracy. A position servo-system based on linear voice-coil motor was designed for one linear oscillation movement application. Besides the conventional position, speed and current control loops, the speed and acceleration feed-forward control of command position signal were also used. The experimental test proved the correctness of the design, and the system can track the given periodic sinusoid position command signal of 15Hz with high accuracy. The linear voice-coil motor is very suitable for short stroke position tracking application with high dynamic response.

Sequential Track Monitoring with Multiple 2-D Passive Sensors

Track monitoring is a fast method of determining incorrect return-to-track and track-to-track assignments. An analytical method to evaluate the effectiveness of track monitoring was presented by reference [1]. In order to reduce the probability of initiating or accepting the ghost track, two sequential track monitoring algorithms, which use the inclination angles to form test statistics, are proposed in this paper. Two decision rules are given and the corresponding thresholds are derived. The paper also gives the estimation methods of the noncentral parameter of noncentral chi-square distribution when the true value of it is unknown, the algorithms can adaptively estimate the decision thresholds and can make sequential track monitoring decision on line. Simulation is also made and the results show that the proposed algorithms are effective and feasible.

Speed-assigned Position Tracking Control of SRM With Adaptive Backstepping Control

A novel speed-assigned method is applied to the position tracking control of switched reluctance motor(SRM).A speed control freedom can be drawn into the position control through speed assignment. Adaptive backstepping control is used to design the position controller for the SRM. The accuracy of position tracking of the SRM can be enhanced with speed assignment. A disturbance observer is further designed to enhance the estimation accuracy of the unknown load torque. Simulation results certify that the design scheme is right and effective.

Global track extraction for probability hypothesis density filter

The probability hypothesis density (PHD), a well-known scheme for multi-target tracking in clutters, can obtain peaks of possible tracks, and its cluster-indexed method is widely accepted in further track extraction. However, the track extraction may face high risk in the case that the targets are so approached that it is hardly to discern their measurements. The concept of the distance between track sets in two adjacent times is defined and a consistency measure metric between any two peaks in two adjacent times is further proposed based on 'global information', containing spatial information (topology feature) among tracks, along with the temporal information of each track. Then, a global track extraction method is proposed based on the consistency belief and four decision rules. Via the simulation comparison with the cluster-indexed method, the proposed method can avoid the track break and mistake association. © 2016 Beijing Institute of Aerospace Information.

Application of Heterogeneous Multiple Feature Data Fusion to Multitarget Association and Tracking

In this paper we discuss a kind of multitarget tracking and association method based on the data fusion of heterogeneous multiple feature data gained by a sensor such as space state, signal amplitude, Doppler frequency and so on. In order to introduce quantitatively those heterogeneous multiple feature data which are possibly gained by a sensor into the discussion of tracking and association problem, we define a correlation measure which we explain as the generalization of conventional association decision. In conventional Nearest Neighbor method, the decision function can take only two values, 1 or 0, to represent the decision of association or not association. In our method, correlation measure can be take any real value from 0 to 1 to represent the extent of correlation. Considering the practical circumstances that some feature data might not be easily gained continuously, we introduce an effective factor to deal with these cases. In the paper we also discuss the comparative computer simulation tests and give the results.

Multimodal background model with noise and shadow suppression for moving object detection

A statistical multimodal background model was described for moving object detection in video surveillance. The solution to some of the problems such as illumination changes, initialization of model with moving objects, and shadows suppression was provided. The background samples were chosen by thresholding inter-frame differences, and the Gaussian kernel density estimation was used to estimate the probability density function of background intensity. Pixel's neighbor information was considered to remove noise due to camera jitter and small motion in the scene. The hue-max-min-diff color information was used to detect and suppress moving cast shadows. The effectiveness of the proposed method in the foreground segmentation was demonstrated in the traffic surveillance application.

Experimental study on aerodynamic characteristics of a high-speed train on viaducts in turbulent crosswinds

In this study, experiments were carried out to investigate aerodynamic characteristics of a high-speed train on viaducts in turbulent crosswinds using a 1:25 scaled sectional model wind-tunnel testing. Pressure measurements of two typical sections, one train-head section and one train-body section, at the windward and leeward tracks were conducted under the smooth and turbulence flows with wind attack angles between-6° and 6°, and the corresponding aerodynamic force coefficients were also calculated using the integral method. The experimental results indicate that the track position affects the mean aerodynamic characteristics of the vehicle, especially for the train-body section. The fluctuating pressure coefficients at the leeward track are more significantly affected by the bridge interference compared to those at the windward track. The effect of turbulence on the train-head section is less than that on the train-body section. Additionally, the mean aerodynamic force coefficients are almost negatively correlated to wind attack angles, which is more prominent for vehicles at the leeward track. Moreover, the lateral force plays a critical role in determining the corresponding overturning moment, especially on the train-body section.

Output Feedback Dynamic Surface Controller Design for Airbreathing Hypersonic Flight Vehicle

This paper addresses issues related to nonlinear robust output feedback controller design for a nonlinear model of airbreathing hypersonic vehicle. The control objective is to realize robust tracking of velocity and altitude in the presence of immeasurable states, uncertainties and varying flight conditions. A novel reduced order fuzzy observer is proposed to estimate the immeasurable states. Based on the information of observer and the measured states, a new robust output feedback controller combining dynamic surface theory and fuzzy logic system is proposed for airbreathing hypersonic vehicle. The closedloop system is proved to be semi-globally uniformly ultimately bounded (SUUB), and the tracking error can be made small enough by choosing proper gains of the controller, filter and observer. Simulation results from the full nonlinear vehicle model illustrate the effectiveness and good performance of the proposed control scheme. © 2014 Chinese Association of Automation.

Research on a combinatorial control method for coaxial rotor aircraft based on sliding mode

Aiming at the position and attitude tracking of coaxial rotor aircraft(CRA),this paper proposes a combinatorial control method of sliding mode control(SMC)coupled with proportional-integralderivative control(PIDC).Considering the complete description of flight dynamics,aerodynamics and airflow interference,the dynamical model of CRA is established.The dynamical model is simplified according to the actual flight,then the simplified dynamical model is divided into two subsystems:a fully-actuated subsystem and an under-actuated subsystem.The controller of the fully-actuated subsystem consists of a SMC controller coupled with a rate bounded PIDC controller,while the controller of the under-actuated subsystem is composed of a SMC controller.The sliding manifold is defined by combining the position and velocity tracking errors of the state variables for each subsystem.Lyapunov stability theory is used to verify the stability of the sliding mode controller,which ensures that all state trajectories of the system can reach and stay on the sliding mode surface,the uncertainty and external interference of the model are compensated.Simulation and experiment compared with the conventional PIDC are carried out,the results demonstrate the effectiveness and the robustness of the proposed control method of this paper.

Nonlinear cascade control of single-rod pneumatic actuator based on an extended disturbance observer

Precise position tracking control of the single-rod pneumatic actuator is considered and a nonlinear cascade controller is developed.The proposed controller comprises an extended disturbance observer(EDOB)and a nonlinear robust control law synthesized by the backstepping method.The EDOB is designed to estimate not only the influence of disturbances but also the parameter uncertainties.With the use of parameter and disturbance estimates,the nonlinear cascade controller,which consists of an outer position tracking loop and an inner load pressure loop,is further designed to attenuate the effects of parameter and disturbance estimation errors.The stability of the closed-loop system is proven by means of Lyapunov theory.Extensive comparative experimental results obtained verify the effectiveness of the proposed nonlinear cascade controller and its performance robustness to parameter and external disturbance variations in practical implementation.

Adaptive high precision position control of servo actuator with friction compensation using LuGre model

Adaptive control of servo actuator with nonlinear friction compensation is addressed. LuGre dynamic friction model is adopted to characterize the nonlinear friction and a new kind of slid ing mode observer is designed to estimate the internal immeasurable state of LuGre model. Based on the estimated friction state, adaptive laws are designed to identify the unknown model parameters and the external disturbances, and the system stability and asymptotic trajectory tracking perform ance are guaranteed by Lyapunov function. The position tracking performance is verified by the ex perimental results.

Simulation-based Optimal Design of α-β-γ-δ Filter

The existing third-order tracker known as α-β-γ-δ filter has been used for target tracking and predicting for years. The filter can track the target＇s position and velocity, but not the acceleration. To extend its capability, a new fourth-order target tracker called α-β-γ-δ filter is proposed. The main objective of this study was to find the optimal set of filter parameters that leads to minimum position tracking errors. The tracking errors between using the α-β-γ-δ filter and the α-β-γ-δ filter are compared. As a result, the new filter exhibits significant improvement in position tracking accuracy over the existing third-order filter, but at the expense of computational time in search of the optimal filter. To reduce the computational time, a simulation-based optimization technique via Taguchi method is introduced.

Review about Wireless Sensor Networks and the Internet of Things

Wireless sensor networks(WSNs)are created and affect our daily lives.You can find applications in various fields such as health,accident,life,manufacturing,production management,network management and many other fields.WSN now connects to the Internet of Things,connects the sensor to the Internet,and then uses it for collaboration and collaboration.However,when WSN is part of the internet we need to be able to study and analyze related terms.In this article,we’re going to look at different ways to getWSN online and identify the challenges that address in future as well.

Particle filter based visual tracking with multi-cue adaptive fusion

To improve the robustness of visual tracking in complex environments such as: cluttered backgrounds, partial occlusions, similar distraction and pose variations, a novel tracking method based on adaptive fusion and particle filter is proposed in this paper. In this method, the image color and shape cues are adaptively fused to represent the target observation; fuzzy logic is applied to dynamically adjust each cue weight according to its associated reliability in the past frame; particle filter is adopted to deal with non-linear and non-Gaussian problems in visual tracking. The method is demonstrated to be robust to illumination changes, pose variations, partial occlusions, cluttered backgrounds and camera motion for a test image sequence.

Tracking and localization for omni-directional mobile industrial robot using reflectors

In this paper, an improved tracking and local-ization algorithm of an omni-directional mobile industrialrobot is proposed to meet the high positional accuracyrequirement, improve the robot＇s repeatability positioningprecision in the traditional trilateral algorithm, and solvethe problem of pose lost in the moving process. Lasersensors are used to identify the reflectors, and by associ-ating the reflectors identified at a particular time with thereflectors at a previous time, an optimal triangular posi-tioning method is applied to realize the positioning andtracking of the robot. The experimental results show thatpositioning accuracy can be satisfied, and the repeatabilityand anti-jamming ability of the omni-directional mobileindustrial robot will be greatly improved via this algorithm.

A hand tracking algorithm with particle filter and improved GVF snake model

To solve the problem that the accurate information of hand cannot be obtained by particle filter, a hand tracking algorithm based on particle filter combined with skin-color adaptive gradient vector flow(GVF) snake model is proposed. Adaptive GVF and skin color adaptive external guidance force are introduced to the traditional GVF snake model, guiding the curve to quickly converge to the deep concave region of hand contour and obtaining the complex hand contour accurately. This algorithm realizes a real-time correction of the particle filter parameters, avoiding the particle drift phenomenon. Experimental results show that the proposed algorithm can reduce the root mean square error of the hand tracking by 53%, and improve the accuracy of hand tracking in the case of complex and moving background, even with a large range of occlusion.

Track before detect for point targets with particle filter in infrared image sequences

The problem of detecting and tracking point targets in a sequence of infrared images with very low signalto-noise ratio (SNR) is investigated in this paper. A track before detect algorithm for infrared (IR) point target is developed based on particle filter. The particle filter is used to estimate the state of the target in track stage. The unnormalized weights of the output of the filter are used to approximately construct the likelihood ratio for hypothesis test in detection stage. Experiment results with the real image sequences that SNR is about 2.0 show that the proposed algorithm can successfully detect and track point target.