A learning-based flexible autonomous motion control method for UAV in dynamic unknown environments

This paper presents a deep reinforcement learning(DRL)-based motion control method to provide unmanned aerial vehicles(UAVs)with additional flexibility while flying across dynamic unknown environments autonomously.This method is applicable in both military and civilian fields such as penetration and rescue.The autonomous motion control problem is addressed through motion planning,action interpretation,trajectory tracking,and vehicle movement within the DRL framework.Novel DRL algorithms are presented by combining two difference-amplifying approaches with traditional DRL methods and are used for solving the motion planning problem.An improved Lyapunov guidance vector field(LGVF)method is used to handle the trajectory-tracking problem and provide guidance control commands for the UAV.In contrast to conventional motion-control approaches,the proposed methods directly map the sensorbased detections and measurements into control signals for the inner loop of the UAV,i.e.,an end-to-end control.The training experiment results show that the novel DRL algorithms provide more than a 20%performance improvement over the state-ofthe-art DRL algorithms.The testing experiment results demonstrate that the controller based on the novel DRL and LGVF,which is only trained once in a static environment,enables the UAV to fly autonomously in various dynamic unknown environments.Thus,the proposed technique provides strong flexibility for the controller.

Relevant experience learning:A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Unmanned Aerial Vehicles(UAVs)play a vital role in military warfare.In a variety of battlefield mission scenarios,UAVs are required to safely fly to designated locations without human intervention.Therefore,finding a suitable method to solve the UAV Autonomous Motion Planning(AMP)problem can improve the success rate of UAV missions to a certain extent.In recent years,many studies have used Deep Reinforcement Learning(DRL)methods to address the AMP problem and have achieved good results.From the perspective of sampling,this paper designs a sampling method with double-screening,combines it with the Deep Deterministic Policy Gradient(DDPG)algorithm,and proposes the Relevant Experience Learning-DDPG(REL-DDPG)algorithm.The REL-DDPG algorithm uses a Prioritized Experience Replay(PER)mechanism to break the correlation of continuous experiences in the experience pool,finds the experiences most similar to the current state to learn according to the theory in human education,and expands the influence of the learning process on action selection at the current state.All experiments are applied in a complex unknown simulation environment constructed based on the parameters of a real UAV.The training experiments show that REL-DDPG improves the convergence speed and the convergence result compared to the state-of-the-art DDPG algorithm,while the testing experiments show the applicability of the algorithm and investigate the performance under different parameter conditions.

A homogenization-planning-tracking method to solve cooperative autonomous motion control for heterogeneous carrier dispatch systems

Taxiing aircraft and towed aircraft with drawbar are two typical dispatch modes on the flight deck of aircraft carriers. In this paper, a novel hierarchical solution strategy, named as the Homogenization-Planning-Tracking(HPT) method, to solve cooperative autonomous motion control for heterogeneous carrier dispatch systems is developed. In the homogenization layer, any towed aircraft system involved in the sortie task is abstracted into a virtual taxiing aircraft. This layer transforms the heterogeneous systems into a homogeneous configuration. Then in the planning layer, a centralized optimal control problem is formulated for the homogeneous system. Compared with conducting the path planning directly with the original heterogeneous system, the homogenization layer contributes to reduce the dimension and nonlinearity of the formulated optimal control problem in the planning layer and consequently improves the robustness and efficiency of the solution process. Finally, in the tracking layer, a receding horizon controller is developed to track the reference trajectory obtained in the planning layer. To improve the tracking performance,multi-objective optimization techniques are implemented offline in advance to determine optimal weight parameters used in the tracking layer. Simulations demonstrate that smooth and collision-free cooperative trajectory can be generated efficiently in the planning phase. And robust trajectory tracking can be realized in the presence of external disturbances in the tracking phase.The developed HPT method provides a promising solution to the autonomous deck dispatch for unmanned carrier aircraft in the future.