深度强化学习在变体飞行器自主外形优化中的应用

Deep Reinforcement Learning and Its Application on Autonomous Shape Optimization for Morphing Aircrafts

基于深度强化学习策略,研究了一类变体飞行器外形自主优化问题。以一种抽象化的变体飞行器为对象,给出其外形变化公式与最优外形函数等。结合深度学习与确定性策略梯度强化学习,设计深度确定性策略梯度(DDPG)学习步骤,使飞行器经过训练学习后具有较高的自主性和环境适应性,提高其在战场上的生存、应变和攻击能力。仿真结果表明,训练过程收敛较快,训练好的深度网络参数可以使飞行器在整个飞行任务过程中达到最优气动外形。

This paper considers a class of simplified morphing aircraft and autonomous shape optimization for aircraft based on deep reinforcement /earning is researched. Firstly, based on the model of an abstract morphing aircraft, the dynamic equation of shape and the optimal shape functions are derived. Then, by combining deep learning and reinforcement learning of deterministic policy gradient, we give the learning procedure of deep deterministic policy gradient （DDPG）. After learning and training for the deep network, the aircraft is equipped with higher autonomy and environmental adaptability, which will improve its adaptability, aggressivity and survivability in the battlefield. Simulation results demonstrate that the convergence speed of learning is relatively fast, and the optimized aerodynamic shape can be obtained autonomously during the whole flight by using the trained deep network parameters.