基于深度强化学习的驾驶仪参数快速整定方法

Autopilot parameter rapid tuning method based on deep reinforcement learning

针对深度强化学习方法对驾驶仪控制参数训练速度慢、奖励函数收敛性不好等问题,以三回路驾驶仪极点配置算法为核心,提出一种将三维控制参数转换为一维设计参量的智能训练方法,构建离线深度强化学习训练叠加在线多层感知器神经网络实时计算的智能控制架构,在提高深度强化学习算法的效率和奖励函数收敛性同时,确保在大范围飞行状态变化条件下控制参数的快速在线自整定。以典型再入飞行器为例,完成深度强化学习训练和神经网络部署。仿真结果表明,强化学习动作空间简化后的训练效率更高,训练得到的驾驶仪对控制指令的跟踪误差在1.2%以内。

Aiming at the problem of slow training speed and poor convergence of deep reinforcement learning method for the autopilot control parameters training,an intelligent training method that converts three-dimensional control parameters into one-dimensional design parameters is proposed with the three-loop autopilot pole placement method as the core.The intelligent control architecture of offline deep reinforcement learning training and online multi-layer perceptron neural network real-time calculation is constructed,which improves the efficiency and convergence of deep reinforcement learning algorithm and ensures the rapid online tuning of control parameters under the condition of large-scale flight state changes.Taking a typical reentry aircraft as an example,the deep reinforcement learning training and neural network deployment are accomplished.The simulation results show that the training efficiency of the simplified reinforcement learning action space is higher,and the tracking error of the controller to the control command is less than 1.2%by the proposed parameter rapid tuning method based on deep reinforcement learning.