摘要
针对深度强化学习方法对驾驶仪控制参数训练速度慢、奖励函数收敛性不好等问题,以三回路驾驶仪极点配置算法为核心,提出一种将三维控制参数转换为一维设计参量的智能训练方法,构建离线深度强化学习训练叠加在线多层感知器神经网络实时计算的智能控制架构,在提高深度强化学习算法的效率和奖励函数收敛性同时,确保在大范围飞行状态变化条件下控制参数的快速在线自整定。以典型再入飞行器为例,完成深度强化学习训练和神经网络部署。仿真结果表明,强化学习动作空间简化后的训练效率更高,训练得到的驾驶仪对控制指令的跟踪误差在1.2%以内。
Aiming at the problem of slow training speed and poor convergence of deep reinforcement learning method for the autopilot control parameters training,an intelligent training method that converts three-dimensional control parameters into one-dimensional design parameters is proposed with the three-loop autopilot pole placement method as the core.The intelligent control architecture of offline deep reinforcement learning training and online multi-layer perceptron neural network real-time calculation is constructed,which improves the efficiency and convergence of deep reinforcement learning algorithm and ensures the rapid online tuning of control parameters under the condition of large-scale flight state changes.Taking a typical reentry aircraft as an example,the deep reinforcement learning training and neural network deployment are accomplished.The simulation results show that the training efficiency of the simplified reinforcement learning action space is higher,and the tracking error of the controller to the control command is less than 1.2%by the proposed parameter rapid tuning method based on deep reinforcement learning.
作者
万齐天
卢宝刚
赵雅心
温求遒
WAN Qitian;LU Baogang;ZHAO Yaxin;WEN Qiuqiu(School of Aerospace Engineering,Beijing Institute of Technology,Beijing 100081,China;Beijing Institute of Space Long March Vehicle,Beijing 100076,China;China Academy of Launch Vehicle Technology,Beijing 100076,China)
出处
《系统工程与电子技术》
EI
CSCD
北大核心
2022年第10期3190-3199,共10页
Systems Engineering and Electronics
基金
航空科学基金(202037012003)资助课题。
关键词
强化学习
自动驾驶仪
参数整定
智能控制
归一化
reinforcement learning
autopilot
parameter tuning
intelligent control
normalization