基于DDPG算法的末制导律设计研究

Terminal Guidance Law Design Based on DDPG Algorithm

末制导律设计是拦截系统中的关键技术,常用的比例制导律及其变型在目标大机动时性能下降,且受到导航比的影响.提出基于DDPG算法的末制导律设计方法,通过对拦截问题的环境状态和动作(控制量)进行设计,实现了从仿真环境交互数据中学习回报最优的制导律;与传统方法相比,该无模型方法更具灵活性;针对强化学习方法动作集假设偏置弱带来训练效率低的问题,进一步提出将导航比作为决策优化参数,加速了训练过程并实现动态调整比例制导律中的导航比.对比实验表明,两种强化学习末制导律设计方法获得了优于比例制导律及其变型的拦截效果,展现出良好的研究前景和潜在的应用价值.

The design of terminal guidance law is the key technology in interception system.The performance of the commonly used proportional guidance law and its variants will degrade under the condition of a large maneuvering target and will be affected by the navigation ratio.A terminal guidance law design method based on the DDPG algorithm is proposed.By designing the environment state and action(control quantity)of interception problem,the guidance law with optimal learning reward from the interactive data of simulation environment is realized.Compared with the traditional method,the model-free method is more flexible.Aiming at the problem of low training efficiency caused by weak hypothesis bias of action set in reinforcement learning method,a further proposal is proposed taking the navigation ratio as the decision optimization parameter,the training process is accelerated and the navigation ratio in proportional guidance law is adjusted dynamically.The comparative experiments show that the two design methods of terminal guidance law based on reinforcement learning obtain better interception effect than proportional guidance law and its variants,showing good research prospects and potential application value.