基于近似动态规划的多级火箭全程任务决策

Multi-stage Rocket Mission Decision Based on Approximate Dynamic Programming

针对火箭发生推力下降故障下的任务决策问题,提出了一种基于近似动态规划的多级火箭全程任务决策方法。首先,通过设置初始状态集合、决策选项、奖励函数、Q函数迭代方法等,建立了火箭任务决策分层强化学习模型,得到对火箭后续飞行进行评价的“评价网络”;然后利用基于凸优化的在线能力评估和轨迹规划方法,得到近似动态规划原理中的“决策生成”模块;最后,通过两者结合完成对火箭故障下后续飞行中连续轨迹和各级飞行段离散轨道根数等的决策。仿真结果表明该方法能够在非致命推力下降故障下实现火箭全程飞行任务决策并给出飞行轨迹。

A multistage mission decision method based on approximate dynamic programming is proposed to solve the problem of mission decision in the case of launch vehicle thrust drop faults.Firstly,a hierarchical reinforcement learning model for launch vehicle mission decision is established by setting the initial state set,decision options,reward function,Q-function iteration,etc.,and an“evaluation network”is obtained to evaluate the subsequent flight of the rocket.Then,an online capability evaluation and trajectory planning method based on convex optimization is used to obtain the“decision generation”module in the frame structure of approximate dynamic programming.Finally,the decision of continuous trajectory and discrete orbital parameters at all levels of each flight stage under launch vehicle failure is completed by combining the two parts.Numerical simulation results show that the multistage launch vehicle mission decision is effectively in the event of launch vehicle thrust failures by the proposed method.