基于强化学习的深空探测器自主任务规划方法

An Autonomous Planning Method for Deep Space Exploration Tasks in Reinforcement Learning Based on Dynamic Rewards

针对深空探测器自主任务规划多约束的需求,提出了基于动态奖励的强化学习深空探测器任务自主规划模型构建方法,建立了深空探测器智能体的交互环境,构建了策略网络和融合资源、时间以及时序约束的损失函数,并提出动态奖励机制对传统策略梯度学习方法进行了改进。仿真实验结果表明:该方法可实现自主任务规划,规划成功率和规划效率相比静态奖励策略梯度算法有明显的提升,并且能在任意状态下开始规划而无需改变模型结构,提高了对不确定规划任务的适应性。该方法为深空探测器自主任务规划与决策提供了一种新的解决方案。

Aiming at the characteristics of multi-system parallelism and the need to meet various constraints in the proceAiming at the characteristics of multi-system parallelism and the need to meet various constraints in the process of autonomous mission planning of deep space detectors,a reinforcement learning task autonomous planning model construction method for deep space detectors was proposed based on dynamic rewards,and a deep space detector agent was established.In the interactive environment,a policy network and a loss function integrating resource constraints,time constraints and timing constraints were constructed,and a dynamic reward mechanism was proposed to improve the traditional policy gradient learning method.The simulation results show that the method in this paper could realize autonomous task planning.Compared with the static reward policy gradient algorithm,the planning success rate and planning efficiency were significantly improved,and the method could start planning in any state without changing the model structure,which improved the accuracy of the algorithm.This method provides a new solution for autonomous mission planning and decision-making of deep space probes.