基于课程强化学习的联合海空博弈决策模型训练方法

Training Method of Joint Sea-air Game Decision-Making Model Based on Curriculum Reinforcement Learning

针对多智能体深度强化学习在解决联合海空作战战术博弈决策模型难以训练优化问题,结合多智能体深度强化学习在智能化指挥决策问题中的应用性优势,以及课程学习在复杂问题研究中的改进优势,构建基于马尔可夫决策过程的联合海空战术决策过程模型,提出基于复杂度指数函数的任务复杂性度量方法,建立基于值分解网络算法的求解模型。针对一个典型联合海空作战战术决策场景,构建从易到难的课程学习任务和模型求解框架,设计针对任务的决策模型训练方法,在兵棋推演仿真系统上,对模型训练方法的可行性进行了验证。

Aiming at the problem that multi-agent deep reinforcement learning is difficult to train and optimize the tactical game decision-making model of joint air-sea combat,combined with the application advantages of multi-agent deep reinforcement learning in intelligent command decision-making problems and the improvement advantages of curriculum learning in complex problems,a tactical decision-making process model of air-sea joint based on Markov decision-making process is constructed,a task complexity measurement method based on complexity exponential function is proposed,and a solution model based on value decomposition network algorithm is established.Finally,aiming at a typical joint air-sea combat tactical decision-making scenario,a curriculum learning task and model solving framework from easy to difficult degree is constructed,and a training method of decision-making model for task is designed,the feasibility of the model training method is verified on the military game simulation system.