基于深度强化学习潜艇攻防对抗训练指挥决策研究

Research on Command Decision-making of Submarine Attack and Defense Confrontation Training Based on Deep Reinforcement Learning

潜艇和水面舰艇编队间的攻防对抗是潜艇作战研究的重点内容,如何确保潜艇在舰艇编队、反潜直升机等兵力的联合封锁下存活和突围,是对潜艇指挥决策的考验。为此,针对潜舰机博弈对抗场景,从深度强化学习和规则推理两个方面构建潜艇智能体,提出两种近端策略优化(Proximal Policy Optimization,PPO)算法改进机制,开展互博弈对抗和分布式训练,最终实现潜艇在对抗过程中的智能决策,相关技术路线和算法在兵棋对战平台上得到实施和验证,算法改进后的收敛速度和稳定性有了较大提升,可为潜艇智能指挥决策的研究提供技术参考。

The offensive and defensive confrontation between the submarine and the surface ship formation is the key content of submarine combat research.How to ensure that the submarine survives and breaks through the joint blockade of the ship formation and anti-submarine helicopters is a test of the submarine command decision.To this end,in view of the asymmetry of the submarine-ship-helicopter confrontation scenario,the submarine agent is constructed from two aspects of deep reinforcement learning and rule inference,and two Proximal Policy Optimization(PPO)algorithm improvement mechanisms are proposed.It carries out mutual game confrontation and distributed training,and finally realizes the intelligent decision-making of submarines in the confrontation process.Related technical routes and algorithms have been implemented and verified on the wargaming platform.The improved algorithm has greatly improved the convergence speed and stability.The research on submarine intelligent command decision-making provides technical reference.