基于强化学习的体系对抗仿真战役层次指控算法

A RL-based command and control algorithm for SoS confrontation simulation at the tactical level

针对传统的认知决策技术无法有效应对体系对抗环境具有的不确定性、未知性以及复杂性问题,提出一种基于强化学习(RL)的体系对抗仿真战役层次指控算法。介绍了包含侦察类、打击类、通信类、补给类、修复类以及指控类Agent的UML体系架构,对自主开发的作战仿真原型系统及其作战想定进行了说明,在对战役层次指控Agent认知域描述与假设的基础上,对改进Q-learning认知决策算法的参数归一化、基于GRBF神经网络的Q离散、基于TD公式的跨步差分机制以及网络结构的学习训练过程进行了详细说明。最后,通过地空一体化联合体系对抗仿真验证了算法的有效性,并通过对算法的大量可视化回溯分析发现,一定程度的火力协调以及不间断的战术机动对于作战效能的提升以及毁伤的减免具有重要的意义。

Aiming at the problem that the traditional cognition techniques are not adaptive to the uncertainty and complexity in the Weapon System-of-Systems（WSoS）confrontation environment,a Command and Control（C2）algorithm based on Reinforcement Learning（RL）is proposed for the WSoS confrontation simulation at the tactical level.The UML architecture of WSoS that consists of a communication class,scouting class,attacking class,command class,supplying class and repairing class is designed and the battle simulation platform with the battle scenario is introduced.Then,based on the illustration and hypothesis for the command agent＇s cognition problem,the parameter＇s normalization,the discrete of the Q table based on GRBF neural network,the strip temporal difference mechanism and the learning process of the structure of the network are explained for the improved Q-leaning cognition algorithm.Finally,the validation and effectiveness of the algorithm is proved through the battle simulation experiment of the air-ground unify confrontation SoS.Besides,through a lot of visualization recall analysis for the C2 algorithm,we found that the coordination of the firepower and the continuous tactical maneuver are important to the operational effectiveness and injure decrease.