基于强化学习的舰船目标跟踪有限理性博弈算法研究

Research on Bounded Rational Game Algorithm for Ship Target Tracking Based on Reinforcement Learning

针对现实中的决策者并非总能完全理性分析问题的情况,提出有限理性下的追逃博弈算法。建立追逃博弈模型,先求解完全理性下博弈双方的鞍点策略。引入有限理性level-k模型,对追击者和躲避者思考策略的层次进行结构性假设,允许追逃双方具备不同的策略推理能力,并给出相应等级的值函数和策略,策略满足HJI方程。随着等级的增加,策略最终会趋于纳什均衡。由于HJI方程难以直接求解,基于强化学习的actor-critic算法进行求解,设计算法使追击者能够估算出躲避者的思维等级并采取合适的策略。以舰船为对象,将舰船运动简化为二维的数学模型,建立舰船追逃博弈模型,对其进行算法仿真验证。

Since decision-makers in reality are not always able to analyze problems perfectly rationally,a pursuit evasion game algorithm based on bounded rationality is proposed.It establishes a pursuit evasion game model and first solves the saddle point strategies of the two players under perfect rationality.Introducing the bounded rationality level-k model,a structural assumption is made on the level of thinking strategies for pursuers and evaders.It allows both parties to have different strategic reasoning abilities,and gives corresponding levels’value functions and strategies,which satisfy the HJI equation.As the level increases,the strategy will eventually tend towards Nash equilibrium.Due to the difficulty in directly solving the HJI equation,an actor critic algorithm based on reinforcement learning is used to solve it.The algorithm is designed to enable pursuers to estimate the thinking level of evaders and adopt appropriate strategies.Simplify the motion of a ship as a two-dimensional mathematical model,this paper establishes a ship pursuit and evasion game model,and performs algorithm simulation verification on it.