一种棋类计算机博弈强化学习智能体的决策依据解释方法

An Interpretation Method of Decision Basis for the Reinforcement Learning Agent of Chess Computer Game

提出一种针对棋类智能体,基于显著图的解释方法,用来解释棋类智能体的决策依据。该解释方法将智能体面对的棋局状态分为落子区域和空白区域,从特征与最终决策的相关性和重要性两方面对棋局落子区域特征进行显著性分析,同时从博弈行为角度出发,分析棋局空白区域特征对于智能体决策的显著性,将两区域的特征显著性进行合并输出,从而较为全面地评估整个棋局特征的显著性。针对基于强化学习的围棋智能体和国际象棋智能体,使用该方法对智能体的决策进行解释性输出,实验结果表明:可以全面地解释智能体的决策依据,验证了该方法的有效性。

In recent years,interpretation methods based on saliency maps have become the main method to explain the decision-making basis of reinforcement learning agents.Such methods are mainly based on perturbation to generate saliency maps to show the saliency of situational features for the agent’s decision-making,that is,its importance in decision-making.Sexual presentation,the saliency of these characteristics represents the basis for the decision-making of the agent.This type of perturbation-based method often deletes features,calculates the degree of influence of the change on the agent’s decision,and uses this as the saliency of each feature in decision-making.However,when this type of method explains the decision-making of the chess agent,it cannot take into account the distinctiveness of the characteristics of the blank area on the chess board.The blank area in the chess game is also very important for the agent’s decision-making.To this end,this article proposes an interpretation method for chess agents based on saliency graphs to explain the decision-making basis of chess agents.The interpretation method divides the chess game state faced by the agent into a move area and a blank area.From the aspects of the relevance and importance of the characteristics and the final decision,the significance analysis of the regional features of the chess game is carried out.At the same time,from the perspective of game behavior,the significance of the features of the blank area of??the chess game for the decision of the agent is analyzed,and the characteristics of the two regions are finally distinguished.After Combining the output of the characteristics,the significance of the characteristics of the whole game is comprehensively evaluated.Finally,for the go agent and chess agent based on reinforcement learning,the method is used to interpret the decision of the agent.From the experimental results,the decision basis of the agent can be fully explained,which verifies the effectiveness of the method.