基于分层强化学习的智能化攻击路径发现方法

Intelligent Attack Path Discovery Based on Hierarchical Reinforcement Learning

智能化攻击路径发现是开展自动化渗透测试的一项关键技术,但现有方法面临着状态、动作空间呈指数型增长和奖励稀疏等问题,导致算法难以收敛。为此,提出了一种基于分层强化学习的智能化攻击路径发现方法iPathD(Intelligent Path Discovery)。iPathD将攻击路径发现过程构建为一个分层的马尔可夫决策过程,以分别描述上层的主机间渗透路径发现和下层的单主机内部攻击路径发现,并在此基础上提出并实现了一种基于分层强化学习的攻击路径发现算法。实验结果表明,与传统基于DQN(Deep Q Learning)及其改进算法的方法相比,iPathD路径发现方法更加快速有效,并且随着主机中漏洞数目的增加,iPathD的效果更好,且适用于大规模的网络场景。

Intelligent attack path discovery is a key technology for automated penetration testing,but existing methods face the problems of exponential growth of state and action space and sparse rewards,which make the algorithm difficult to converge.To this end,an intelligent attack path discovery method(iPathD)based on hierarchical reinforcement learning is proposed.iPathD constructs the attack path discovery process as a layered Markov decision process to describe the upper-layer inter-host penetration path discovery and the lower-layer single-host internal attack path discovery,respectively.On this basis,an attack path discovery algorithm based on hierarchical reinforcement learning is proposed and implemented.Experimental results show that compared with the traditional method based on deep Q learning(DQN)and its improved algorithm,the iPathD path discovery method is faster and more effective.With the increase of the number of vulnerabilities in the host,the effect of iPathD is better,and it is suitable for large-scale network scenarios.