自动化红队测试中强化学习策略的实现与验证

Implementation and Verification of Reinforcement Learning Strategy in Automated Red Teaming Testing

红队测试是一种通过模拟真实黑客攻击行为来对网络系统进行安全测评的方法。然而,目前人工测试存在成本较高与适应性较差的问题。红队测试智能化与自动化是当前研究的热点问题,旨在降低红队测试的成本,提高网络安全测评的测试性能与测试效率。自动化攻击策略是自动化红队测试的核心,其作用是替代安全专家进行攻击技术的决策。文中将红队攻击技术映射到强化学习,从而将红队测试过程建模为马尔可夫决策模型,通过有限状态机模型实现了固定策略与强化学习策略;在真实网络环境中对不同的强化学习策略进行训练和测试,验证了强化学习策略的收敛性和可行性。实验结果表明,基于SARSA(λ)算法的强化学习策略优于其他强化学习策略,收敛速度最快;3种强化学习策略均能在测试实验中稳定完成测试目标,且性能远优于固定策略。

Red teaming testing is a method to evaluate the security of network system by simulating real hacker attack behavior.However,manual test has the problems of high cost and poor adaptability at present.Red teaming testing intelligence and automation is currently a hot research topic,aiming at reducing the cost of red teaming testing and improving the test performance and efficiency of cybersecurity assessments.Automated attack strategy is the core of automated red teaming testing,it is designed to replace security experts in the attack technology decision-making process.In this paper,the red teaming attack technique is mapped to reinforcement learning,the red teaming testing process is modeled as a Markov decision process model,and the fixed strategy and reinforcement learning strategy are implemented through the finite state machine.Reinforcement learning strategy is trained and tested in the real network environment to verify the convergence and feasibility.Experimental results show that the SARSA(λ)algorithm is superior to other reinforcement learning algorithms and has the fastest convergence speed.The three reinforcement learning strategies can achieve the test objective stably in the test experiment,and the performance is much better than that of the fixed strategy.