Efficient Penetration Testing Path Planning Based on Reinforcement Learning with Episodic Memory

Intelligent penetration testing is of great significance for the improvement of the security of information systems,and the critical issue is the planning of penetration test paths.In view of the difficulty for attackers to obtain complete network information in realistic network scenarios,Reinforcement Learning(RL)is a promising solution to discover the optimal penetration path under incomplete information about the target network.Existing RL-based methods are challenged by the sizeable discrete action space,which leads to difficulties in the convergence.Moreover,most methods still rely on experts’knowledge.To address these issues,this paper proposes a penetration path planning method based on reinforcement learning with episodic memory.First,the penetration testing problem is formally described in terms of reinforcement learning.To speed up the training process without specific prior knowledge,the proposed algorithm introduces episodic memory to store experienced advantageous strategies for the first time.Furthermore,the method offers an exploration strategy based on episodic memory to guide the agents in learning.The design makes full use of historical experience to achieve the purpose of reducing blind exploration and improving planning efficiency.Ultimately,comparison experiments are carried out with the existing RL-based methods.The results reveal that the proposed method has better convergence performance.The running time is reduced by more than 20%.

Software Vulnerability Mining and Analysis Based on Deep Learning

In recent years,the rapid development of computer software has led to numerous security problems,particularly software vulnerabilities.These flaws can cause significant harm to users’privacy and property.Current security defect detection technology relies on manual or professional reasoning,leading to missed detection and high false detection rates.Artificial intelligence technology has led to the development of neural network models based on machine learning or deep learning to intelligently mine holes,reducing missed alarms and false alarms.So,this project aims to study Java source code defect detection methods for defects like null pointer reference exception,XSS(Transform),and Structured Query Language(SQL)injection.Also,the project uses open-source Javalang to translate the Java source code,conducts a deep search on the AST to obtain the empty syntax feature library,and converts the Java source code into a dependency graph.The feature vector is then used as the learning target for the neural network.Four types of Convolutional Neural Networks(CNN),Long Short-Term Memory(LSTM),Bi-directional Long Short-Term Memory(BiLSTM),and Attention Mechanism+Bidirectional LSTM,are used to investigate various code defects,including blank pointer reference exception,XSS,and SQL injection defects.Experimental results show that the attention mechanism in two-dimensional BLSTM is the most effective for object recognition,verifying the correctness of the method.