基于机器学习的日志分析系统的设计与实现

Design and Implementation of Log Analysis System based on Machine Learning

随着网络应用的发展,针对Web服务的恶意攻击也日益增多,如何在第一时间找到恶意的攻击日志,确认攻击者IP和漏洞位置,为后续的漏洞修补和攻击溯源提供有效帮助,是本文的研究重点。本系统通过漏洞测试工具收集恶意请求URL,使用Python的Sklearn(Scikit-learn)框架实现SVM(SupportVectorMachines,支持向量机)模型,对收集到的恶意URL进行关键词和特征提取,再对模型进行训练,训练结果通过Pickle方式保存。使用本系统可以对常见的漏洞利用方式如SQL注入、XSS、远程代码执行等进行检测,为Web服务的安全运行以及漏洞修复、重新上线等提供有效帮助,减少漏洞攻击事件带来的损失。

With the development of network applications, malicious attacks against Web services are also increasing.The research focuses on how to find the malicious attack log at the first time, confirm the attacker’s IP and vulnerability location, and provide effective help for subsequent vulnerability repair and attack traceability. Malicious request URLs are collected through vulnerability testing tools and Python’s Sklearn(Scikit-learn) framework is used to implement SVM(Support Vector Machines) model. Keywords and features are extracted from the collected malicious URLs and then the models are trained. The training results are saved in Pickle mode. This system can detect common vulnerability utilization methods such as SQL(Structured Query Language) injection, XSS(Cross Site Script), remote code execution, etc., and provide effective help for the safe operation of Web services, vulnerability repair and re-launch, so to reduce the losses caused by vulnerability attacks.