一种基于HTML动态化的恶意爬虫主动防御方法

An Active Defense Method of Malicious Crawler Based on HTML Dynamic

目前针对爬虫的防御技术主要是根据检测结果对恶意爬虫进行封禁,检测和防御机制单一,无法应对复杂多变的恶意爬虫,检测误判率高,容易造成误封、漏封。本文提出一种基于移动目标防御思想的恶意爬虫主动防御技术,使用一种新型的HTML动态化方法,结合自适应跳变策略来遏制爬虫规则的制订和后续的恶意操作。通过实验对比,本文提出的主动防御的方法可以有效地限制恶意爬虫自动化、批量化的行为;在保证数据内容安全的前提下有效地降低了其对服务性能的影响。

Nowadays, the researches of malicious crawler defense mainly depends on the detection results to block the malicious crawlers IP address. The single detection and defense mechanism unable to deal with the complex and changeable malicious crawlers, the high error rate of detection resulting in false blocking and leaky blocking. This paper proposes a malicious crawler active defense technology based on the idea of moving target defense, and uses a new dynamic method of HTML. Combined with adaptive jump strategy to curb the formulation of crawler rules and subsequent malicious operations. Through experimental comparison, the active defense method proposed in this paper can effectively limit the automated and batch behavior of malicious crawlers;in the premise of ensuring the security of data content, it can effectively reduce the impact on service performance.