基于大规模URL模式串匹配方法的网络过滤算法

Network filtering algorithm based on large scale URL pattern string matching method

针对电力信息网络需要处理海量URL(Uniform Resource Locator,URL)审查过滤的情况,提出了以经典的Wu-Manber算法为基础进行改良的一种大规模URL模式串匹配算法。该方法采用减少哈希冲突和精确校验次数的设计理念,同时引入多种优化措施,达到提高算法的匹配性能的目标。通过真实数据集上的测试表明,该算法的内存消耗较低,在大规模URL快速匹配方面的性能有很大提高。该检测方法可以应用到多个网络过滤场合。

In view of the situation of mass URL examination and filtering in the power information network,this paper proposes a large-scale URL pattern string matching algorithm based on the Wu-Manber algorithm. The proposed algorithm improves the matching of the algorithm from the angle of reducing the hash conflict and reducing the number of accurate checksum. Several optimization measures are introduced to improve the matching performance of the algorithm. The test on the real data set shows that the memory consumption of the algorithm is low,and the performance of the algorithm in large-scale URL rapid matching is greatly improved.