搜索引擎中基于密度聚类的混合编码检测算法

Density-Based Clustering Algorithm for Hybrid Coding Detection in Search Engines

搜索引擎有很多的关键技术,本文主要针对互联网中文HTML混合编码文件,研究了中文HTML文件的字符编码组成结构,然后对混合编码文件内容进行聚类,采用了数据挖掘领域的经典算法DBSCAN,将HTML文件分成几个大类,然后分别对各个类进行了基于特征编码检测。实验结果显示,当选取合适的参数时,对混合编码文件的聚类后,每个类与中文字符特征编码相符率达100%,可以广泛应用于搜索领域。

Aimed at Chinese HTML hybrid coding documents on the internet,this paper studies the character encoding composition of Chinese HTML files and clusters the contents of the hybrid coding files.The HTML files are separated into several categories using the classical data mining algorithms DBSCAN.Then,based on feature encoding each class is detected,after clustering hybrid encoding files.Experimental results show that when selecting the appropriate parameters each class in line with the Chinese character encoding features reaches 100%.The method can be used in the field of search engines.