基于语义相似度的Web文档聚类算法

Web document clustering algorithm based on semantic similarity

文章提出基于语义相似度的Web文档聚类算法——WDCSS算法,依据文档关键词之间的相似度生成最小树,通过概率统计来确定最小树中相似度阈值,并对最小树中进行切割,同时对较小的子类进行划分合并。实验表明,WDCSS不仅能为具有各种不同聚类形状的数据集准确地分析出数据中存在的合理聚类和例外样本,而且避免了用户参数选择所造成聚类质量降低问题。

A Web document clustering algorithm using semantic similarity（WDCSS） is presented in this paper. Firstly, the minimal spanning tree is obtained from the similarity of document key words. Secondly, the smallest similarity threshold is determined based on the analysis of probability and statistics. Finally, the minimal spanning tree is cut. Simultaneously, the subclasses are divided and merged. The experiment indicates that the WDCSS can not only accurately analyze the reasonable clusters and the exceptional samples from data sets with different cluster shapes, but it can also avoid the decreasing of the clustering quality influenced by the choice of users＇parameters.