基于遗传算法的Web文档聚类算法

Web documents clustering algorithm based on GA

考虑到传统Web文档聚类算法聚类效果差、速度慢等问题,针对Web文档聚类算法进行深入研究,使用目标优化策略将Web文档聚类认为是最佳划归文档集合的范畴,并通过引入优化算法进行聚类划分。针对使用SVD表示的Web文档向量存在高维稀疏性等问题,使用LDA对Web文档簇的潜在语义子空间进行重构,从而降低Web文档向量空间的维数,最后在低维空间使用遗传算法进行寻优。常规的GA算法通常存在算法早熟以及局部寻优能力弱等问题。故提出一种改进型GA算法,通过引入自适应对偶种群、自适应终止规则以及新的生成子代规则来保证种群在迭代过程中的多样性以解决算法早熟问题,并且要提高算法的搜索效率以提高算法对局部寻优的能力。最后通过实验验证提出的基于改进型GA算法的Web文档聚类算法的聚类有效性。

Considering the traditional Web documents clustering algorithm has poor clustering effect and slow clustering speed,the Web documents clustering algorithm is studied in depth. The target optimization strategy is used to cluster the Web documents into the optimum document collection partition,and then the optimization algorithm is introduced to cluster and partition the documents. For the Web documents vector with SVD representation has highdimensional sparsity,LDA is used to reconstruct the potential semantic subspace of the Web documents clustering,so the dimension of the Web documents vector space can be reduced. The Web documents in lowdimensional space is searched by means of genetic algorithm（GA）. The general GA algorithm has the problems of premature convergence and poor local optimization ability,so an improved GA algorithm is proposed,which can ensure the population diversity in the iterative process and solve the algorithm premature problem by introducing adaptive dual population,adaptive termination rules and new filial generation rules. The algorithm searching efficiency can be promoted to improve the local optimization ability. The clustering validity of the proposed Web documents clustering algorithm based on the improved GA algorithm was verified by experiments.