K-均值文本聚类算法在智能答疑系统中应用

Application on K-means Text Clustering Algorithm in Intelligent Question Answering System

本文将数据挖掘算法应用于智能答疑系统中,提出了一套基于数据挖掘算法的答疑设计方案并加以改进,传统的K-均值算法聚类虽然速度快,在文本聚类中易于实现,但其同样依赖于所有变量,聚类效果往往不尽如人意。为了克服这一缺点,提出一种改进的K-均值文本聚类算法,它在K-均值聚类过程中,向每一个聚类簇中的关键词自动计算添加一个权重,重要的关键词赋予较大的权重。经过实验测试,获得了一种基于子空间变量自动加权的适合文本数据聚类分析的改进算法,它不仅可以在大规模、高维和稀疏的文本数据上有效地进行聚类,还能够生成质量较高的聚类结果。实验结果表明基于子空间变量自动加权的K-均值文本聚类算法是有效的大规模文本数据聚类算法。

The paper applies the data mining algorithm into the QA system,puts forward a set of scheme about question answering system based on data mining algorithm and improves it. K-means is one of the widely used text clustering techniques due to its rapidity, simplicity and high scalability. However, since tradi- tional K-means algorithm treats all variables equally as well as the sparse of text characteristicmatrix, it is not good enough in clustering effect. In this paper it proposes an improved K-means text clustering algorithm. In the process of K-means clustering, it can automatically append the weight value to keywords in each cluster, but the important keywords will be assigned the greater value.Through experiments and tests, the researchers obtained an optimized algorithm based on subspace variable self-weighting which suits the text data clustering analysis, it can cluster large-scale, high dimension and sparse text data effectively, and can form high quality clustering results. It was shown by the experimental result that this algorithm is effect for large-scale text data clustering.