基于特征词典构建和BIRCH算法的中文百科文本聚类研究

Research on the text clustering method of Chinese encyclopedia based on feature dictionary construction and BIRCH algorithm

针对传统文本聚类存在数据维度过高,无法深层次理解语义等问题,提出一种基于特征词典构建和BIRCH算法的文本聚类方法。该方法通过LDA主题模型和语义特征构建特征词典,利用BIRCH算法进行文本聚类,并对维基百科、百度百科和互动百科中的景点、动物、人物和国家四个主题的网页文档进行实验分析。实验结果表明,特征词典结合了主题关键词和语义相似度,其准确率、召回率和F特征值较传统方法有所提高,该方法可以广泛应用于文本挖掘、知识图谱和自然语言处理等领域。

Aiming at the problem that traditional text clustering has too high data dimension and deep understanding of semantics,a text clustering method based on feature dictionary construction and BIRCH algorithm is proposed. This method builds a feature dictionary through LDA topic model and semantic features, uses BIRCH algorithm to perform text clustering. an experimental analysis on Wikipedia, Baidu Encyclopedia and Interactive Encyclopedia WebPages is conducted on attractions, animals, characters and countries four topics, the results show that combined with the topic keywords and semantic similarity, the accuracy, recall and F eigenvalues of the feature dictionary are improved compared with traditional methods. This method can be widely used in text mining, knowledge mapping and natural language processing.