共词分析与LDA模型分析在文本主题挖掘中的比较研究

A Comparative Study of Co-word Analysis and LDA Model Analysis in Mining Text Topic

【目的/意义】大数据时代文本主题挖掘在情报分析领域中的作用日趋重要,通过特征比较共词分析和LDA模型分析两种主流文本主题挖掘方法,研究两者的具体特点,为相关人员合理地运用文本主题挖掘方法处理数据提供一定的参考。【方法/过程】本文分两种情况对比研究:第一、两者挖掘不同时段同一种类文本数据的主题分布信息和主题演化信息的能力;第二、两者挖掘同一时段不同种类文本数据的提取正确主题的能力。【结果/结论】在不同时段LDA模型分析与共词分析相比挖掘主题分布信息的能力可不断提升,并且其可挖掘出更加细化的主题演化信息;在同一时段LDA模型分析对语义关系模糊逻辑结构粗糙的文本提取正确主题的效果明显优于共词分析。

[Purpose/significance]The text topic mining in the era of big data is playing an increasingly important role in the field of intelligence analysis. Some reference involving the proper text mining way of dealing with data will be provided through the comparison of two main methods of text topic mining including co-word analysis and LDA model analysis and the exploration of their specific characteristics. [Method/process] The paper carries on the comparative study in two condi- tions ： firstly, the abilities of the two analyzing methods will be compared based on the topic distribution information and evolution information of the text data in the same type in different times; secondly, the abilities of the two analyzing methods will be compared in accordance with the extraction of text data in different kinds in the same time. [Result/conclusion] comparing with the co-word analysis in different periods, the LDA model analysis can continuously improve its ability of mining topic distribution information , and can dig out more refined topic evolution information. In the same period, the LDA model analysis is more effective than the co-word analysis in extracting the correct topic from the text with fuzzy semantic relations and rough structures in logic.