基于LDA模型的文本时间窗口划分研究

Research on Text Time Window Partition Based on LDA Model

[目的/意义]鉴于静态主题模型难以满足用户动态分析需求,为解决现有动态主题模型存在计算成本高或者受主观因素影响深等问题,从时间窗口相似度出发,以LDA模型为基础,提出一种文本时间窗口划分算法。[方法/过程]构建集时间窗口间差异性和时间窗口内一致性于一体的时间窗口相似度指标,基于该指标构建文本时间窗口划分算法,并以创新研究领域为例开展实证研究。[结果/结论]综合每一个时间窗口内最优主题个数下主题之间的平均JS散度和相邻时间窗口间不同主题之间的平均JS散度两个指标结果来看,使用本文提出的算法得出的划分结果明显优于多种固定时间窗口长度划分的结果,验证了本研究提出的文本时间窗口划分算法的有效性。该算法在一定程度上解决了现有动态主题模型计算成本高、主观性强等缺点,增加了文本时间窗口划分结果的客观性与准确性,可以为主题演化等相关研究提供技术支持。

[Objective/Significance]Considering that static topic models are difficult to meet users'dynamic analysis needs,in order to solve the problems of high computational costs or deep influence from subjective factors in existing dynamic topic models,this study proposes a text time window partitioning algorithm based on the LDA model,starting from time window similarity.[Method/Process]This study constructs a time window similarity index that integrates differences between time windows and consistency within time windows.This study constructs a time window partitioning algorithm based on this indicator and conducts empirical research using the innovation research field as an example.[Results/Conclusions]By analyzing the average JS divergence between topics under the optimal number of topics within each time window,as well as the average JS divergence between different topics between adjacent time windows,the partitioning results obtained by the algorithm proposed in this study are significantly better than those obtained by multiple fixed time window length partitioning methods,verifying the effectiveness of using the improved LDA model proposed in this study for text time window partitioning.The algorithm proposed in this study to some extent solves the shortcomings of existing dynamic topic models such as high computational costs and strong subjectivity,increases the objectivity and accuracy of text time window partitioning results,and can provide technical support for related research such as theme evolution.