基于LDA和语步标注的主题识别与分析方法研究

Research on Topic Recognition and Analysis Based on LDA and Move Tagging

【目的】从主题表征词抽取和主题句功能分类两个维度,设计基于潜在狄利克雷分布模型(Latent Dirichlet Allocation,LDA)和语步标注的主题分析方法,并探究方法的有效性与实用性。【方法】采用LDA模型进行主题识别,利用Sentence Transformer模型对主题词短语进行提取,同时构建句子功能分类模型进行语步标注,识别文本句子功能类型,从句子功能维度对主题内容进行细粒度分析。【结果】以农业资源与环境领域论文数据为例进行实证研究,结果表明,相比传统LDA模型,经过提取主题词短语后,识别出的主题表征词可读性和解释性更强,进一步结合语步标注后,主题句子内容分析更为深入。【局限】主题短语表征词扩展内容存在含义相同问题,有待进一步改进表征词的多样性,以整合相同含义的主题短语表征词。【结论】本研究所提方法在主题表征词抽取、主题内容分析方面具有较好的效果,可以提高文本主题挖掘的效率与深度。

[Objective]From the two dimensions of topic representation word extraction and topic sentence function classification,this paper demonstrates a new topic analysis method based on Latent Dirichlet Allocation(LDA)model and move tagging,and explores the effectiveness and practicality of the method.[Methods]LDA model is used to identify the topic,and the Sentence Transformer model is used to extract the subject phrases.Meanwhile,a sentence function classification model is constructed to annotate the steps,identify the functional types of text sentences,and analyze the topic content from the perspective of sentence function.[Results]Based on the data of papers in the field of agricultural resources and environment,the empirical study shows that,compared with the traditional LDA model,the identified subject characterizing words are more readable and explanatory,and further combined with the step annotation,the content analysis of the subject sentence is more in-depth.[Limitations]There is a problem that the extended content of the subject phrase token words are of the same meaning.It is necessary to further improve by integrating the subject phrase token words with the same meaning.[Conclusions]The proposed method in this study achieves a good effect on topic representation word extraction and topic content analysis,which can improve the efficiency and depth of text topic mining analysis.