共词网络LDA模型的中文文本主题分析:以交通法学文献(2000-2016)为例

A CA-LDA Model for Chinese Topic Analysis:Case Study of Transportation Law Literature

【目的】通过结合传统LDA模型的概率主题抽取方法和共词网络分析发现文献词汇间的联系结构的两者优势,降低由少量文献产生的高频词汇的干扰,提高主题凝聚性。【方法】在交通法学文献摘要文本主题分析中,加入文献的关键词作为分词复合词典,提高语义识别度;提出CA-LDA模型(Latent Dirichlet Allocation Model with Co-word Analysis),在传统LDA模型的基础上加入共词网络分析,以共词网络拓扑结构参数作为权重控制词汇主题分配(采用介数中心度),优先提取同时具有高共现性(中介性)和高频率的词汇。【结果】CA-LDA模型可以得到多篇文献同时共现的高频词汇,这样产生的重点词汇表对主题分析更有意义。该算法的结果不仅仅反映词频概率,同时也能从词汇关联上发现枢纽词汇,更深入理解该领域的研究热点。【局限】CA-LDA模型主题数目K的取值采用混淆度标准交叉验证获得,如果在实际分析中K值太大,不利于文献主题的分类整理,未来研究需要对该结果进一步处理来凝聚主题。【结论】本文将该模型应用于交通法学研究领域热点主题分析,在处理大规模文献数据中取得较好效果。相关研究可以拓展应用于各种领域的大规模文献数据自动化处理中。

[Objective] This paper aims to improve the effectiveness of extracting Chinese literature topics with the help of LDA model and co-word network analysis. [Methods] First, we added keywords to the word segmentation dictionary for the abstracts, which improved the semantic recognition of topic analysis. Second, we proposed a Latent Dirichlet Allocation Model with Co-word Analysis （CA-LDA） to control the topic distribution generated by the weight of co-word network topology parameters （i.e. Betweenness Centrality）. Finally, we extracted the words with high connectivity （Betweenness Centrality） and frequency. [Results] The CA-LDA model retrieved high frequency and high connectivity words simultaneously, which were important for subject analysis. The proposed algorithm could also identify key node technical vocabularies with the help of co-word analysis. [Limitations] The K value （number of topics） was obtained by cross validation with perplexity. Thus, it was difficult to classify the document topics with larger K value. More research is needed to deal with this issue. [Conclusions] The proposed model effectively analyzes the topics of Chinese literature on transportation laws, which could also process literature data from other fields automatically.