时代精神:基于1946年到2015年报刊语料和隐含主题模型的历史热词提取

Zeitgeist:LDA-Based Historical Buzzwords Extraction from the Newspaper Corpus of 1946-2015

语言监测是语言规划学的基础任务。一定时间范围内的热词或关键词则是语言监测工作的重要研究对象。目前向前挖掘历史热词的工作较少。本文采用计算语言学中广泛使用的隐含主题建模方法,对第二次世界大战后70年的大陆中文报刊历时语料库进行了主题词簇的抽取,获得了相应时间段的时代热词与年度词。相比于依照频率排序的方法,使用隐含主题模型的方法获得的热词与历史直观较为吻合。所提取出的2006年以后的热词与'汉语盘点'活动中的专家与投票相结合所获得的评选结果有较好可比性。这都验证了隐含主题模型加少量人工修正的方法在历时热词提取方面的性能。

Language monitoring is a basic task in language planning. The buzzwords(or popular words) within a certain time range are important objects of language monitoring, while the research focusing on extracting historical buzzwords is rare, to our knowledge. This article employs Latent Dirichlet Allocation(LDA) in text modeling of the diachronic text from newspapers dating to the end of WWII. Compared to the methods based on word frequency, LDA offers better buzzwords extraction that matches the historical social psychology and the annual buzzwords selected by 'expert-crowdsourcing' since 2006. LDA plus manual amendment is proved to be a practical pipeline in historical buzzwords extraction.