网络舆情与上证指数涨跌幅的关联性分析——基于LDA主题模型的文本挖掘

A Correlation Analysis between Online Public Opinion and Shanghai Securities Composite Price Limit: A Text Mining Based on LDA Theme Model

基于对网络信息进行LDA主题模型的文本挖掘和实证分析,考察网络舆情的变化与上证指数的涨跌幅之间的关联性。编写网络爬虫抓取"今日头条"网站中的一千三百余万条帖子,通过LDA主题模型的方法将网络新闻和帖子分为100类主题,考察各类主题在每天的网络舆情中所占的比重及其变化情况。分析结果显示,网络舆情中部分主题所占比重的变化情况,与上证综指涨跌幅之间具有关联性。这既反映着网络舆情作为"社会传感器"与金融市场中的社会后果之间的关联,也从网络信息挖掘的角度为有限理性和有限注意力背景下的行为经济学研究提供有益路向。

Based on the text mining and the empirical analysis of the LDA theme model of online information, the correlation between the change of the online public opinion and the price limit of Shanghai securities index is investigated. The web crawlers is written and captures more than one thousand and three million posts in ＂Today＇s Headlines＂ website. With the LDA theme model, the online news and posts are divided into 100 categories, and the proportion of various topics in daily online public opinion and its daily changes are examined. The result of the analysis shows that the change of the proportion of some themes in online public opinion is related with the price limit of Shanghai Securities Composite Index. This not only reflects the correlation between the online public opinion as a social sensor and the social consequences at the financial market, but also provides a useful way tor the research of behavioral economics under the background of the bounded rationality and the limited attention from the aspect of the network information mining.