基于互信息的微博新词发现算法

微博是一种近些年来兴起的互联网媒体,每时每刻都会产生各种新生的网络词汇。对于新词发现算法中表现出的缺点,文中提出了一种基于互信息的微博新词发现算法,将互信息合并多字词的方式应用到微博新词的发现中,并且通过实验验证了本文算法对于微博新词发现的有效性。

Micro-blog is a new kind of social network, a variety of nascent network vocabulary is produced at all times. In order to make up for these deficiencies in the previous new word detection algorithms, this paper presents a new word detection algorithm in micro-blog based on mutual information. In this algorithm, the mutual information with multiple word is applied to the micro-blog new word detection. The experiments show that this algorithm is more effective for micro-blog new word detection.