基于全切分获取网络流行语方法研究

Study on popular words and phrases extraction of network based on omni-segmentation

利用统计和规则相结合的算法从互联网的动态信息流中提取网络流行语。在利用全切分算法获取候选词集的基础上,依次对候选词集进行三次过滤:首先基于向量空间模型的权重过滤,运用语言模型进行过滤;然后利用垃圾串过滤规则获取网络流行词语候选词集;最后利用提出的流行词语评分模型进行筛选得到网络流行词语。实验表明,在不影响流行词语准确率的前提下,利用该方法自动获取网络流行词语的速度明显提高。

This paper aimed to extract popular words and phrases of network by specific algorithm. It filtrated the candidate words set three times based on the algorithm of omni-segmentation. The first was the weight filtration based on the vector space model , then used the model of language regulation , and the last through the filtration of rubbish cluster. Finally, it mined the popular words and phrases from the candidate set by the popular words determinant formula. The experimentation indicates that without reducing the correct rate of catchwords, the speed of extacting, the popular words and phrases of network impoves distinctly.