微博内容自动抽取方法研究

Automatically Extracting Microblog Posts From Different Service Platforms

在采用分类机器学习算法的基础上,提出有效的特征集,实现不同微博平台上内容的自动抽取。根据对微博页面的预处理粒度不同,讨论了token和line两种粒度。在特征选取方面沿用了传统的Text-to-Tag Ratio,并结合微博内容文本的相关特性,提出了Text-to-Text Relation和Post-Social特征。实验结果表明,支持向量机与Text-to-Tag Ratio、Text-to-Text Relation和Post-Social 3个特征相结合的抽取结果最佳。

This paper aims at studying efficient algorithms to extract the microblog posts in a HTML page, based on machine learning algorithm. According to the different granularities of the web page prepossessing, we consider token and line based text blocks. Then in terms of feature selection, we propose the Text-to-Text Relation and Post-Social features that are combined with the traditional Text-to-Tag Ratio feature. Our experimental results show that SVM with the three features has the best performance in terms of precision.