基于向量表示和标签传播的半监督短文本数据流分类算法

Semi-supervised Short Text Stream Classification Based on Vector Representation and Label Propagation

社交网络平台产生海量的短文本数据流,具有快速、海量、概念漂移、文本长度短小、类标签大量缺失等特点.为此,文中提出基于向量表示和标签传播的半监督短文本数据流分类算法,可对仅含少量有标记数据的数据集进行有效分类.同时,为了适应概念漂移,提出基于聚类簇的概念漂移检测算法.在实际短文本数据流上的实验表明,相比半监督分类算法和半监督数据流分类算法,文中算法不仅提高分类精度和宏平均,还能快速适应数据流中的概念漂移.

The huge volume of short text streams produced by social Network is fast, high-volume and it contains concept drift, short length of texts and massive unlabeled data. Therefore, a semisupervised short text stream classification algorithm based on vector representation and label propagation is proposed in this paper to classify short text stream with a few labeled data. Besides, to adapt to the concept drift, a concept drift detection algorithm based on clusters is proposed. Experimental results on real short text streams show that the proposed algorithm improves the classification accuracy and the macro average compared with classical semi-supervised classification algorithms and semi-supervised data stream classification algorithms, and it adapts to the concept drift quickly in data stream.