Hadoop框架下的多标签传播算法

A Label Propagation Algorithm for Multi-Label Classification Using Hadoop Technology

标签传播算法的主要思想是利用已标注数据的标签信息预测未标注数据的标签信息。然而,传统传播算法没有区别对待未标注数据与已标注数据相互之间的转移信息,导致算法的收敛速度较慢,影响了算法的性能。针对传统算法的不足,提出了差异权重标签传播算法,算法按标注信息的重要性赋予不同的权重。在解决了大规模特征矩阵相乘问题之后,将提出的差异权重标签传播算法应用到Hadoop框架下,采用分布式计算,实现了能够处理大规模数据的多标签分类算法(HSML),并将提出的HSML算法与现有主流多标签分类算法进行了性能比较。实验结果表明,HSML算法在多标签分类的各项性能评测指标和执行速度上都是有效的。

A method of label propagation using Hadoop technology,named HSML,is proposed,to cope with the challenge of exponential-sized output space learning from multi-label data.Label propagation algorithms are graph-based semi-supervised learning methods,and use the label information of labeled data to predict the label information of unlabeled data.Traditional label propagation algorithms do not consider the posterior probability and distinguish information between labeled data and unlabeled data during the label propagation process,hence,the performance of traditional label propagation algorithms is affected. Therefore, a label propagation algorithm with different weights is proposed.After the multiplication problem of large-scale feature matrices is solved,the proposed algorithm is applied to the framework of Hadoop to deal with the problem of multi-label classification learning from big data.Experimental results and comparisons with some well-established multi-label learning algorithms,show that the performance of HSML is superior,and that the bigger test set is the faster HSML runs.