云计算平台上基于选择性集成的增量学习研究

Selective Ensemble-based Incremental Learning on Cloud Computing Platform

增量学习是处理数据流的有效方式。文中针对已有增量分类算法只是作用于小规模数据集或者在集中式环境下进行的不足,提出了一种基于Hadoop云计算平台的增量分类模型,以解决大规模数据集的增量分类。该增量分类模型主要基于选择性集成学习思想,设计相应Map函数对不同时刻的增量样本块进行学习,以及设计Re-duce函数对不同时刻的分类器进行选择性集成以实现云计算平台上的增量学习。仿真实验表明该方法具有更好的性能,且能较好地解决数据流中的概念漂移问题。

Incremental learning is an effective way for data stream processing.To alleviate some issues about the current incremental learning algorithms,such as only for small-scale data sets or in a centralized environment,an incremental classification algorithm on Hadoop cloud computing platform is proposed to deal with large-scale data sets.Based on the idea of selective ensemble learning,Map function is designed to obtain base classifiers on incremental data blocks at different times,meanwhile,Reduce function is constructed to integrate different classifiers based on selective ensemble learning.The simulation results indicate that the proposed method can achieve higher performance than other ones and deal with concept drift problem in data stream.