云计算平台上的增量分类研究

Incremental classification method based on cloud computing

针对已有增量分类算法只是作用于小规模数据集或者在集中式环境下进行的缺点,提出一种基于Hadoop云计算平台的增量分类模型,以解决大规模数据集的增量分类。为了使云计算平台可以自动地对增量的训练样本进行处理,基于模块化集成学习思想,设计相应Map函数对不同时刻的增量样本块进行训练,Reduce函数对不同时刻训练得到的分类器进行集成,以实现云计算平台上的增量学习。仿真实验证明了该方法的正确性和可行性。

To alleviate some issues about the current incremental learning algorhtms, such as only for small-scale data sets and work in a centralized environment, an incremental classification algorithm based on Hadoop cloud computing platform is proposed to deal with large data sets. In order to automatically process the incremental training samples on cloud computing platform, we design Map function to train incremental data blocks at different times, and design reduce function to integrate different classifiers based on modular ensemble learning. The simulation results indicate that the proposed method is correct and feasible.