基于分布式聚类的作物生长环境异常检测方法

Environmental Anomaly Detection Method during Crop Growth Based on Distributed Clustering

为了处理大量分布式存储的农业环境数据,提高农业生产效率,对高斯混合模型聚类算法进行了改进,提出了一种基于分布式聚类的农业环境数据异常检测方法.在Spark分布式计算框架下,首先对数据进行粗聚类,得到初始化模型;然后利用Spark迭代更新模型直至稳定,其中Map阶段将样本点分配到模型,Reduce阶段更新模型个数及参数;最后利用聚类结果,实现环境异常值的检测.实验结果表明该方法可行有效.

In order to process the massive agricultural environmental data stored in distributed system and improve the production efficiency, the clustering algorithm based on Gaussian Mixture Model （GMM） is modified in this paper. Based on this, an environmental anomaly detection method during crop growth is proposed. Under the Spark distributed computing framework, firstly, a pre-clustering algorithm is adopted to initialize the models. Secondly, Spark is utilized to update the models iterationally until it gets stable. In each iteration, Map phase distributes sample points to the models, Reduce phase renews the numbers of models and parameters. Finally, the detection of environmental anomaly is completed by taking advantages of the clustering result. The experimental results show that this approach is practically feasible and effective.