云环境中大数据挖掘的有效花费研究

Effective cost of big data mining in cloud environment

为平衡云计算资源的租用量与云环境中数据挖掘的计算结果准确率,得到最优的性价比,以监督式学习的卷积神经网络(CNN)为例,探究了CNN迭代次数与准确率的演化规律。选择经典图像数据集MNIST和文本数据集IMDB作为代表展开实验,发现在不同类型的数据集中,当CNN迭代接近最优解时,每提高很小的准确率,耗费的机时陡增,称之为长尾现象。验证在真实云环境中,当大数据挖掘的长尾现象发生且满足企业准确率需求的情况下,选择提前结束取代最高精度时结束,均可以节省大量云资源成本。研究结果对于合理运用云计算资源,降低云服务租用成本,具有实用价值与现实意义。

In order to balance the renting quantity of cloud computing resources and the accuracy of data mining in cloud,the optimum cost performance ratio is obtained.Taking the convolution neural network(CNN)as an example,the evolution patterns of the number of iterations and accuracy of CNN was explored.A lot of experiments were performed upon the image dataset MNIST and the text dataset IMDB.The results show that in different types of data sets,the machine time consumed increases sharply with a small increase in accuracy when the optimal solution is approached,which is called the long tail phenomena.Correspondingly,in the real cloud environment,when the long tail phenomenon of big data mining occurs and the accuracy is satisfied,terminating the performance of CNN in cloud in advance rather than at the convergence time can save a lot of cloud resource costs.The results have practical value and practical significance for the rational use of cloud computing resources and the reduction of cloud rental cost.