基于MapReduce框架的实时大数据图像分类研究

图像数据属于大数据的一种,其蕴含着大量的知识,并且图像分类被广泛地应用于各个领域中。传统图像分类模式过于落后和单一,已经无法满足大数据时代实时计算的需求,为了解决这一问题,现利用MapReduce框架,提出一种新型、先进的实时大数据图像分类算法。首先,针对MapReduce并行化计算框架特征,利用在线极端学习机得出权值矩阵;然后采用矩阵分割的方式,取代并淘汰传统的大规模矩阵累乘操作;再对分割后的矩阵进行节点并行计算。在此基础上,将各个节点的最终计算结果进行汇总和合并,从而得到图像分类器,在保证最终计算结果真实性、准确性和完整性的基础上,对MapReduce框架不断拓展和优化,并采用实时大数据分类的方式对人脸图像进行分类。结果表明:MapReduce框架具有很高的有效性和可行性,不仅可以实现对大数据图像的精确化、科学化和规范化分类,还能保证大数据图像分类的效率和效果。

Image data is a kind of big data,which contains a lot of knowledge,and image classification is widely used in various fields.The traditional image classification mode is too backward and single to meet the needs of real-time computing in the era of big data.In order to solve this problem,a new and advanced real-time big data image classification algorithm is proposed using MapReduce framework.Firstly,according to the characteristics of MapReduce parallel computing framework,online extreme learning machine is used to get the weight matrix.Then,matrix segmentation is used to replace and eliminate the traditional large-scale matrix multiplication operation.Then,node parallel computing is performed on the segmented matrix.On this basis,the final calculation results of each node are summarized and merged to get the image classifier,and on the basis of ensuring the authenticity,accuracy and integrity of the final calculation results,the MapReduce framework is continuously expanded and optimized,and the face image is classified by real-time big data classification.The results show that MapReduce framework has high effectiveness and feasibility,which can not only achieve accurate,scientific and standardized classification of big data images,but also ensure the efficiency and effectiveness of big data image classification.