基于卷积神经网络提取多尺度分层特征识别玉米杂草

Recognition of maize and weed based on multi-scale hierarchical features extracted by convolutional neural network

为提高作物与杂草识别的准确率、稳定性和实时性,该文以幼苗期玉米及杂草为研究对象,提出了基于卷积神经网络提取多尺度分层特征的玉米杂草识别方法。首先建立卷积神经网络模型,以从图像的高斯金字塔中提取多尺度分层特征作为识别依据,再与多层感知器相连接实现图像中各像素的识别;为了避免目标交叠所带来的问题,对图像进行超像素分割,通过计算每个超像素内部的平均像素类别分布确定该超像素块的类别,再将相同类别的相邻超像素合并,最终实现图像中各目标的识别。试验结果表明:该方法的平均目标识别准确率达98.92%,标准差为0.55%,识别单幅图像的平均耗时为1.68 s,采用GPU硬件加速后识别单幅图像的平均耗时缩短为0.72 s。该方法实现了精确、稳定和高效的玉米与杂草识别,研究可为精确除草的发展提供参考。

Effective recognition method of crop and weed is the basis for promoting the development of intelligent mechanization weeding pattern. Summarizing the previous research, we found that the accuracy and stability of the recognition model could be improved by natural and diversified feature presentation, but there are still 2 main problems. On the one hand, feature presentation of the natural property of target was difficult to be obtained by the hand-engineered feature extractor. The spatial consistency of the obtained features was bad, and the real-time performance of recognition system was reduced for the complex feature extraction algorithm. On the other hand, the effect of image preprocessing has important influence on recognition results, especially the overlapping segmentation of crop and weed. For overlapped objects, it is usually difficult to segment them without affecting their respective feature presentations, resulting in low recognition accuracy and stability. In order to solve the main problems in the current research, we explored the way to improve the recognition accuracy, stability and real-time performance, and a recognition method of crop and weed based on multiscale hierarchical feature learning combined with superpixels segmentation was proposed. The main research contents of this paper were as follows: 1) Excellent internal features of image are hierarchical. In this research, the multiscale hierarchical feature is a scene level feature with invariance and consistency in scale space. Multiscale convolutional neural network was built to extract multiscale hierarchical feature. Multiscale convolutional neural network contains multiple copies of a single CNN(convolutional neural network) that are applied to multi-scale Gaussian pyramid of the input image. The CNN model as feature extractor in this paper includes 3 stages. In the first 2 stages, it contains a bank of filters(convolution kernels) to produce dense feature maps, a point to point nonlinear mapping activation function, and a spatial pooling module for sub-sampling of each feature map, While the last stage only contains a bank of filters. Each filter is applied to the input feature maps through 2-dimensional convolution operation, in which local feature presentations are detected at all pixel locations on the input image. For each pixel, the CNN model is used to collectively encode the internal information in a large sense region around the location of given pixel. The CNN is fed with raw pixels and trained with back propagation method. With complete training, this CNN model can automatically extract hierarchical feature representations from the input image, thereby decreasing the need for hand-engineered features extracting. A series of feature maps for multiscale regions centered on each pixel of the image are produced by the multiscale convolutional neural network; these representations contain shape, texture and sense information. Therefore, the multiscale hierarchical feature is learned to allow the recognition of the class of all pixels in the image. The average pixel recognition rate is 93.41%. 2) In this research, multiscale hierarchical feature is used to produce the class distribution for every pixel through a 2-layer MLP(multi-layer perceptron). But recognizing the class of pixel independently from its surrounding regions could produce some interference at the boundary of targets. Accurate boundary segmentation for each target in the image is not provided. In this paper, the superpixel method is used for generating an over segmentation of the image, where each segmentation component(superpixel) is an irregular pixel block composed of similar pixels in texture, color and brightness characteristics. Object in the image is composed of these superpixels, and has the exact original boundary. The superpixels segmentation is computed following the simple linear iterative clustering method described in this paper. 3) We proposed the recognition strategy of multiscale hierarchical feature learning combined with the superpixels segmentation. Firstly, an over segmentation of the original image was produced through the superpixels segmentation. At the same time, each pixel location of the image was classified densely based on the multiscale hierarchical features. These predictions of pixels in each superpixel were aggregated to produce the class prediction of superpixel, through computing the average class distribution within the superpixel. Adjacent superpixels with the same class were merged to obtain the final target class prediction and image segmentation. The accurate image segmentation was achieved while recognizing the target in the image by this recognition method, which effectively avoided the problems caused by targets overlapping, and the recognition results were more stable and accurate. Maize seedlings at 2-5 leaves stage and weed during the same stage were used as research object, and the recognition method of multiscale hierarchical feature learning combined with superpixels segmentation was tested. The results showed that the average target recognition rate with this method was 98.92%, and the standard deviation was 0.55%. The average target recognition rate with the method in previous research was 98.36%, and the standard deviation was 1.05%. So, the accuracy and stability of recognition results by this method in this paper were improved with different degrees. In aspect of real-time performance, the average time to recognize a single image was 1.68 s with this method, and compared with the method proposed in previous research, it was decreased by 1.58 s. The real-time performance of this method can be further improved by GPU(graphics processing unit) hardware acceleration, and the average time to recognize a single image was only 0.72 s. Therefore, the recognition method based on multiscale hierarchical feature learning combined with superpixels segmentation can effectively achieve accurate, stable and efficient recognition of maize and weed. The research results provide reference for the development of precision weeding.