基于轻量化卷积神经网络的苹果表皮损伤分级研究

Research on apple epidermal damage grading based on lightweight convolutional neural network

【目的】苹果在销售过程中,其表皮的损伤情况会直接影响果实的经济价值。运用相机采集苹果表皮的损伤图像,对获取到的图像进行分类和数据预处理,基于迁移学习的方法对苹果表皮损伤进行直接分级研究,为提高苹果表皮损伤分级效率进而更好地指导苹果采收后的分类售卖提供理论依据。【方法】首先对采集到的富士和丹霞两个苹果品种图像进行对比度调整、旋转、翻转、添加噪声等11种批量操作,将数据集扩充到9360张,同时对扩充后的样本集统一调整为224×224像素。针对预处理好的数据集,选取5种20 MB以下的轻量化卷积神经网络在相同超参数设置条件下进行初始化训练、引入迁移学习训练以及在迁移学习基础上增加冻结网络层权重3种方法进行训练对比。【结果】5种网络初始化训练后的测试精度仅为56.32%~71.98%;基于迁移学习的MobileNet-v2模型最终训练精度达99.04%,在轻量级卷积神经网络中,比表现性能最差的EfficientNet-b0模型最终训练精度高18.79%;在基于迁移学习的MobileNet-v2模型基础上冻结不同模块参数,得出模型选择冻结至第1个卷积模块到Bottleneck 3-1模块时均可在缩短模型训练时间的基础上提高模型验证精度,其中在冻结到Bottleneck 3-1模块时比基于迁移学习的MobileNet-v2模型训练时间缩短了29.32%,同时验证精度提高了0.93%,测试精度提升了1.12个百分点达91.58%,检测单张图片所用平均时间为0.14 s,网络大小为8.15 MB,可以满足快速识别需求。【结论】基于迁移学习加冻结训练的MobileNet-v2模型具有较好的鲁棒性和分级性能,可为移动终端和嵌入式设备在苹果损伤直接分级方面提供技术参考。

【Objective】In the process of apple selling,the damage of its epidermis will directly affect the economic value of the fruit.The presence and severity of apple surface damage directly affect the sales link,and customers often care about the epidermis damage when choosing apples.At present,most studies focus on apple size,color and appearance classification,and the use of high-end instruments to detect the damage inside the apple,while the study on the direct classification of surface damage is rare.The camera was used to collect apple epidermis damage images,classify and preprocess the acquired images,and conduct a direct classification on apple epidermis damage based on transfer learning method,which can provide a theoretical basis for improving the classification efficiency of apple epidermis damage and guiding the classification and apple sale after harvesting.【Methods】Firstly,the camera was used to collect the top,side and bottom images of Fuji and Danxia apples to form the firststage data set.Then,11 batch of operations,such as contrast adjustment,rotation,flip and noise addition,were carried out to expand the data set to 9360 pieces to form the second-stage data set.At the same time,the expanded sample set was uniformly adjusted to 224×224 pixels to form the final data set.According to the ratio of 7∶3∶3,the preprocessed data set was divided into training set,verification set and test set.Five lightweight convolutional neural networks less than 20 MB,MobileNet-v2,SqueezeNet,ShuffleNet,NASNet-Mobile and EfficientNet-b0,were selected for initial training,introduction of migration learning training and migration learning under the same super-parameter Settings On a Bottleneck basis,and three methods were added for detailed freezing network layer weights(the MobileNet-v2 network structure is specifically divided into 21 modules for freezing training,which contain 3 convolutional modules,1 average pooling module,and 17 Bottleneck modules).【Results】The test accuracy of the five kinds of networks after initial training was only 56.32%-71.98%.The final training accuracy of MobileNet-v2 model based on transfer learning was 99.04%,18.79%higher than that of the worst EfficientNet-b0 model among lightweight convolutional neural networks.After freezing different module parameters on the basis of the MobileNet-v2 model were based on transfer learning,it was concluded that models Bottleneck 3-1,when they select to freeze to the first convolutional module,can shorten model training time and improve model validation accuracy.When Bottleneck 3-1 module was frozen,the training time for Bottleneck 3-1 was shortened by 29.32%compared to MobileNet-v2 model based on transfer learning,the verification accuracy increased by 0.93%,and the test accuracy increased by 1.12 percentage points to 91.58%.The average time for detecting a single image was 0.14 s.The network size was 8.15 MB,which can meet the requirements of fast identification.The final training loss value of the MobileNet-v2 model based on transfer learning was less than 0.04,which was 0.5 lower than that of the worst performing EfficientNet-b0 model in lightweight convolutional neural networks.The test results showed the recall rate and precision rate of MobileNet-v2 confusion matrix diagram based on transfer learning and five kinds of lightweight convolutional neural networks were based on transfer learning in the test set.Among them,the MobileNet-v2 model based on transfer learning had the best performance,and the recall rate of 6 types of data in the test set ranged from 89.40%to 100%.The precision ranged from 53.52%to 99.78%.The Grad-CAM visualization comparison of the trained network showed that the SqueezeNet model based on transfer learning had the worst visualization effect and the lowest recognition accuracy.The visualization effect of NASNet-Mobile model based on transfer learning was poor.It can only display a large range of concern areas,and the recognition degree of some pictures was not high.The visualization effect of the MobileNet-v2 model based on transfer learning was obviously better than the previous two models,but the key areas identified by the model were different from the reality.A MobileNet-v2 model based on transfer learning tended to have the best visualization effect on a network that was Bottleneck 3-1 when it was frozen to a Bottleneck 3-1 module,and the key areas identified by the model had the highest compatibility with the actual situation.【Conclusion】In this study,five kinds of lightweight models with Bottleneck 3-1 were selected for initialization training and transfer learning training,and it was concluded that MobileNet-v2 model with transfer learning had the best effect.Then,the freezing strategy was used for hierarchical training.The verification accuracy reached 92.23%when Bottleneck 3-1 was frozen.The test accuracy was 91.58%,the average recognition time was 0.14 s,and the network size was 8.15 MB,which can provide technical reference for mobile terminals and embedded devices in the direct classification of apple fruit damage.