注意力与多尺度特征融合的水培芥蓝花蕾检测

Flower bud detection model for hydroponic Chinese kale based on the fusion of attention mechanism and multi-scale feature

准确辨识水培芥蓝花蕾特征是区分其成熟度,实现及时采收的关键。该研究针对自然环境下不同品种与成熟度的水培芥蓝花蕾外形与尺度差异大、花蕾颜色与茎叶相近等问题,提出一种注意力与多尺度特征融合的Faster R-CNN水培芥蓝花蕾分类检测模型。采用InceptionV3的前37层作为基础特征提取网络,在其ReductionA、InceptionA和InceptionB模块后分别嵌入SENet模块,将基础特征提取网络的第2组至第4组卷积特征图通过FPN特征金字塔网络层分别进行叠加后作为特征图输出,依据花蕾目标框尺寸统计结果在各FPN特征图上设计不同锚点尺寸。对绿宝芥蓝、香港白花芥蓝及两个品种的混合数据集测试的平均精度均值mAP最高为96.5%,最低为95.9%,表明模型能实现不同品种水培芥蓝高准确率检测。消融试验结果表明,基础特征提取网络引入SENet或FPN模块对不同成熟度花蕾的检测准确率均有提升作用,同时融合SENet模块和FPN模块对未成熟花蕾检测的平均准确率AP为92.3%,对成熟花蕾检测的AP为98.2%,对过成熟花蕾检测的AP为97.9%,不同成熟度花蕾检测的平均准确率均值mAP为96.1%,表明模型设计合理,能充分发挥各模块的优势。相比VGG16、ResNet50、ResNet101和InceptionV3网络,模型对不同成熟度花蕾检测的mAP分别提高了10.8%、8.3%、6.9%和12.7%,检测性能具有较大提升。在召回率为80%时,模型对不同成熟度水培芥蓝花蕾检测的准确率均能保持在90%以上,具有较高的鲁棒性。该研究结果可为确定水培芥蓝采收期提供依据。

An accurate detection of flower bud features can greatly contribute to classifying the maturity for the timely harvesting of the hydroponic Chinese kale.The Faster Region-based-convolutional neural network(R-CNN)can be widely expected to serve as a compelling accuracy of detection without high real-time performance.However,the shape and size of flower buds vary greatly in the different varieties of hydroponic Chinese kale.The flower buds,stems,and leaves are also similar in color features.In this study,an improved Faster R-CNN model was proposed to accurately detect the flower buds of hydroponic Chinese kale in natural environment using the fusion of attention mechanism and multi-scale feature.The first 37 layers of InceptionV3 network were first selected as the basic network of feature extraction for the rich features without overfitting.The Squeeze-and-Excitation Network(SENet)was embedded with the ReductionA,InceptionA,and InceptionB modules to enhance the weight of the channels containing valid feature information,but to reduce the interference from the irrelevant background.The extraction features from the second to the fourth convolution group were output to the Feature Pyramid Network(FPN)layer,where a multi-scale FPN layer was obtained for the Region Proposal Network(RPN)during fusion operation.Different anchor sizes were also designed for each FPN feature map,according to the target frame size of flower buds.The improved model was verified using the dataset of Lubao Chinese kale(1255 images),and Hongkong Chinese kale(1319 images),as well as the mixed dataset of two varieties.The precision rate,recall rate,average precision,and mean average precision were also selected to evaluate the performance of the improved model in the experiments.The results showed that:1)The average accuracy of the model increased,while,the comprehensive loss declined gradually,with an increase of the iteration.The peak value of average precision appeared stable after 10 iterations,indicating the strong fitting and generalization ability of the model.The mean average precisions of the model were 96.5%,95.9%,and 96.1%for the Lubao Chinese kale,the Hongkong Chinese kale,and the mixed dataset,respectively,indicating a high detection accuracy of the model for different varieties.2)An ablation experiment was carried out on the mixed dataset.The mean average precision of the basic feature extraction network was 90.7%.The mean average precisions were 94.3%and 94.8%for the basic feature extraction networks combining with the SENet and FPN module,respectively.3)In the immature,mature,and over-mature flower buds,the average precisions were 92.3%,98.2%,and 97.9%,respectively,where the mean average precision was 96.1%.As such,either the SENet or FPN module contributed greatly to improving the detection accuracy for the hydroponic Chinese kale with different maturity.4)The mean average precisions of the improved model were improved by 10.8,8.3,6.9,and 12.7 percentage points,respectively,compared with the VGG16,ResNet50,ResNet101,and InceptionV3 networks.Furthermore,the mean average precision of hydroponic Chinese kale with different maturity was all above 90%,when the recall rate was 80%,indicating the high robustness of the model.The finding can provide a strong reference to determine the harvest period of hydroponic Chinese kale.