结合轻量级麦穗检测模型和离线Android软件开发的田间小麦测产

Combining lightweight wheat spikes detecting model and offline Android software development for in-field wheat yield prediction

单位面积麦穗数是重要的产量构成因素之一,通过该性状和不同品种历史数据在田间完成对小麦产量的预估,对育种栽培和农业生产具有非常重要的意义。该研究基于小麦田间栽培试验提出了一套结合轻量级深度学习技术和小麦测产算法在Android(安卓)智能手机上离线分析单位面积穗数和田间测产的技术方案。首先介绍了手机标准化俯拍小麦冠层和手机端图像预处理算法,再根据灌浆期小麦冠层图像构建了MobileNetV2-YOLOV4深度学习模型对单位面积中的麦穗进行识别,然后结合迁移学习和TensorFlow.lite转换器完成了模型轻量化,最后通过AndroidSDK和SQLite构建了不同小麦品种在手机端的产量数据库和人机交互图形界面。开发的安卓软件"YieldQuant-Mobile"(YQ-M)可离线识别手机拍摄的麦穗数量,并在田间完成产量预测和结果输出等功能。基于从中国各小麦主产区中选择的80个代表性品种(共240个1 m^(2)小区),使用YQ-M完成了这些品种的麦穗检测和小区测产研究。结果显示YQ-M的精确率、召回率、平均精确度和F1分数分别为84.43%,91.05%,91.96%和0.88。单位面积测产结果和实际产量的决定系数为0.839,均方根误差为17.641 g/m^(2)。研究表明YQ-M对麦穗识别精度高,在田间环境下测产结果和算法鲁棒性良好。此外,YQ-M还具有良好的扩展性,可为其他作物的离线智能测产提供借鉴,并为小麦研究和生产实践提供低成本、便捷可靠的田间测产方法。

The number of spikes per unit area is a key yield component for cereal crops such as wheat, which is popularly used in wheat research for crop improvement. With the fast maturity of smartphone imaging hardware and recent advances in image processing and lightweight deep learning techniques, it is possible to acquire high-resolution images using a smartphone camera, followed by the analysis of wheat spikes per unit area through pre-trained artificial intelligence algorithms. Then, by combining detected spike number with variety-based spikelet number and grain weight, it is feasible to carry out a near real-time estimation of yield potential for a given wheat variety in the field. This AI-driven approach becomes more powerful when a range of varieties are included in the training datasets, enabling an effective and valuable approach for yield-related studies in breeding, cultivation, and agricultural production. In this study, we present a novel smartphone-based software application that combines smartphone imaging, lightweight and embedded deep learning, with yield prediction algorithms and applied the software to wheat cultivation experiments. This open-source Android application is called YieldQuant-Mobile(YQ-M), which was developed to measure a key yield trait(i.e. spikes per unit area) and then estimate yield based on the trait.Through YQ-M and smartphones, we standardized the in-field imaging of wheat plots, streamlined the detection of spikes per unit area and the prediction of yield, without a prerequisite of in-field WiFi or mobile network. In this article, we introduce the YQ-M in detail, including: 1) the data acquisition designed to standardize the collection of wheat images from an overhead perspective using Android smartphones;2) the data pre-processing of the acquired image to reduce the computational time for image analysis;3) the extraction of wheat spike features through deep learning(i.e. YOLOV4) and transfer learning;4) the application of TensorFlow.lite to transform the trained model into a lightweight MobileNetV2-YOLOV4 model, so that wheat spike detection can be operated on an Android smartphone;5) finally, the establishment of a mobile phone database to incorporate historic datasets of key yield components collected from different wheat varieties into YQ-M using Android SDK and SQLite. Additionally, to ensure that our work could reach the broader research community, we developed a Graphical User Interface(GUI) for YQ-M, which contains: 1) the spike detection module that identifies the number of wheat spikes from a smartphone image;2) the yield prediction module that invokes near real-time yield prediction using detected spike numbers and related parameters such as wheat varieties, place of production, accumulated temperature, and unit area. During our research, we have tested YQ-M with 80 representative varieties(240 one-square-meter plots, three replicates) selected from the main wheat producing areas in China. The computed accuracy, recall, average accuracy, and F1-score for the learning model are 84.43%, 91.05%, 91.96%, and 0.88, respectively. The coefficient of determination between YQ-M predicted yield values and post-harvest manual yield measurement is 0.839(n=80 varieties, P<0.05;Root Mean Square Error=17.641 g/m^(2)). The results suggest that YQ-M presented here has a high accuracy in the detection of wheat spikes per unit area and can produce a consistent yield prediction for the selected wheat varieties under complex field conditions. Furthermore, YQ-M can be easily accessed and expanded to incorporate new varieties and crop species, indicating the usability and extendibility of the software application. Hence, we believe that YQ-M is likely to provide a step change in our abilities to analyze yield-related components for different wheat varieties, a low-cost, accessible, and reliable approach that can contribute to smart breeding,cultivation and, potentially, agricultural production.