基于阵列式ESP32-CAM的番茄根系表型原位测量方法

In-situ measuring tomato root phenotype using array ESP32-CAM

为原位采集番茄根系图像,解决番茄根系表型原位测量问题,该研究提出一种基于阵列式ESP32-CAM的番茄根系表型原位测量方法。通过4×4阵列式ESP32-CAM结合OV2640镜头模组实现土壤中根系图像原位自动化无线采集,并采用张正友标定法实现相机标定和畸变校正,利用尺度不变特征转换和最邻近分类的特征检测匹配算法实现图像配准,基于离线标定方法获取相机间变换矩阵实现根系图像拼接;通过引入多头自注意力机制改进U型卷积神经网络(Uarchitecture convolutional networks,U-Net)模型对根系图像进行语义分割,采用形态学处理和骨架提取测量根系长度、面积、平均直径、根深和根宽。研究结果表明:相机阵列图像的拼接迭代均方根误差小于1.11 mm,全局拼接图像的拼接融合质量评分大于0.85;改进后的U-Net模型应用于番茄根系分割的精度、召回率、交并比和F1值分别为86.06%、78.98%、71.41%和82.37%,相比于原始U-Net模型分别提高了18.97、13.21、21.67和16.30个百分点;与人工测量值相比,根系的面积、长度、平均直径、根深和根宽的平均绝对百分比误差分别为7.78%、5.66%、8.48%、2.40%和2.23%,决定系数R2分别为0.91、0.93、0.84、0.98和0.99。该方法适用于番茄根系表型原位测量,并可推广至其他植物或果树根系表型原位测量。

The root is one of the most crucial parts of the plant to affect the overall healthy plant.The leaf can be the corresponding indicator at various stages of growth,even in the ultimate crop.However,some significant challenges still remain in the in-situ detection of soil roots.In this study,an in-situ measurement approach was presented for the tomato root phenotypes using the array ESP32-camera(CAM).Tomato root images were also captured for the in-situ measurement of root phenotypic parameters.A 4×4 array ESP32-CAM combined with 4×4 OV2640 lens module was used for the in-situ automatic wireless acquisition of soil root images.The camera was calibrated and corrected for the aberrations using the Zhang Zhengyou calibration.The image alignment was achieved using scale-invariant feature transform and K-nearest neighbor feature detection matching.The image stitching was obtained for the inter-camera transformation matrix using offline calibration.The semantic segmentation of root images was improved to introduce the efficient multi-head self-attention mechanism.The U-Net model was improved to mix the dice loss and cross-entropy loss.In-situ image acquisition experiments of tomato root systems were also conducted to obtain the images of periodic root changes.The root system was measured manually,i.e.,the root length was measured in sections by a soft ruler,the root diameter was obtained by averaging the points,the root area was approximated by the product of length and average diameter,and the root depth and width were obtained by measuring the vertical longitudinal depth and horizontal longitudinal width.Morphological processing and skeleton extraction were used to measure the root length,root area,and root mean diameter using pixel point scanning.By contrast,the root phenotypic parameters(such as the root depth and root width)were measured using the root convexity package.The results showed that the root mean square error(RMSE)of standard shape image stitching iterations of 4×4 camera array was less than 1.11 mm,and the subjective quality scores of the structural similarity index and difference of edge map of global stitched images were above 0.85,the peak signalto-noise ratio(PSNR)was greater than 32 dB.The improved U-Net model shared the greater improvement in the search for root system completeness and accuracy.Specifically,the precision,recall,intersection over the union,and F1 value of the improved U-Net model on the tomato root segmentation were 86.06%,78.98%,71.41%,and 82.37%,respectively,which were 18.97 percentage points,13.21 percentage points,21.67 percentage points,and 16.30 percentage points higher than those of the original U-Net model training,respectively.Furthermore,the mean absolute percentage errors of calculated area,length,mean diameter,root depth,and root width of root systems were 7.78%,5.66%,8.48%,2.40%,and 2.23%,respectively,and the coefficients of determination were 0.91,0.93,0.84,0.98,and 0.99,respectively,compared with the manual measurements.Insitu measurement of the root phenotype of tomato plants can also be extended to the other plants or fruit trees.