基于三维点云的番茄植株茎叶分割与表型特征提取

Stem-leaf segmentation and phenotypic trait extraction of tomatoes using three-dimensional point cloud

针对当前温室番茄表型参数难以自动获取的问题,研究提出通过对三维点云进行配准、骨架提取以及分割从而自动获取苗期番茄植株株高、茎粗、叶倾角和叶面积参数的方法。首先通过机器人搭载机械臂在温室中自动获取多视角番茄点云,并通过配准得到完整植株点云;对番茄点云利用拉普拉斯收缩的骨架提取算法获取植株骨架,对骨架进行修正后分解为茎秆和叶片子骨架,实现茎秆叶柄分割;再通过基于区域生长的MeanShift聚类方法对叶片和叶柄进行分割;最后通过番茄点云获取株高、茎粗参数,通过骨架测量叶倾角,对叶片点云进行曲面拟合提取叶面积参数。试验结果表明,茎叶分割与叶片分割的精确率、召回率、F1分数和平均总体准确率分别为0.84、0.91、0.87、0.92和0.92、0.91、0.91、0.93。株高、茎粗、叶倾角和叶面积参数的提取值与人工测量值的决定系数分别为0.97、0.53、0.90和0.87,均方根误差分别为1.40 cm、1.52 mm、5.14°和37.56 cm^(2)。结果表明该研究方法与人工测量值具有较强的相关性,可以为温室番茄的高通量自动化表型测量提供技术支持。

Crop phenotyping is the quantitative assessment of complex plant traits. The crop phenotypic parameters can greatly contribute to crop breeding, product development, and quality evaluation in modern agriculture. However, the traditional and manual measurement can be time-consuming and labor-intensive with low data accuracy. In this study, an intelligent measurement system was developed to automatically detect the phenotypic parameters of tomatoes in a greenhouse with high throughput using a mobile robot platform. A Kinect V2 depth camera was first installed at the end of the robot arm, and then a multi-perspective point cloud was collected via the movement of the robot arm. An automatic and accurate inspection was conducted in the greenhouse for the automatic collection of the tomato point cloud. The tomato point cloud was filtered and registered to obtain the complete plant point cloud. In the registration process, the end joint pose recorded by the robot arm was used to calculate the rotation and translation matrix of point cloud registration using the change table relationship of the pose construction. After that, the tomato point cloud skeleton was extracted to realize the stem and petiole segmentation using Laplace contraction. An experiment was also carried out in the greenhouse of the National Experiment Station for Precision Agriculture in Beijing in March 2022. Multi-view point cloud collection and phenotypic parameter extraction were performed on 10 tomato seedlings at 7, 15, and 25 days after transplanting to the greenhouse. The accuracy, recall rate, F1 fraction, and average overall accuracy of stem and leaf segmentation were 0.84, 0.91, 0.87, and 0.92, respectively. The MeanShift clustering and regional growth were combined to divide the leaf and leaf petiole for the better segmentation of tomato leaves. Among them, the regional growth algorithm presented an excellent performance to segment the leaf and leaf petiole, whereas, the MeanShift algorithm was to segment the different leaves. The accuracy rate, recall rate, F1 score, and average overall accuracy of leaf segmentation were 0.92, 091, 0.91, and 0.93, respectively, indicating better performance than the regional growth and MeanShift algorithms alone. The parameters of plant height and stem diameter were measured by the plant point cloud after registration, and the parameters of leaf inclination were measured by the skeleton. Finally, a greedy projection triangulation algorithm was selected to convert the leaf point cloud into the triangular mesh for the parameters of leaf areas. The determination coefficients of plant height, stem diameter, leaf inclination, and leaf area were 0.97, 0.53, 0.90, and 0.87,respectively, and the root mean square errors were 1.40 cm, 1.52 mm, 5.14°, and 37.56 cm^(2), respectively, compared with the measured values. Specifically, the larger error of stem thickness parameter can be attributed to the distance measurement error of the depth sensor for the very thin stem of the tomato plant at the seedling stage during automatic measurement. Subsequent research can be required to accurately locate the tomato stem. A vernier caliper was installed at the end of the robotic arm to perform the contact measurement, in order to effectively improve the accuracy of the stem diameter parameter. The finding can provide technical support to the high-throughput, accurate and automated measurement for the phenotypic parameters of greenhouse tomatoes.