零样本深度学习驱动的杨树叶片表型检测方法研究

Poplar leaf phenotyping methods using zero-shot deep learning

叶片表型检测是感知杨树生长状态的重要手段之一,叶片颜色、姿态、纹理等形态结构表型信息可揭示植株所受胁迫的程度。其中,单个叶片分割是计算、统计其表型参数的基础。当前流行的AI算法已可满足叶片分割任务的性能需求,然而常规深度学习模型训练需要大量人工标签,制约了其发展和应用。本研究提出一种融合零样本学习和迁移学习的杨树叶片实例分割方法:运用视觉大模型GroundingDINO检索杨树苗图像中的叶片,获取对应的边界框;使用Segment Anything 2模型(segment anything model v2,SAM2)分割图像中全部对象,得到对应的掩膜(mask);将GroundingDINO模型生成的边界框作为提示,辅助SAM2过滤出叶片类别的掩膜;利用迁移学习策略,将AI生成的叶片掩膜作为标签信息,训练轻量化的YOLOv8-Segment模型。此外,构建独立测试集用于评估模型分割精度,选择交并比阈值为50%的平均精度(average precision using 50%intersection over union threshold,AP_(50))和平均交并比(mean intersection over union,mIoU)作为性能指标。结果表明,基于“Leaf”这一检索词,GroundingDINO与SAM2的组合(权重约810 MB)可实现高性能的杨树叶片分割,AP_(50)为0.936,mIoU为0.778。通过过滤异常尺寸的提示边界框,AP_(50)提升至0.942。迁移学习得到的YOLOv8-Segment模型权重仅6.5 MB,AP_(50)为0.888,大幅精简模型的同时保障了精度。本研究涉及的叶片分割模型构建过程均无须人工标注,实现了高效率、低成本的杨树叶片实例分割,可为杨树叶片计数和叶面积计算等后续表型分析应用提供技术支持。

Leaf phenotyping is one of the important approaches to perceive the growth status of poplar trees.Morphological structural phenotype information such as leaf color,posture,and texture can provide insights into the stress levels that trees are experiencing.Single-leaf segmentation is essential for calculating and statistically analyzing these phenotypic parameters.At the current stage,various artificial intelligence-based algorithms have been widely applied for leaf segmentation tasks,which can meet the performance requirements.However,conventional deep learning model training requires a large number of manual annotations,restricting its development and application.This article proposed a poplar leaf instance segmentation method that integrated zero-shot deep learning and transfer learning.Poplar seedlings of different varieties were considered and used for experiments.Different irrigation frequencies were applied to obtain trees of different drought levels.Images of these mentioned plant samples were captured for analysis.Leaf objects were retrieved from poplar seedling images using the large vision-language model GroundingDINO to obtain the corresponding bounding boxes.Segment anything model v2(SAM2) was adopted to segment all objects in the input images and obtain corresponding masks.Next,the bounding boxes generated by the GroundingDINO model were used as prompts to assist SAM2 in filtering out masks for “Leaf” category.Then,using transfer learning strategy,the AI generated leaf masks were used as ground-truth to train a lightweight YOLOv8(you look only once version 8) segment model.In addition,an independent test dataset was constructed to evaluate the segmentation accuracy of the models.The average precision using 50% intersection over union threshold(AP_(50)) and mean intersection over union(mIoU) were selected as performance indicators.The results indicated that the combination of GroundingDINO and SAM2(with a weight of approximately 810 MB) based on the text prompt “Leaf” achieved high-performance segmentation of poplar leaves,realizing AP_(50)=0.936,mIoU = 0.778.By filtering out bounding boxes with abnormal sizes,the AP_(50) value was increased to 0.942.The YOLOv8 segment model established by transfer learning possessed a weight file of only 6.5 MB,achieving an AP_(50) of 0.888,significantly simplifying the model while ensuring its accuracy.In this study,the leaf segmentation models did not require manual annotations,achieving efficient and low-cost segmentation of poplar leaf instances.Finally,the applications of the methods presented on leaf counting and leaf area calculation were conducted.Promising performances could be observed that the average percentage leaf counting error was approximately 6%,and the average percentage error of leaf area was about 12%.