基于YOLO V7及TSM模型的橘小实蝇梳理行为检测与识别

Detection and recognition of grooming behaviors of Bactrocera dorsalis(Diptera:Trypetidae)based on YOLO V7 and TSM model

【目的】昆虫梳理行为的统计分析和研究对害虫控制和人类健康非常重要,针对传统的人工记录梳理行为的方法费时费力且易出错,提出了一种基于计算机视觉和深度学习的橘小实蝇Bactrocera dorsalis梳理行为检测和识别方法。【方法】首先将橘小实蝇视频数据进行图像处理得到帧图像,筛选其中3000张图像作为训练数据集;建立YOLO V7目标检测算法来检测视频数据中的橘小实蝇目标个体,框选中目标后通过视频处理算法进行裁剪;最后通过迁移学习方法将预训练权重迁移到训练模型中,利用基于非局部注意力改进的时间转换模块(temporal shift module,TSM)深度学习模型识别橘小实蝇的7种梳理行为(前足梳理、头部梳理、前中足梳理、中后足梳理、后足梳理、翅梳理以及静止)。【结果】橘小实蝇原始视频数据集通过YOLO V7目标检测算法训练的准确率为99.2%,召回率为99.1%。本研究算法处理后的视频数据集通过基于非局部注意力模块改进的TSM模型识别和统计梳理行为,最终平均准确率达到了97%以上,标准偏差低于3%。与其他4种深度学习模型(I3D,R2+1D,SlowFast和Timesformer)对比,本研究方法的准确率最高提升了9.76%,保证了橘小实蝇梳理行为检测和识别的准确率和正确性。【结论】本研究提出的方法大大减少了人工观察的时间,同时保证了橘小实蝇梳理行为识别的准确性,为研究昆虫行为提供了新的思路和方法,为智慧农业的现代化发展添砖加瓦。

【Aim】Statistical analysis and study of insect grooming behaviors are important for pest control and human health.In view that the traditional method of manually recording grooming behaviors is time-consuming and error-prone,we proposed a computer vision and deep learning-based method for the detection and recognition of grooming behaviors of the Oriental fruit fly,Bactrocera dorsalis.【Methods】First,we processed the B.dorsalis video data to obtain frame images,and screened 3000 images as the training dataset.We built the YOLO V7 target detection algorithm to detect B.dorsalis target in video data,and framed the target and cropped it by video processing algorithm.Finally,we migrated the pre-training weights to the training model by transfer learning method,and recognized seven grooming behaviors(foreleg grooming,head grooming,fore-midleg grooming,mid-hindleg grooming,hindleg grooming,wing grooming and stationary)of B.dorsalis using the temporal shift module(TSM)deep learning model based on non-local attention improvement.【Results】The accuracy and recall rate of the original video of the B.dorsalis dataset trained by YOLO V7 target detection algorithm were 99.2%and 99.1%,respectively.Applying this research algorithm to process the video dataset,and then recognizing and counting grooming behaviors through an improved TSM model based on the non-local attention module,we got the final average accuracy of over 97%with a standard deviation of less than 3%.Compared with the other four deep learning models(I3D,R2+1D,SlowFast and Timesformer),this research method had a~9.76%improvement in accuracy,ensuring the accuracy of B.dorsalis grooming behavior detection and recognition.【Conclusion】The method proposed in this study greatly reduces the time of manual observation,and ensures the accuracy of grooming behavior recognition of B.dorsalis,providing new ideas and methods for researching insect behavior and contributing to the modern development of intelligent agriculture.