基于多模态图像信息及改进实例分割网络的肉牛体尺自动测量方法

Automatic Measurement Method of Beef Cattle Body Size Based on Multimodal Image Information and Improved Instance Segmentation Network

[目的/意义]牛的体尺参数是反映牛身体发育状况的关键指标,也是牛选育过程的关键因素。为解决规模化肉牛牧场复杂环境对肉牛体尺的测量需求,设计了一种图像采集装置以及体尺自动测量算法。[方法]首先搭建肉牛行走通道,当肉牛通过通道后进入限制装置,用英特尔双目深度相机D455对牛只右侧图像进行RGB与深度图的采集。其次,为避免复杂环境背景的影响,提出一种改进后的实例分割网络Mask2former来对牛只二维图进行前景轮廓提取,对轮廓进行区间划分,利用计算曲率分析方法找到所需体尺测点。然后,将原始深度图转换为点云数据,对点云进行点云滤波、分割和深度图牛只区域的空值填充,以保留牛体区域的点云完整,从而找到所需测点并返回到二维数据中。最后,将二维像素点投影到三维点云中,利用相机参数计算出投影点的世界坐标,从而进行体尺的自动化计算,最终提取肉牛体高、十字部高、体斜长和管围4种体尺参数。[结果与讨论]改进的实例分割网络与Mask R-CNN、PointRend、Queryinst等模型相比具有更好的分割结果。采用本研究测得的这4种体尺平均相对误差分别为4.32%、3.71%、5.58%和6.25%。[结论]本研究开发的肉牛图像采集装置及相应的图像处理方法可以满足该牧场对肉牛体尺无接触自动测量误差小于8%的精度要求,为非接触式肉牛体尺自动化测量提供了理论与实践指导。

[Objective]The body size parameter of cattle is a key indicator reflecting the physical development of cattle,and is also a key factor in the cattle selection and breeding process.In order to solve the demand of measuring body size of beef cattle in the complex environment of large-scale beef cattle ranch,an image acquisition device and an automatic measurement algorithm of body size were designed.[Methods]Firstly,the walking channel of the beef cattle was established,and when the beef cattle entered the restraining device through the channel,the RGB and depth maps of the image on the right side of the beef cattle were acquired using the Inter RealSense D455 camera.Secondly,in order to avoid the influence of the complex environmental background,an improved instance segmentation network based on Mask2former was proposed,adding CBAM module and CA module,respectively,to improve the model's ability to extract key features from different perspectives,extracting the foreground contour from the 2D image of the cattle,partitioning the contour,and comparing it with other segmentation algorithms,and using curvature calculation and other mathematical methods to find the required body size measurement points.Thirdly,in the processing of 3D data,in order to solve the problem that the pixel point to be measured in the 2D RGB image was null when it was projected to the corresponding pixel coordinates in the depth-valued image,resulting in the inability to calculate the 3D coordinates of the point,a series of processing was performed on the point cloud data,and a suitable point cloud filtering and point cloud segmentation algorithm was selected to effectively retain the point cloud data of the region of the cattle's body to be measured,and then the depth map was 16.Then the depth map was filled with nulls in the field to retain the integrity of the point cloud in the cattle body region,so that the required measurement points could be found and the 2D data could be returned.Finally,an extraction algorithm was designed to combine 2D and 3D data to project the extracted 2D pixel points into a 3D point cloud,and the camera parameters were used to calculate the world coordinates of the projected points,thus automatically calculating the body measurements of the beef cattle.[Results and Discussions]Firstly,in the part of instance segmentation,compared with the classical Mask R-CNN and the recent instance segmentation networks PointRend and Queryinst,the improved network could extract higher precision and smoother foreground images of cattles in terms of segmentation accuracy and segmentation effect,no matter it was for the case of occlusion or for the case of multiple cattles.Secondly,in three-dimensional data processing,the method proposed in the study could effectively extract the three-dimensional data of the target area.Thirdly,the measurement error of body size was analysed,among the four body size measurement parameters,the smallest average relative error was the height of the cross section,which was due to the more prominent position of the cross section,and the different standing positions of the cattle have less influence on the position of the cross section,and the largest average relative error was the pipe circumference,which was due to the influence of the greater overlap of the two front legs,and the higher requirements for the standing position.Finally,automatic body measurements were carried out on 137 beef cattle in the ranch,and the automatic measurements of the four body measurements parameters were compared with the manual measurements,and the results showed that the average relative errors of body height,cross section height,body slant length,and tube girth were 4.32%,3.71%,5.58%and 6.25%,respectively,which met the needs of the ranch.The shortcomings were that fewer body-size parameters were measured,and the error of measuring circumference-type body-size parameters was relatively large.Later studies could use a multi-view approach to increase the number of body rule parameters to be measured and improve the accuracy of the parameters in the circumference category.[Conclusions]The article designed an automatic measurement method based on two-dimensional and three-dimensional contactless body measurements of beef cattle.Moreover,the innovatively proposed method of measuring tube girth has higher accuracy and better implementation compared with the current research on body measurements in beef cattle.The relative average errors of the four body tape parameters meet the needs of pasture measurements and provide theoretical and practical guidance for the automatic measurement of body tape in beef cattle.