基于EfficientDet网络的湖羊短时咀嚼行为识别方法

Automatic Identification Method of Short-term Chewing Behaviour for Sheep Based on EfficientDet Network

为分析羊进食行为、自动估算其进食量,提出一种从舍饲湖羊采食视频中自动识别其短时咀嚼行为的方法。首先,针对舍饲湖羊采食区域特点,在EfficientDet网络架构中增加目标框筛选模块,检测视频帧中羊嘴张开、上下颌错开及闭合3种状态,根据羊脸与相机拍摄角度的方位关系检测羊嘴状态,并为各状态赋编码值;然后,利用正则表达式提取连续视频帧中的一次上下颌张合对应的羊嘴状态编码值序列片段;最后,针对羊侧脸面对相机咀嚼、抬头正脸面对相机咀嚼、低头正脸面对相机咀嚼以及鸣叫等一次上下颌张合动作对应的羊嘴状态编码值序列片段构建分类规则,实现短时咀嚼行为的自动识别。对比了基于EfficientDet-D0~D4、YOLO v5和SSD网络的羊嘴状态检测性能,结果表明,改进的EfficientDet-D1网络能以28.18 f/s的传输速率,获得95.64%和98.84%的羊嘴状态检测精确率和均值平均精确率,优于YOLO v5和SSD网络。利用湖羊采食视频测试EfficientDet-D1网络结合正则表达式的湖羊短时咀嚼行为识别分类规则性能,结果表明,分类规则能以91.42%的自动识别正确率和90.85%的平均正确率直接从视频中提取湖羊短时咀嚼行为发生次数和持续时长。本研究将基于视频的湖羊短时咀嚼行为识别问题转换为羊嘴状态编码值序列分类问题,降低了分类模型的复杂度,为湖羊短时咀嚼行为的自动识别提供了一种新的研究思路。

Animal’s short-term chewing behaviour is accomplished with discrete jaw movements which occurs through a repeating jaw opening-closing cycle.Recognition of short-term chewing behaviour is the foundation of feeding behaviour analysis and feed/pasture intake estimation for sheep.Several attempts have been made to establish models for short-term chewing behaviour recognition based on the jaw pressure or acoustic signal obtained using wearable sensors.However,such data collection methods have shortages such as difficulty in battery replacement,low stability of the data,and the sensors are vulnerable to damage.A short-term chewing behaviour identification method using computer vision technology was presented,which can extract the frequency and duration of each individual short-term chewing from the feeding video of sheep.Firstly,based on the characteristics of sheep feeding area,a module for target box selection was added in the EfficientDet network architecture.This modified EfficientDet network was employed to detect the three status of sheep mouth,that was opening,stagger of the upper/lower jaw,and closing,in each video frame.Once the sheep mouth status in a video frame was determined,a numerical label was assigned.Then,the regular expression was employed to extract the numerical label sequence segment corresponding to each individual jaw opening-closing cycle.Finally,classification rules were constructed for short-term chewing behaviour identification,where chewing with the side face facing the camera,chewing while facing the camera with the head down,chewing while facing the camera with the head up,and sheep chirping were distinguished.The performance of the sheep mouth status detection obtained by the modified EfficientDet-D0~D4 networks were compared with those obtained by the YOLO v5 and SSD networks.The comparison results indicated that the precision rate,mean average precision,and frame rate of the modified EfficientDet-D1 network was 95.64%,98.84%,and 28.18 f/s,respectively,which were better than those of YOLO v5 and SSD.Short-term chewing behaviour classification rules,which consisted of EfficientDetD1 network and regular expression,were applied to the testing videos.The testing results indicated that the frequency and duration of short-term chewing can be extracted from the videos with the accuracy of 91.42%and 90.85%,respectively.The developed method transformed the video-based sheep short-term behaviour identification problem into the problem of the status label sequences classification,which reduced the complexity of the short-term chewing behaviour classification task.The presented method provided a solution for the automatic short-term chewing behaviour recognition for sheep.