基于PWC-Net的多层权值和轻量化改进光流估计算法

Multi-layer weight and lightweight improved optical flow estimation algorithm based on PWC-Net

针对现有光流估计方法实时性不够的问题,提出轻量化的深度可分离卷积的PWC-Net改进模型(depth separable pyramid,warping and cost volume,DS-PWC)。其改进是将常规二维卷积网络层解耦为深度可分离卷积层,并且DS-PWC在金字塔层增加基于层数的权重系数,从而使得网络结构在不损失精度的情况下大幅减少模型参数量。在训练过程中,使用图像及对象感知数据随机擦除(image and object-aware random erasing,I+ORE)等数据增强技术,进一步提升估计预测结果泛化能力。实验结果表明,在数据集测试DS-PWC模型,在保持质量的同时运行效率达到约58 fps(frame per second)。同时为了验证算法有效性,进行了模型结构和数据增强的消融实验。结果证明了DS-PWC模型的有效性。

To solve the problem of insufficient real-time performance of existing optical flow estimation methods,this paper proposed a lightweight improved PSC-NET model with deep separable convolution(depth pyramid,warping and cost volume,DS-PWC).One of the improvements was decoupling the conventional two-dimensional convolutional network layer into a deep separable convolutional layer.The other one was adding a weight coefficient based on the number of layers in the pyramid layer by DS-PWC,which greatly reduced the number of model parameters in the network structure without loss of precision.In addition,in the training process,the paper applied data enhancement technologies such as I+ORE to further improve the generalization ability of estimation and prediction results.The experimental results show that the DS-PWC model was tested in the dataset and the operating efficiency reaches about 58 fps while maintaining the quality.To verify the effectiveness of the algorithm,this paper carried out the ablation experiments of model structure and data enhancement.The results verify the effectiveness of the DS-PWC model.