结合内卷与卷积算子的视频预测模型

Video prediction model combining involution and convolution operators

针对基于传统深度学习的视频预测中对数据空间特征提取效果不佳及预测精度低的问题,提出一种结合内卷与卷积算子(CICO)的视频预测模型。该模型主要通过以下三个方面提高视频序列的预测性能:首先,采用不同大小的卷积核增强对数据多粒度空间特征的提取能力,较大的卷积核能够提取更大空间范围的特征,而较小的卷积核可更精确地捕获视频目标的运动细节,实现对目标多角度表征学习;其次,用计算效率更高、参数更少的内卷算子替代核较大的卷积算子,内卷通过高效的通道间交互避免了大量的不必要参数,在降低计算和存储成本的同时提升模型预测能力;最后,引入核为1×1的卷积进行线性映射,增强不同特征之间的联合表达,提高了模型参数的利用效率并增强了预测的鲁棒性。通过多个数据集对该模型进行全面测试,结果表明,相较于目前最优的SimVP(Simpler yet better Video Prediction)模型,所提模型在多项指标上均有显著提升。在移动手写数据集上,均方误差和平均绝对误差分别降低25.2%和17.4%;在北京交通数据集上,均方误差降低1.2%;在人体行为数据集上,结构相似性指数和峰值信噪比分别提高0.66%和0.47%。可见,所提模型在提升视频预测精度方面十分有效。

To address the inadequate feature extraction from data space and low prediction accuracy in traditional deep learning based video prediction,a video prediction model Combining Involution and Convolution Operators(CICO)was proposed.The model enhanced video prediction performance through three aspects.Firstly,convolutions with varying kernel sizes were adopted to enhance extraction ability of multi-granularity spatial features and enable multi-angle representational learning of targets.In particular,larger kernels were applied to extract features from broader spatial ranges,while smaller kernels were employed to capture motion details more precisely.Secondly,large-kernel convolutions were replaced by the computationally efficient involution operators with fewer parameters in order to achieve efficient inter-channel interaction,avoid redundant parameters,decrease computational and storage costs.The predictive capacity of the model was enhanced at the same time.Finally,convolutions with kernel size 1×1 were introduced for linear mapping to strengthen joint expression between distinct features,improve parameter utilization efficiency,and strengthen prediction robustness.The proposed model’s superiority was validated through comprehensive experiments on various datasets,resulting in significant improvements over the state-of-the-art SimVP(Simpler yet Better Video Prediction)model.On Moving MNIST dataset,the Mean Squared Error(MSE)and Mean Absolute Error(MAE)were reduced by 25.2%and 17.4%,respectively.On Traffic Beijing dataset,the MSE was reduced by 1.2%.On KTH dataset,the Structure Similarity Index Measure(SSIM)and Peak Signal-to-Noise Ratio(PSNR)were improved by 0.66%and 0.47%,respectively.It can be seen that the proposed model is very effective in improving accuracy of video prediction.