基于神经网络的HEVC帧间预测方法及其硬件研究

HEVC inter prediction method and its hardware research based on neural network

相比于H.264,高效视频编码标准(HEVC)提出了许多新技术,提高了编码性能,但是也显著提高了编码复杂度。本文从硬件实现的角度出发,对已有的帧间CU划分预测神经网络的结构进行了多方面的优化,使其参数减少了70%,加法和乘法运算分别减少了60%、58.2%。并对优化后的卷积神经网络参数采用10位定点数方案进行定点化处理,进一步有效减少硬件资源的开支。对比于HEVC参考软件(HM16.5),优化后网络引起的BD-BR和BD-PSNR平均损失为1.718%和-0.056dB,平均节省35%~52%的编码复杂度,并且定点化处理后引起的性能损失可忽略不计。

Compared with H.264,the High Efficiency Video Coding Standard(HEVC)proposes many new technologies,which improve the coding performance,but also significantly increase coding complexity.From the perspective of hardware implementation,this paper optimizes the structure of the existing inter frame CU partition prediction neural network in many aspects,reducing its parameters by 70%,and reducing addition operations and multiplication operations by 60%and 58.2%,respectively.The optimized convolutional neural network parameters are processed by a 10-bit fixed-point number scheme for fixed-point processing,which further effectively reduces the expenditure of hardware resources.Compared with the HEVC reference software(HM16.5),the average loss of BD-BR and BD-PSNR caused by the optimized network is 1.718%and-0.056dB,and the average coding complexity is saved by 35%~52%,and the performance loss caused by fixed-point processing is negligible.