基于均方误差的8位深度神经网络量化

8-BIT deep neural network quantization based on mean square error

为模型量化后具有更高的准确度,提出以量化均方误差(QMSE)为指标的确定量化系数的方法,针对量化后性能损失严重的小型网络,进一步提出更新统计参数(USP)的方法。QMSE将量化过程中的舍入和截断操作产生的噪声相结合,以此作为选取合适量化系数的指标;USP通过更新批次归一化层中的均值和方差,矫正模型量化产生的均值和方差偏移。实验结果表明,在不进行重训练的情况下,使用QMSE+USP对常见的深度神经网络量化,模型性能优于其它算法。

To achieve higher accuracy after the model is quantized,a method to determine the quantization scale with the quantized mean square error(QMSE)as an indicator was proposed.For small models with severe performance loss after quantization,the update of statistical parameters(USP)was further proposed.QMSE combined the noise generated by the rounding and truncation operations in the quantization process as an indicator for selecting appropriate quantization scale.USP corrected the mean and variance generated by model quantization by updating the mean and variance in the batch normalization layer.Experimental results show that using QMSE+USP to quantify common deep neural networks without retraining,the model perfor-mance is better than using other algorithms.