Deep convolutional neural networks(DCNNs)have shown outstanding performance in the fields of computer vision,natural language processing,and complex system analysis.With the improvement of performance with deeper laye...Deep convolutional neural networks(DCNNs)have shown outstanding performance in the fields of computer vision,natural language processing,and complex system analysis.With the improvement of performance with deeper layers,DCNNs incur higher computational complexity and larger storage requirement,making it extremely difficult to deploy DCNNs on resource-limited embedded systems(such as mobile devices or Internet of Things devices).Network quantization efficiently reduces storage space required by DCNNs.However,the performance of DCNNs often drops rapidly as the quantization bit reduces.In this article,we propose a space efficient quantization scheme which uses eight or less bits to represent the original 32-bit weights.We adopt singular value decomposition(SVD)method to decrease the parameter size of fully-connected layers for further compression.Additionally,we propose a weight clipping method based on dynamic boundary to improve the performance when using lower precision.Experimental results demonstrate that our approach can achieve up to approximately 14x compression while preserving almost the same accuracy compared with the full-precision models.The proposed weight clipping method can also significantly improve the performance of DCNNs when lower precision is required.展开更多
基金the National Natural Science Foundation of China(NSFC)under Grant Nos.61772077 and 61370192Beijing Natural Science Foundation of China under Grant No.4192051NSFC under Grant Nos.61428203 and 61572347.
文摘Deep convolutional neural networks(DCNNs)have shown outstanding performance in the fields of computer vision,natural language processing,and complex system analysis.With the improvement of performance with deeper layers,DCNNs incur higher computational complexity and larger storage requirement,making it extremely difficult to deploy DCNNs on resource-limited embedded systems(such as mobile devices or Internet of Things devices).Network quantization efficiently reduces storage space required by DCNNs.However,the performance of DCNNs often drops rapidly as the quantization bit reduces.In this article,we propose a space efficient quantization scheme which uses eight or less bits to represent the original 32-bit weights.We adopt singular value decomposition(SVD)method to decrease the parameter size of fully-connected layers for further compression.Additionally,we propose a weight clipping method based on dynamic boundary to improve the performance when using lower precision.Experimental results demonstrate that our approach can achieve up to approximately 14x compression while preserving almost the same accuracy compared with the full-precision models.The proposed weight clipping method can also significantly improve the performance of DCNNs when lower precision is required.
