深度强化学习模型轻量化算法研究

Research on Lightweight Algorithms for Deep Reinforcement Learning

针对深度强化学习网络难以部署到资源受限终端设备的问题,本文提出一种深度神经网络优化压缩算法。该算法引入倒残差模块作为主干网络,实现网络的轻量化;采用基于响应的知识蒸馏,以动作策略为蒸馏目标,弥补网络轻量化造成的精度损失;采用基于特征的知识蒸馏,对网络中间层的特征向量进行蒸馏,进一步提升网络精度。实验结果表明,轻量化后的网络参数量为19.79M,参数量为原网络的59.8%,性能提升约12.1%,且在网络轻量化的同时,提升了模型表现,验证了所提算法的有效性。

In response to the difficulty of deploying deep reinforcement learning networks on resource- constrained terminal devices, a deep neural network optimization compression algorithm is proposed in this paper. This algorithm introduces an inverse residual module as the backbone network to achieve the lightweight of network;adopts response-based knowledge distillation, with action strategy as the distillation target, to make up for the accuracy loss caused by the lightweight of network;adopts feature-based knowledge distillation to distill the feature vectors in the middle layer of the network, further improving network accuracy. Experimental results show that the parameter size of the lightweight network is 19.79M, the parameter size is 59.8% of the original network, the performance is improved by about 12.1%, and the model performance is improved while the network is lightweight, verifying the effectiveness of the proposed algorithm.