自动化构建移动端神经网络的技术研究

Research on the technology of automatic construction of mobile terminal neural networks

深度学习已在视觉、语音、自然语言等应用领域取得巨大成功,然而随着网络结构日趋复杂,神经网络参数量也迅速增长,设计网络结构和调节参数这一过程需要大量的专业知识与反复试验,成本极高。此外,由于功耗限制与存储空间等因素,移动端设备上的神经网络模型规模受限。设计了一种高效的移动端神经网络架构搜索算法,具体包括:(1)设计了一种在预先给定神经网络架构的情况下可以自动计算模型浮点数运算次数的算法;(2)改进现有的基于梯度的神经网络架构搜索算法,设计了一种带约束的架构搜索算法;(3)在神经网络架构搜索过程中加入对浮点数运算次数的约束,通过调节约束的强弱搜索到几种不同的神经网络架构。训练搜索到的神经网络,测试其在图像分类任务上的性能,并与工业界常用的模型相比较。实验结果表明,该方法搜索到的模型能达到目前工业界主流模型性能。

Deep learning has achieved great success in many areas such as computer vision, speech signal processing and natural language processing. However, as the neural architecture becomes more complex, the number of parameters increases rapidly. Designing efficient neural architectures requires expertise knowledge and quantities of repetitive experiments, which leads to high cost.Besides, the scale of neural networks running in mobile devices is strictly limited due to the power consumption limit and relatively small storage space. This paper proposes an efficient architecture searching algorithm for mobile devices. The main contributions include :(1) We propose an algorithm for calculating the number of floating point operations in neural networks under given architecture.(2) We improve an existing gradient based neural architecture search algorithm and propose a constraint NAS algorithm.(3) We search several efficient neural architectures by adding the constraint of the number of floating point operations to the architecture searching process and adjusting its strength. We train the searched neural networks, test their performance on image classification tasks and compare with other neural networks which are commonly used in industry. The experimental results show that the performance of the model searched by our method can reach the performance of mainstream models in industry.