神经网络结构搜索方法综述

A review of neural network structure search method

如今,深度学习广泛地应用于生活、工作中的各个方面,给我们带来了极大的便利.在此背景下,需要设计针对不同任务的神经网络结构,满足不同的需求.但是,人工设计神经网络结构需要专业的知识,进行大量的实验.因此,神经网络结构搜索算法的研究显得极为重要.神经网络结构搜索(NAS)是自动深度学习(AutoDL)过程中的一个基本步骤,对深度学习的发展与应用有着重要的影响.早期,一些神经网络结构搜索算法虽然搜索到了性能优越的神经网络结构,但是需要大量的计算资源且搜索效率低下.因此,研究人员探索了多种设计神经网络结构的算法,也提出了许多减少计算资源、提高搜索效率的方法.本文首先简要介绍了神经网络结构的搜索空间,其次对神经网络结构搜索算法进行了全面的分类汇总、分析,主要包括随机搜索算法、进化算法、强化学习、基于梯度下降的方法、基于顺序模型的优化算法,再其次探索并总结了提高神经网络结构搜索效率的方法,最后探讨了目前神经网络结构搜索工作中存在的问题以及未来的研究方向.

Nowadays,deep learning is widely used in all aspects of life and work,which brings us great convenience.In this context,we need to design neural network structure for different tasks to meet different needs.However,manually design of neural network structure needs professional knowledge and a lot of experiments.Therefore,the research of neural network structure search algorithm is very important.Neural network structure search(NAS)is a basic step in the process of automatic deep learning(AutoDL),which has an important impact on the development and application of deep learning.In the early stage,although some neural network structure search algorithms have found excellent neural network structure,they need a lot of computing resources and search efficiency is low.Therefore,researchers have explored a variety of neural network structure design algorithms,and also proposed many methods to reduce computing resources and improve search efficiency.Firstly,we briefly introduce the search space of neural network structure.Secondly,we make a comprehensive classification,summary and analysis of neural network structure search algorithms,including random search algorithm,evolutionary algorithm,reinforcement learning,gradient descent based method and optimization algorithm based on sequential model.Thirdly,we explore and summarize methods to improve the efficiency of neural network structure search.Finally,we discuss the problems existing in the current neural network structure search work and the future research direction.