基于有偏采样的连续进化神经架构搜索

Continuous Evolutionary Neural Architecture Search Based on Biased Sampling

由于需要对每一个搜索到的架构进行独立的性能评估,神经架构搜索(NAS)往往需要耗费大量的时间和计算资源。提出一种基于有偏采样的连续进化NAS方法(OEvNAS)。OEvNAS在架构搜索过程中维护一个超网络,搜索空间中所有的神经网络架构都是该超网络的子网络。在演化计算的每一代对超网络进行少量的训练,子网络直接继承超网络的权重进行性能评估而无需重新训练。为提高超网络的预测性能,提出一种基于有偏采样的超网络训练策略,以更大的概率训练表现优异的网络,在减少权重耦合的同时提高训练效率。此外,设计一种新颖的交叉变异策略来提高算法的全局探索能力。在NATS-Bench和可微分架构搜索(DARTS)两个搜索空间上验证OEvNAS的性能。实验结果表明,OEvNAS的性能超越了对比的主流算法。在NATS-Bench搜索空间上,提出的超网络训练策略在CIFAR-10、CIFAR-100和ImageNet16-200上均取得了优异的预测性能;在DARTS搜索空间上,搜索到的最优神经网络架构在CIFAR-10和CIFAR-100上分别取得了97.67%和83.79%的分类精度。

Neural Architecture Search(NAS)typically requires a considerable amount of time and computing resources due to the independent performance evaluation of each architecture it searches.To address this challenge,the continuous evolutionary NAS method based on biased sampling(OEvNAS)is proposed.This method involves the maintenance of a supernet during the architecture search,where all neural network architectures within the search space are subsets of this supernet.Throughout each evolutionary computation generation,the supernet is trained for a few epochs.Subsequently,the subnets inherit the supernet's weights for performance evaluation,eliminating the need for retraining.To enhance the supernet's prediction performance,a training strategy based on biased sampling is introduced.This strategy prioritizes training superior networks,thereby augmenting training efficiency and diminishing weight coupling.Additionally,an innovative crossover and mutation strategy is implemented to enhance global exploration capabilities.The effectiveness of OEvNAS is tested on two search spaces,NATS-Bench and Differentaible Architecture Search(DARTS).Results indicate that OEvNAS outperforms comparative leading algorithms.In the NATS-Bench search space,the new supernet training strategy demonstrates remarkable prediction accuracy on CIFAR-10,CIFAR-100 and ImageNet16-200.In the DARTS search space,the optimally searched neural network architecture exhibits classification accuracies of 97.67%and 83.79%on CIFAR-10 and CIFAR-100,respectively.