注意力机制引导的神经架构搜索在图像分类中的应用

Application of Attention-Guided Neural Architecture Search in Image Classification

神经架构搜索(Neural Architecture Search, NAS)是一种能够在预定义搜索空间内找到最优神经网络架构的方法,在图像分类等任务上具有重要的意义。注意力引导的架构搜索(Attention-guided Micro and Macro-Architecture Search, AGNAS)能根据注意力机制对重要操作赋予高的权重,避免了传统NAS方法中操作被错误选择的问题。本文分别从注意力机制和操作单元两个方面进一步对AGNAS方法进行改进。注意力机制方面,将通道注意力更换为瓶颈注意力(Bottleneck Attention Module, BAM)模块,即在通道注意力模块后增加空间注意力模块,以同时关注通道和空间重要特征。操作单元方面,引入ShuffleUnit操作,增加操作选择的多样性,进一步提升网络的特征表达能力。CIFAR-10数据集上的实验结果表明,与原始AGNAS相比,本文的改进方法能获得更低的分类错误率和更高的稳定性;与其他一些架构搜索方法相比,本文方法在分类错误率、参数量和搜索时间等方面的总体性能上更具优势。Neural Architecture Search (NAS) is a method that can find the optimal neural network architecture within a predefined search space and is of great significance in tasks such as image classification. Attention-Guided Micro and Macro-Architecture Search (AGNAS) can assign high weights to important operations based on attention mechanisms, avoiding the problem of incorrect selection of operations in traditional NAS methods. In this paper, we improve the AGNAS method from two aspects: attention mechanism and operation unit. Regarding attention mechanism, channel attention is replaced with bottleneck attention (BAM) module, which adds spatial attention module after channel attention module to focus on channel and spatial important features simultaneously. In terms of operation units, ShuffleUnit operation is introduced to increase the diversity of operation selection, further improving the feature expression ability of the network. The experimental results on the CIFAR-10 dataset show that compared with the original AGNAS, our method can achieve lower classification error and higher stability;Compared with the other architecture search methods, our method has overall advantages in classification error, parameters, and search time.