面向分类网络的视觉语义解释模型

Visual Semantic Interpretation Model for Classification Network

深度学习的可解释性对推动其在军事场景中应用至关重要。当前主流方法使用类激活图的方式可视化最后一层卷积特征,然而对于网络根据该特征进行分类解释比较模糊。针对此问题,设计一种面向分类网络的视觉语义解释模型。综合考虑前向传播与反向传播,提出CGNIS算法获取对分类结果起重要作用的神经元,并将其映射到原图,得到更加细化的视觉特征。提出分类网络IRENet,在VGG16中间层添加SIRM和ECA对视觉特征进行识别,更加客观地提取视觉特征中包含的语义特征,并结合视觉特征、语义特征、重要神经元分数信息生成描述模型分类过程的解释性语句。在ImageNet2012数据集上提取10类图像进行实验,结果表明,删除CGNIS算法得到的某一类重要神经元后,对应类的分类准确率下降3%以上,在语义特征提取任务上,IRENet的F1值、准确率、精确率和召回率4项指标较ResNet101等分类网络提升2%以上。此外,利用CGNIS、IRENet对飞机类别进行实验,可生成模型对其分类过程的解释性语句。

The interpretability of deep learning is critical for promoting its application in military scenarios.Currently used mainstream methods involve class activation maps to visualize the last layer of convolutional features,but it is not clear how the network classifies based on this feature.To solve this problem,a visual semantic interpretation model is proposed for classification network.First,considering forward propagation and backward propagation,the CGNIS algorithm is proposed to obtain neurons that are important for classification and map them to the original image to obtain more refined visual features.Second,the classification network Indian Rural Energy Network(IRENet)is proposed.SIRM and ECA are added to the middle layer of VGG16 to recognize visual features.The semantic information contained in these visual features can be extracted more objectively,and the obtained visual features,semantic features,and important neuron score information can be combined to generate the explanatory statements describing the classification process of the model.Finally,10 classes are extracted from ImageNet2012 dataset for experiments.Experiments show that the classification accuracy of the corresponding class decreases by more than 3%after removing the important neurons of a certain class obtained by applying the CGNIS algorithm.In terms of semantic feature extraction task,the F1 value,accuracy,precision and recall of IRENet are improved by more than 2%compared with ResNet101 and other classification networks.In addition,the experiments are conducted on the aircraft class using CGNIS and IRENet to generate model statements explaining its classification process.