Analyzing Interpretability Semantically via CNN Visualization

Deep convolutional neural networks are widely used in image recognition,but the black box property is always perplexing.In this paper,a method is proposed using visual annotation to interpret the internal structure of CNN from the semantic perspective.First,filters are screened in the high layers of the CNN.For a certain category,the important filters are selected by their activation values,frequencies and classification contribution.Then,deconvolution is used to visualize the filters,and semantic interpretations of the filters are labelled by referring to the visualized activation region in the original image.Thus,the CNN model is interpreted and analyzed through these filters.Finally,the visualization results of some important filters are shown,and the semantic accuracy of filters are verified with reference to the expert feature image sets.In addition,the results verify the semantic consistency of the same important filters under similar categories,which indicates the stability of semantic annotation of these filters.