面向对抗样本的深度神经网络可解释性分析

Interpretability Analysis of Deep Neural Networks With Adversarial Examples

虽然深度神经网络(Deep neural networks,DNNs)在许多任务上取得了显著的效果,但是由于其可解释性(Interpretability)较差,通常被当做"黑盒"模型.本文针对图像分类任务,利用对抗样本(Adversarial examples)从模型失败的角度检验深度神经网络内部的特征表示.通过分析,发现深度神经网络学习到的特征表示与人类所理解的语义概念之间存在着不一致性.这使得理解和解释深度神经网络内部的特征变得十分困难.为了实现可解释的深度神经网络,使其中的神经元具有更加明确的语义内涵,本文提出了加入特征表示一致性损失的对抗训练方式.实验结果表明该训练方式可以使深度神经网络内部的特征表示与人类所理解的语义概念更加一致.

Deep neural networks(DNNs)have demonstrated impressive performance on many tasks,but they are usually considered opaque due to their poor interpretability.In this paper,we examine the internal representations of DNNs on image classification tasks using adversarial examples,which enable us to analyze the interpretability of DNNs in the perspective of their failures.Based on the analyses,we find that the learned features of DNNs are inconsistent with human-understandable semantic concepts,making it problematic for understanding and interpreting the representations inside DNNs.To realize interpretable deep neural networks,we further propose an adversarial training scheme with a consistent loss such that the neurons are endowed with the human-interpretable concepts to improve the interpretability of DNNs.Experiments show that the proposed method can make the features in DNNs more consistent with semantic concepts.