基于决策边界分析的深度神经网络鲁棒性评估与优先次序验证

Robustness Evaluation and Prioritization Verification for Deep Neural Networks via Decision Boundary Analysis

随着深度学习技术在现实世界的广泛应用,人们对基于深度神经网络的系统安全性提出了更高要求.鲁棒性是神经网络的重要安全性质,对网络鲁棒性的量化分析和验证是深度学习模型安全性研究的关键问题.针对神经网络验证技术中难以解决的效率问题,提出了一种新颖的优先次序优化方法.结合局部鲁棒性的规约方式,在一组待验证输入内选择具有更高验证需求的不稳定点代替常规的逐点验证模式.根据对鲁棒性问题与决策边界距离的关联性分析,提出了一种基于网络输出单元值大小的鲁棒性评估方法作为优先验证的输入点选择依据.在此基础上将其扩展为输入的预分析模块与验证工具集成,进而设计了基于优先次序的验证框架.在常用的验证基准上进行了实验,结果表明,该方法的决策边界分析理论与突变测试结果一致,鲁棒性评估中选择不安全样本的平均准确率高于90%,通过减少安全样本的验证开销使验证效率提高了148.6%~432.6%.

With the wide application of deep learning in the real world,people put forward higher requirements for the security of the systems based on deep neural networks.Robustness is an im-portant safety property of neural networks,which is reflected in the vulnerability of models to adversarial perturbations.The quantitative analysis of network robustness is a key issue in the security research of deep learning models.Formal verification is an important technique to ensure the reliability of the models,using mathematical methods to construct rigorous encodings for models.Since neural networks have non-linear and large-scale structures,the existing verification technologies have intractable efficiency deficiencies.In view of this,a novel prioritization optimi-zation method is proposed,which reduces verification time by introducing a pre-analysis process for inputs during verification to reduce the scale of the tasks.Specifically,combined with the lim-itations of local robustness specification,unstable points are defined as the inputs with higher verification requirements within a set of inputs to be verified,in instead of the conventional point-by-point verification mode.The proposed optimization method does not break the balance be-tween verification accuracy and efficiency.In order to accurately select unstable points that are prone to unsafe,the causes of model robustness problems are analyzed in detail from the perspec-tive of decision boundaries,involving the generalization ability and overfitting issues.Points that are closer to the decision boundary are more likely to misclassify the neural network when per-turbed.Then,according to the correlation analysis of the robustness problem and the distance of the decision boundary,a robustness evaluation method based on the value of the network output unit is proposed as the input point selection basis for priority verification.A lightweight robust-ness metric based on the output difference is defined to reflect the distance relationship between the input point and the decision boundary,and advanced extension forms are presented.Also,the theoretical basis for this metric is provided in terms of adversarial attack and defense,and mutation testing.On this basis,the input pre-analysis module is extended to integrate with the verification tools.Furthermore,a prioritization-based verification framework is designed,and the working principle and specific process of the framework are demonstrated.The integration ways and implications of proposed method in different types of verification tools are discussed from practical application of view.Extensive experiments on commonly used verification benchmarks demonstrate the rationality and effectiveness of the proposed method.The accuracy of the metric is proved by comparing the consistency of the output results of different input points divided based on the robustness evaluation method in strict formal verification tools.The selected unsta-ble points are sequentially used as representatives of input points that need to be heavily consid-ered in the verification,and increasing efficiency by ignoring points that are probabilistically safe during the execution of the tools.The results show that the decision boundary analysis theory is consistent with the results of mutation testing,the average accuracy of selecting unsafe samples in robustness evaluation is higher than 90%,and the verification time is reduced by 148.6%~432.6%by declining the verification costs of safe samples.