基于混合扰动的差分隐私贝叶斯神经网络

Differential Privacy Bayesian Neural Network Based on Mixed Perturbation

深度学习在大数据技术中的使用给训练数据的保密性造成了极大的威胁。基于差分隐私的深度学习,能对深度学习模型的训练数据提供隐私保护。但现有差分隐私保护方式会造成深度学习模型可用性下降及隐私保护效果评估不够精确。为此,提出了一种基于混合扰动的差分隐私贝叶斯神经网络(Differential Privacy Bayesian Neural Network Based on Mixed Perturbation,DPBNNMP)。该方法将神经网络权重分解为网络权重和噪声权重,二者的叠加作为输出扰动(Output Perturbation,OTPN)。同时利用目标扰动(Objective Perturbation,OEPN)来避免噪声权重对数据的拟合。用神经网络权重分布计算的隐私开销评估隐私保护效果。实验结果表明,在相同的反向传播的隐私开销下,混合扰动的神经网络准确率比梯度扰动的神经网络准确率和目标扰动的神经网络准确率分别高出4%~70%和0~15%。前向传播的隐私开销也随着噪声标准差的增加而降低。

The use of deep learning in big data poses a great threat to the confidentiality of training data.Deep learning based on differential privacy can provide privacy protection for training data in the process of model training.However,the existing differential privacy protection methods lead to the decline of model availability and inaccurate evaluation of the effect of privacy protection.Therefore,a Differential Privacy Bayesian Neural Network Based on Mixed Perturbation(DPBNNMP)is proposed,which consists of output perturbation and objective perturbation.In this method,weights of neural networks are divided into network weights and noise weights,the sum of which serves as the Output Perturbation(OTPN).At the same time,training data fitting by the noise weights is avoided by using Objective Perturbation(OEPN).Finally,the privacy overhead calculated by neural network weights distribution is used to evaluate the effect of privacy protection.The evaluation results show that under the same privacy overhead of back propagation,the accuracy of mixed disturbance neural network is 4%to 70%higher than that of gradient disturbance neural network and 0 to 15%higher than that of target disturbance neural network,respectively.The privacy overhead of forward propagation also decreases with the increase of noise variance.