基于主成分分析和函数机制的差分隐私线性回归算法

Differential Privacy Linear Regression Algorithm Based on Principal Component Analysis andFunctional Mechanism

随着人工智能应用的不断落地以及隐私保护法律法规的持续出台,机器学习中的隐私保护问题已成为目前信息安全领域的一个研究热点。文中针对现有的差分隐私线性回归算法全局敏感度大、模型可用性较差的问题,基于高斯机制代替传统的Laplace机制,并通过在算法的两个主要阶段分别添加噪声的方法,提出了一种基于主成分分析和函数机制的差分隐私线性回归算法(PCAFM-DPLR)。首先,为了在降维的同时兼顾数据的隐私性,向原始数据集的协方差矩阵中注入高斯噪声,基于主成分分析得到具有差分隐私保护效果的低维数据集;其次,为防止模型训练过程中可能存在的隐私泄露,再向目标函数的展开多项式系数添加高斯噪声,并以扰动后的目标函数最小化为目标,求得最优模型参数。理论分析和实验结果表明,PCAFM-DPLR算法训练出的线性回归模型能够在有效保证隐私性的同时,具有良好的可用性。

With the continuous development of artificial intelligence applications and the subsequent promulgation of privacy protection laws and regulations,the privacy protection issue in machine learning has become a research hotspot in the field of information security.To overcome the issues of high global sensitivity and poor model usability of the existing differential privacy linear regression algorithms,we present a differential privacy linear regression algorithm based on principal component analysis and functional mechanism(PCAFM-DPLR).In the PCAFM-DPLR algorithm,the traditional Laplace mechanism is replaced by the Gaussian mechanism,and the noise is added in the two major stages of the algorithm respectively.First,in order to take into account the privacy of the data while reducing the dimensionality,Gaussian noise is injected into the covariance matrix of the original data set,and a low-dimensional data set with differential privacy protection effect is obtained based on principal component analysis.Second,to prevent the possible privacy leakage during the model training,Gaussian noise is then added to the expansion polynomial coefficients of the objective function,and the minimization of the perturbed objective function is used as the objective to find the optimal model parameters.Theoretical analysis and experimental results show that the linear regression model trained by the PCAFM-DPLR algorithm can effectively guarantee privacy while having good utility.