基于广义马氏距离的缺损数据补值算法

Missing Data Imputation Based on Generalized Mahalanobis Distance

在数据收集过程中数据缺损是不可避免的。如何还原这些缺损数据,成为数据挖掘研究的热点问题之一。与许多现有算法一样,基于马氏距离的缺损数据补值算法充分利用了实际数据之间的相关性,具有较好的补值效果,但它要求数据的相关性协方差矩阵可逆,使其应用范围受到了极大的限制。在改进传统主成分分析方法的基础上,利用矩阵的奇异值分解理论和Moore-Penrose广义逆性质,提出了广义马氏距离的概念,并运用于SOFM神经网络,结合信息熵理论设计了基于广义马氏距离的缺损数据补值算法——GS算法。理论分析和数值仿真结果表明,广义马氏距离完全继承了马氏距离在处理相关性数据上的性能优势,新算法不仅在补值的精确度和稳定性上有很好的效果,而且适用于任意数据集合。

Missing data are inevitable in data-collection,how to restore these data has become one of the hottest issues in data mining.Just like most algorithms,missing data imputation algorithms based on Mahalanobis Distance make full use of relationships between data.Though the results are acceptable,the covariance matrixes are not always reversible,which limit the algorithms greatly.This paper improved a traditional principal component analysis（PCA） method,proposed a new distance named Generalized Mahalanobis Distance according to SVD and Moore-Penrose pseudoinverse.Combining with SOFM neural network and entropy,we designed GS missing data imputation algorithms.After academic analysis and simulation,it was proved that Generalized Mahalanobis Distance inherits the advantages of Mahalanobis Distance wonderfully in dealing with relatived data.Not only the new algorithm has good accuracy and stability,but also suits for any datasets.