一种基于属性显著度的实体解析算法

An Entity Resolution Algorithm Based on Attribute Salience

实体解析(ER)是数据集成和数据清洗的一个重要步骤。在领域数据清洗与集成中,实体中不同的属性通常能表现出不同的区分能力,计算并利用属性的区分能力能够提高记录相似度的精确度。目前实体解析的方法有采用基于字符串的记录相似度算法以及基于机器学习的算法等方法来计算记录相似度,缺少考虑不同属性的重要程度。因此本文利用SimRank和PageRank算法的思想并结合随机抽样得到的属性显著度提出了一种基于属性显著度的计算记录相似度算法。首先,构造一个加权的属性记录对二部图来表示属性与记录对之间的关系;其次,根据属性显著度结合图论相似度算法提出了基于属性显著度的计算记录相似度的迭代算法。最后,构造一个记录图来表示记录对之间的匹配概率(二部图中的权值 w(ri,rj)),并使用改进的随机游走算法估计记录对匹配的概率。再将记录对的匹配概率反馈给加权的属性记录对二部图,并对基于属性显著度的计算记录相似度算法中的权值w(ri,rj) 进行修正,直到收敛。利用房地产领域数据集进行了实验评估,结果表明,本文提出的基于属性显著度的实体解析算法与主流方法相比,具有较高的精确度。

Entity resolution (ER) is an important step in data integration and data cleansing. In domain data cleaning and integration, different attributes in an entity usually exhibit different discriminating abilities. Calculating and utilizing the discriminating abilities of attributes can improve the accuracy of record similarity. Current entity resolution methods include record similarity algorithm based on string and algorithm based on machine learning to calculate record similarity, which lacks the im-portance of considering different attributes. Therefore, this paper uses the idea of SimRank and PageRank algorithm and combines the attribute salience obtained by random sampling to propose a similarity algorithm based on attribute salience. Firstly, a weighted attribute record pair bipartite graph is constructed to represent the relationship between attribute and record pair. Secondly, an iterative algorithm for calculating record similarity based on attribute significance is proposed ac-cording to attribute significance combined with graph similarity algorithm. Finally, a record graph is constructed to represent the matching probability between the record pairs (the weight in the bipartite graph), and the improved random walk algorithm is used to estimate the matching probability of the record pairs. Then, the matching probability of record pairs is fed back to the weighted bipartite graph of attribute record pairs, and the weight in the algorithm of calculating record similarity based on attribute salience is modified until convergence. Experi-mental evaluation using real estate data sets shows that the proposed entity resolution algorithm based on attribute salience is more accurate than the mainstream methods.