基于关联规则的电网大数据质量校验方法研究

Research on quality check method of power grid big data based on association rules

为解决基于传统数据库质量校验方法、基于独立服务器质量校验方法受到冗余数据干扰,导致大数据质量校验结果不理想的问题,提出了基于关联规则的电网大数据质量校验方法研究。分析基于关联规则电网不安全大数据诱因,在关联规则支持度、置信度和相关度支持下,计算不安全大数据诱发度,剔除冗余数据。采用增量式方法校验诱发因子,构建校验索引,遵照关联规则并行化处理诱发因子数量。将Oracle数据库中数据交付到集成服务器上,设置4条数据集成路径,通过Java消息服务应用程序接口,触发数据转换功能,借助网络浏览器生成大数据质量校验统计结果。由实验结果可知,该方法当月新增和当日新增数据校验结果更加贴近理想校验结果,对电网大数据合理设置提供较好的指导意义。

In order to solve the problem that the traditional database quality verification method and the independent server quality verification method are interfered by redundant data,resulting in unsatisfactory results of large data quality verification,a large data quality verification method based on association rules is proposed the study.The analysis is based on the association rules for unsafe big data in the power grid.With the support of association rules support,confidence and relevance,the inducibility of unsafe big data is calculated and redundant data is eliminated.Adopt an incremental method to check the induced factors,construct a verification index,and parallelize the number of induced factors in accordance with the association rules.Deliver the data in the Oracle database to the integration server,set up four data integration paths,trigger the data conversion function through the Java message service application program interface,and generate the big data quality verification statistical results with the help of a web browser.From the experimental results,it can be seen that the new and new data verification results of this method are closer to the ideal verification results,and provide better guidance for the reasonable setting of grid big data.