基于选择性加载策略的电能质量数据处理

Method for processing power quality data based on selective reloading

根据电能质量系统中监测数据海量化的趋势,提出了一种基于部分存储和选择性加载的数据处理算法,彻底解决了现有数据处理算法中重复排序和多余处理的问题。在计算日指标时,根据存储率存储部分日排序数据;在计算周(月、季、年)指标时,利用多路归并算法将存储的部分日排序数据合并,计算出临时95概率大值(CP95);根据临时CP95确定需要重载的日数据,对部分存储的日数据和重载数据重新排序以计算稳态指标。部分存储的日排序数据可以重复利用,有效解决了传统处理方案中的重复排序问题;排序过程中只需读取部分日排序数据和少量重载数据,有效解决了传统处理方案中冗余处理问题。与传统的数据处理方法做测试对比,结果表明:日采样数据较小时,性能提升3倍以上;日采样数据超过2880时,性能提升15倍以上。数据量越大,性能提升越明显。所提方案已在山西、河北等监测系统中成功应用,实践证明所提方案正确、有效。

The monitoring data in the power quality monitoring system increased quickly. A new method based on partial storage and selective reloading was proposed,which can solve the problem of repetitive sorting and redundant processing in the tradition methods. In the calculation of the daily index,daily data was sorted and stored partly based on saving rate. In the calculation of week（ month,season or year） index,the partly saved daily data in a week（ month,season or year） were merged by the multiple merge algorithm to calculate a temporary 95 percentile（ CP95）,which could be used to determine which daily data should be reloaded. Besides the reloaded data,all other needed data were reordered to calculate the steady index. The sorting process only needed part of the stored daily data and a small amount of reloaded data,so the redundant processing problem in traditional processing method was solved effectively. Compared with the traditional data processing method,the experimental results show the efficiency can be increased more than 3 times using the proposed method when daily sampling data is relatively small. When the number of daily sampling data is more than 2 880,the efficiency can be increased more than 15 times. The larger the amount of sampling data is,the more obviously the performance improves. The method has been applied in the monitoring system of Shanxi,Hebei and other provinces successfully. It is proved in practice that the method is correct and effective.