基于动态融合LOF的城市污水处理过程数据清洗方法

Data-cleaning method based on dynamic fusion LOF for municipal wastewater treatment process

围绕城市污水处理过程数据存在连续噪声和缺失的问题,提出一种基于动态融合局部异常因子(dynamic fusion local outlier factor,DFLOF)的污水处理过程数据清洗方法.首先,设计一种基于滑动窗口的数据动态分段方法,通过计算每个子段数据的均值、最大值和峰值区间信息获得数据异常属性值;其次,建立一种基于DFLOF的数据可信度评价模型,利用基于动态融合局部异常因子算法评估数据的可信度,保证异常数据检测和剔除的准确率;最后,提出一种基于径向基函数(radial basis function,RBF)神经网络的数据补偿方法对缺失数据进行补偿,实现污水处理过程数据的清洗.将该数据清洗方法应用于实际污水处理过程,实验结果表明:基于动态融合局部异常因子的数据清洗方法能够实现污水处理过程中异常数据的清洗,从而提高数据质量.

In order to reduce the impact of continuous data noise and loss,a dynamic fusion local outlier factor(DFLOF)method is proposed for data-cleaning of the municipal wastewater treatment process(WWTP).First,a data dynamic segmentation method based on sliding window is designed to obtain the abnormal attribute of each segment,including mean value,maximum value and peak interval.Then,a data reliability evaluation model based on the DFLOF is established to evaluate each data segment by using the dynamic fusion local outlier factor algorithm,which improves the accuracy of abnormal data detection and elimination.Finally,a data compensation method based on radial basis function neural network is proposed to compensate the missing data and further realize the data-cleaning of the WWTP.The proposed cleaning method is applied to a real WWTP,the experimental results show that the data-cleaning method based on the dynamic fusion local outlier factor is able to clear abnormal data and improve the data quality.