基于联邦学习的水文遥测数据异常识别与修复

Anomaly identification and repair of hydrological telemetry data based on federated learning

已有的数据修复主要是针对数据缺失问题,即利用变分自编码器学习相关分布,并生成能够表征缺失部分的内容,从而实现对数据的修复。然而这种方法仅作用于数据缺失,且无法对异常数据进行识别。同时,随着数据隐私问题不断暴露在公众视野里,这对传统方法再次发起了挑战。提出基于联邦学习框架及长短时记忆网络的生成对抗网络模型,即FedLGAN模型,以实现对水文遥测数据的异常识别与修复。在该模型中,通过生成对抗网络结构中的判别器鉴别真实数据与虚假数据的能力实现异常识别,通过生成器学习正常数据特征并生成的能力实现异常数据修复。此外,为了捕获原始数据的时序特征,将基于注意力机制的双向长短时记忆网络结构嵌入到生成对抗网络中。联邦学习的框架则避免了训练过程中水文遥测数据隐私的泄露。基于4个真实水文遥测设备的实验证明:FedLGAN模型能够在保护隐私的同时实现对异常数据的识别和修复。

Existing data repair methods are mainly aimed at the problem of missing data,that is,the variational autoencoder is used to learn the data distribution and generate content that can represent the missing part,so as to realize the repair of the data.However,this method only works on missing data and cannot identify abnormal data.At the same time,as data privacy issues continue to be exposed to the publice view,this challenges traditional approaches once again.A generative adversarial network model based on the federated learning framework and long-short-term memory network,namely the FedLGAN model,is proposed to realize the abnormal identification and repair of hydrological telemetry data.In this model,anomaly recognition is realized through the ability of the discriminator in the generative adversarial network structure to distinguish between real data and fake data,and the ability of the generator to learn and generate the characteristics of normal data realizes the repair of abnormal data.Furthermore,in order to capture the temporal features of the original data,an attention-based bidirectional long-short-term memory network structure is embedded into the generative adversarial network.The framework of federated learning can avoid the leakage of the privacy of hydrological telemetry data during the training process.The experimental verifications based on four real hydrological telemetry devices show:the FedLGAN model can realize the identification and repair of abnormal data while protecting privacy.