基于Storm架构的电力物联网流数据处理

Stream data processing of the power internet of things based on Storm architecture

数据的采集、分析和处理是目前电力系统重点关注的问题之一。电力物联网系统中数据业务处理需求多样,尤其是流数据的处理对时延要求严格。然而,现有的数据处理方法形式单一,不能很好地满足低时延处理要求。因此,提出了基于Storm架构的电力物联网流数据处理方法。首先,基于Storm拓扑结构提出分布式流数据处理框架。进而采用流水线式的处理方式,从而达到缩短处理时间的效果。在数据接入后,采用循环队列和流转算子的方法。然后再通过边缘计算处理海量的数据,实现高效的协同工作。此外,通过支持向量机预测算法预测数据的发展趋势,采用清洗技术对脏数据进行处理,解决数据传输过程中的污染情况。仿真结果表明,与传统数据处理方法相比,所提流数据处理方法大大缩短了数据处理时间,满足了大量流数据处理的需求。

Data collection,analysis and processing is one of the most important problems in power systems.There are various requirements for data business processing in the power internet of things(IoT)systems.In particular,there are strict requirements on delay for stream data processing.However,the existing data processing methods are single in form and cannot well meet the requirements of low-delay processing.Therefore,a stream data processing method of power IoT based on Storm architecture is proposed.First,a distributed stream data processing framework is proposed based on Storm topology.Then the pipeline-based processing method is adopted to shorten the processing time.Then data access,a circular queue and a flow operator are adopted.Then,through the edge computing processing of massive data,efficient collaborative work is achieved.In addition,the support vector machine prediction algorithm is used to predict the development trend of data,and the dirty data is processed by cleaning technology to solve the pollution in the process of data transmission.The simulation results show that,compared with the traditional data processing methods,the proposed method can greatly shorten data processing time and meet the needs of processing a large number of data streams.