基于无共享架构的海量感知数据实时处理系统

A Real-Time Processing System for Massive Sensing Data

为了满足具有海量性、连续性及不确定性的感知数据实时处理需求,采用云计算典型的分布式无共享集群架构,文中提出一种并行化的海量感知数据实时处理模型,并给出了相应的编程接口.在此基础上设计了一种去中心化的分布式感知数据实时处理系统架构以及基于ZooKeeper的集群伸缩管理方案,从而保证了感知数据处理系统的实时性及扩展性.通过一个结合城市车辆监管实际应用的实验,验证了该系统在负载均衡的情况下,其处理性能够随着计算节点的增加而接近线性增长.

With the development of Internet of Things, the sensing data gathered by large amounts of sensors shows the massive, continuous and probabilistic characteristics. In order to satisfy the requirements of real-time processing of such sensing data, the share-nothing architecture of cloud computing is adopted to support the parallel sensing data processing with extendibility, and a parallel computing model and corresponding programing interface for real- time sensing data processing is proposed by extending the MapReduce. Furthermore, a decentralized distributed architecture and cluster management method are designed to implement the parallel computing model. The experiment shows that the system has good scalability and the processing performance increases in linear progression as the number of server increases.