大规模数据中心监控数据并发处理

Concurrent Processing of Monitoring Data in Large-scale Data Center

在大规模数据中心场景下,采集系统要实时并发处理大规模数据来保证监控数据的实时性。论文首先描述了大规模数据中心监控数据的规模,并指出监控数据的并发处理中存在的问题,然后针对问题提出关键技术和解决方案,进而基于上述研究结果进行架构设计。经过实验验证Storm应用于大规模数据中心监控数据的并发处理业务,可以保证监控数据的实时性,并且系统具备一定的扩展性。

In large-scale data center applications,in order to ensure real-time of business for the monitoring data,the mass data should be concurrently processed in real time in the data acquisition system.In this paper,at first,the scale of the monitoring data is explained,and the problems of concurrent processing for monitoring data are pointed out,and then the key technologies and solutions are proposed.At last,system architectures are designed based on the solutions.The experiments show that storm is efficient to guarantee the real-time of monitoring data used in concurrent data processing for the large-scale data center,and the system is scalable.