摘要
云计算环境下的监控系统会实时产生大量监控数据,如何在大数据的环境下实现对监控数据的高效存储和处理尤为重要。针对这一问题提出一种基于Hadoop的监控数据存储与处理的方案。该方案采用HBase数据库存储时序监控数据,并用提升字段法的宽表存储模型改进HBase数据库提升监控数据的存储效率;针对流量数据,采用MapReduce进行分布式计算处理提高处理效率。经过实验测试,验证了该方案的科学性和有效性,提高了海量监控数据下监控系统数据处理速度,解决了云计算环境下监控数据的计算瓶颈问题。
The monitoring system in the cloud computing environment will generate a large amount of monitoring data in real time. How to effectively store and process the monitoring data in the environment of big data is particularly important. To solve this problem,a scheme based on Hadoop＇s monitoring data storage and processing was proposed.The scheme used HBase database storage timing monitoring data,and enhanced the HBase database to improve the storage efficiency of monitoring data by using the wide table storage model of the improved field method. For traffic data,MapReduce was used for distributed computing to improve processing efficiency. After experimental tests, the scientificity and effectiveness of the scheme were verified,the data processing speed of the monitoring system under the massive monitoring data was improved,and the bottleneck problem of the monitoring data in the cloud computing environment was solved.
作者
池亚平
杨垠坦
许萍
杨建喜
Chi Yaping;Yang Yintan;Xu Ping;YangJianxi(Department of Communication Engineering ,Beijing Electronic Scienee and Technology Institute, Beijing 100070, China;School of Communication Engineering,Xidian University, Xi＇ an 710071, Shaanxi, China;Key Laboratory of Network Assessment Technology, Institute of Information Engineering, Chinese Academy of Scienees, Beijing 100093, China)
出处
《计算机应用与软件》
北大核心
2018年第6期58-63,157,共7页
Computer Applications and Software
基金
国家发改委信息安全专项(发改办高技[2015]289号)
国家高技术研究发展计划项目(2015AA017202)
