摘要
在"辽河流域水环境管理技术综合示范"项目中,随着时间的累积,环境监测数据处理系统采集到的数据量越来越大.然而目前辽宁省环境监测数据处理系统无法有效处理日益增长的海量数据.研究运用大数据技术,改进环境监测数据处理系统中的数据中心.利用HDFS强大的数据存储、管理功能,以应对数据量的增长,利用Map Reduce及Hadoop其他相关工具,快速处理海量数据,降低数据规模,最后将数据存储到数据库中.
With the accumulation of time, data collected by environmental monitoring data processing system is more in the project of ＂Comprehensive Demonstration of Water Environment Management about Liao He River＂. However, Liaoning province environmental monitoring data processing system cannot effectively deal with massive data growing. In this paper, we study and use big data technology. After that, we improve the data center of the environmental monitoring data processing system. We can use HDFS powerful function of data storage and management to cope with the growth of the massive data and take advantage of MapReduce and other Hadoop related tools, rapidly processing massive data and reducing the data size. Finally, the data will be stored in the database.
出处
《计算机系统应用》
2015年第1期60-64,共5页
Computer Systems & Applications
基金
国家水体污染控制与治理科技重大专项(2012ZX07505003)
