摘要
本文根据水文数据的特点探讨了水文大数据标准化方法,探索数据预处理、数据索引、数据高效存储等水文大数据共享平台关键技术;利用Hadoop对多源异构数据的海量存储能力及高速计算能力,研究基于MapReduce的水文大数据分布式数据处理模型,设计和实现水文大数据共享平台,为水利及跨行业跨部门的信息共享、空间集成,以及跨学科的可持续发展研究提供技术支撑。
In this paper, the methodology of hydrological big data standardization is discussed upon analyzing on the characteristics of hydrological data. Solutions on data preprocessing, data indexing and highly efficient data reading and writing are also introduced. The mass storage capacity and high speed computing capability of Hadoop are utilized for designing and implementing hydrological big data sharing platform. Accordingly, the platform can be technical support for information sharing and space integration between water conservancy industry and other industries, as well as the interdisciplinary sustainable development.
出处
《水资源研究》
2018年第1期10-18,共9页
Journal of Water Resources Research
基金
国家自然科学基金重点项目(51539009)
