期刊文献+

水利普查成果数据立方体并行计算方法研究 被引量:1

Research on Water Census Data Cube Parallel Computing Method
下载PDF
导出
摘要 水利普查成果数据具有数据量大、维度多、维度分层等特点,因此物化水利普查成果数据立方体,所需的时间空间成本非常高。提出一种基于Map/Reduce计算模型进行外壳片段立方体并行计算的新方法。实验结果表明,该方法能够有效地提高在大数据集上计算外壳片段立方体的效率,降低物化水利普查成果数据立方体的时间空间成本。将水利普查成果数据立方体应用于多维分析系统,从多方面清晰直观地展现水利普查成果数据。 Water census data has the following characteristics: large amount of data, many dimensions and dimension hierarchy, etc. Chemical water census data cube, the time and space cost is very high. This paper puts forward a new method of parallel computing for shell fragments cube based on Map/Reduce calculation model. The experimental results indicate that the method can effectively improve the computation efficiency of shell fragments cube on large data sets, decrease the cost of the time and space of water census data cube. Water census data cube is applied to multidimensional analysis system. It clearly and intuitively shows water census data from several aspects.
出处 《水利信息化》 2015年第4期1-7,共7页 Water Resources Informatization
基金 水利部公益性行业科研专项经费项目(201501022)
关键词 水利普查 数据立方体 外壳片段立方体 Map/Reduce技术 water census data cube shell fragments cube technology of Map/Reduce
  • 相关文献

参考文献9

  • 1庞进武,程益联,罗志东.水利普查与信息化[J].水利信息化,2012(1):19-22. 被引量:15
  • 2Gray J, Chaudhuri S, Bosworth A, et al. Data cube: A Rel- ational Aggregation Operator Generalizing Group-by, Cross- tab, and Sub-totals[J]. Data Mining and Knowledge Discovery, 1997, 1 (1): 29-53.
  • 3Jiang F M, Pei J, Fu A W. Ix-cubes: Iceberg Cubes for Data Warehousing and Olap on Xml Data[C]//Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management. New York: ACM, 2007: 905-908.
  • 4Wang Z, Xu Y. Minimal Condensed Cube: Data Organization, Fast Computation, and Incremental Update[C]//Internet Computing in Science and Engineering, Harbin: International Conference on. IEEE, 2008: 60-67.
  • 5Li C, Cong G, Tung A K H, et al. Incremental Maintenance of Quotient Cube for Median[C]//Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Washington: ACM, 2004: 226-235.
  • 6朱凯,万定生,程习锋.水利普查成果分析中数据立方体计算研究[J].计算机与数字工程,2014,42(9):1591-1594. 被引量:3
  • 7Dean J, Ghemawat S. MapReduce: Simplified Data Proce- ssing on Large Clusters[J]. Communications of the ACM, 2008, 51 (1): 107-113.
  • 8Li X, Han J, Gonzalez H. High-dimensional OLAP: A Minimal Cubing Approach[C]//Proceedings of the Thirtieth International Conference on Very Large Data Bases-Volume 30. Urbana: VLDB Endowment, 2004: 528-539.
  • 9You J G, Xi J Q, Zhang P J, et al. A Parallel Algorithm for Closed Cube Computation[C]//Proceedings of the 7th IEEE/ ACIS International Conference on Computer and Information Science (ACIS-ICIS), Portland, Oregon, USA, 2008. Wash- ington: IEEE Computer Society, 2008: 95-99.

二级参考文献10

  • 1胡孔法,陈崚,顾颀,蔡俊杰,董逸生.数据仓库系统中一种高效的多维层次聚集算法[J].计算机集成制造系统,2007,13(1):196-201. 被引量:4
  • 2Gray J, Chaudhuri S, Bosworth A, et al. Data cube: A relational aggregation operator generalizing group-by, cross-tab, and sub-totals[J] Data Mining and Knowl- edge Discovery, 1997,1(1) : 29-53.
  • 3Jiang F M, Pei J, Fu A W. Ix-cubes: iceberg cubes for data warehousing and olap on xml data[C]//Proceed- ings o{ the sixteenth ACM con{erence on Conference on information and knowledge management. ACM, 2007 : 905-908.
  • 4Wang Z, Xu Y. Minimal Condensed Cube: Data Or- ganization, Fast Computation, and Incremental Update [C]//Internet Computing in Science and Engineering, 2008. ICICSE'08. International Conference on. IEEE, 2008: 60-67.
  • 5Xin D, Han J, Li X, et al. Computing iceberg cubes bytop-down and bottom-up integration: The starcubing approach[J]. Knowledge and Data Engineering, IEEE Transactions on, 2007,19 (1) : 111-126.
  • 6Li C, Cong G, Tung A K H, et al. Incremental main- tenance of quotient cube for median[C]//Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2004: 226-235.
  • 7Li X, Han J, Gonzalez H. High-dimensional OLAP: A minimal cubing approach[C]//Proceedings of the Thir- tieth international conference on Very large data bases- Volume 30. VLDB Endowment, 2004. 528-539.
  • 8中华人民共和国国务院.国务院关于开展第一次全国水利普查的通知(国发[2010]4号)[M]北京:国务院,20101.
  • 9中华人民共和国水利部.第一次全国水利普查实施方案(水规计[2010]498号)[M]北京:中华人民共和国水利部,20104-5.
  • 10占军,万定生,李宇.基于Oracle数据仓库的水利普查数据展现系统[J].计算机与数字工程,2012,40(10):55-57. 被引量:7

共引文献15

同被引文献5

引证文献1

二级引证文献6

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部