期刊文献+
任意字段
题名或关键词
题名
关键词
文摘
作者
第一作者
机构
刊名
分类号
参考文献
作者简介
基金资助
栏目信息

构建高维度数据立方体的有向图方法 被引量：1

A directed graph method for building high dimension data cube
下载PDF
导出
摘要 Hadoop、spark等软件框架为大数据的并行快速处理提供了技术支持,同时大数据环境也对OLAP提出了准实时和实时响应的要求。数据立方体是OLAP的多维数据模型抽象,大数据的多变性分析使数据立方体呈现高维特点,大数据的数据量也造成了数据立方体的膨胀。利用有向图描述数据立方体,可以为数据分析提供数据片和数据块的全集,通过提取全集中的某个元素,提高数据分析的效率。对高维度的数据立方体,采用降低维度的办法进行立方体规模的控制。根据各个维度的使用频度和方式,提出了可无维度、必须维度和联合维度的概念,并分别给出了各种维度的判断方法,实现了所涉及的数据立方体的调整简化方法。 Hadoop,spark and other software frameworks provide technical support for parallel and fast processing of big data.At the same time,the big data environment also puts forward the requirements of quasi-realtime and real-time response to OLAP.Data cube is the abstraction of multidimensional data model for OLAP.The variability analysis of large data makes the data cube high dimensional features,the amount of large data also causes the expansion of the data cube.A digraph is used to describe a data cube,which can provide a complete set of data pieces and data blocks for data analysis,and improve the efficiency of data analysis by extracting an element in the complete set.For a high dimension data cube,a dimension reduction method is used to control the size of the cube.Based the frequency and mode of using each dimension,the concepts of non-dimension,necessary dimension,and joint dimensions are proposed,the methods of judging all kinds of dimensions are given,and a simplified method of adjusting the data cube is implemented.
作者 张岩 吕梦儒 ZHANG Yan;LYU Mengru(Computer and Basic Mathematics Education Department,Shenyang Normal University,Shenyang 110034,China)
机构地区 沈阳师范大学计算机与数学基础教学部
出处 《沈阳师范大学学报（自然科学版）》 CAS 2018年第1期77-81,共5页 Journal of Shenyang Normal University:Natural Science Edition
基金 辽宁省自然科学基金资助项目(2015020055)
关键词 大数据 MAPREDUCE 数据立方体 有向图 OLAP Big-data MapReduce data-cube Digraph OLAP
分类号 TP311.13 [自动化与计算机技术—计算机软件与理论]
  • 相关文献

参考文献6

二级参考文献102

  • 1董新华,李瑞轩,周湾湾,王聪,薛正元,廖东杰.Hadoop系统性能优化与功能增强综述[J].计算机研究与发展,2013,50(S2):1-15. 被引量：70
  • 2Zhou MQ, Zhang R, Zeng DD, Qian WN, Zhou AY. Join optimization in the MapReduce environment for column-wise data store. In: Fang YF, Huang ZX, eds. Proc. of the SKG. Ningbo: IEEE Computer Society, 2010.97-104. [doi: 10.1109/SKG.2010.18].
  • 3Afrati FN, Ullman JD. Optimizing joins in a Map-Reduce environment. In: Manolescu I, Spaecapietra S, Teubner J, Kitsuregawa M, Leger A, Naumann F, Ailamaki A, Ozcan F, eds. Proc. of the EDBT. Lausanne: ACM Press, 2010. 99-110. [doi: 10.1145/ 1739041.1739056].
  • 4Sandholm T, Lai K. MapReduce optimization using regulated dynamic prioritization. In: Douceur JR, Greenberg AG, Bonald T, Nieh J, eds. Proc. of the SIGMETRICS. Seattle: ACM Press, 2009. 299-310. [doi: 10.1145/1555349.1555384].
  • 5Hoefler T, Lumsdaine A, Dongarra J. Towards; efficient MapReduce using MPI. In: Oster P, ed. Proc. of the EuroPVM/MPI. Berlin: Springer-Verlag, 2009. 240-249. [doi: 10.100'7/978-3-642-03770-2_30].
  • 6Nykiel T, Potamias M, Mishra C, Kollios G, Koudas N. MRShare: Sharing across multiple queries in MapReduce. PVLDB, 2010, 3(1-2):494-505.
  • 7Kambatla K, Rapolu N, Jagannathan S, Grama A. Asynchronous algorithms in MapReduce. In: Moreira JE, Matsuoka S, Pakin S, Cortes T, eds. Proc. of the CLUSTER. Crete: IEEE Press, 2010. 245-254. [doi: 10.1109/CLUSTER.2010.30].
  • 8Polo J, Carrera D, Becerra Y, Torres J, Ayguad6 E, Steinder M, Whalley I. Performance-Driven task co-scheduling for MapReduce environments. In: Tonouchi T, Kim MS, eds. Proc. of the 1EEE Network Operations and Management Symp. (NOMS). Osaka: IEEE Press, 2010. 373-380. [doi: 10.1109/NOMS.2010.5488494].
  • 9Zaharia M, Konwinski A, Joseph AD, Katz R, Stoica I. Improving MapReduce performance in heterogeneous environments. In: Draves R, van Renesse R, eds. Proc. of the ODSI. Berkeley: USENIX Association, 2008.29-42.
  • 10Xie J, Yin S, Ruan XJ, Ding ZY, Tian Y, Majors J, Manzanares A, Qin X. Improving MapReduce performance through data placement in heterogeneous Hadoop clusters. In: Taufer M, Rfinger G, Du ZH, eds. Proc. of the Workshop on Heterogeneity in Computing (IPDPS 2010). Atlanta: IEEE Press, 2010. 1-9. [doi: 10.1109/IPDPSW.2010.5470880].

共引文献396

同被引文献9

引证文献1

二级引证文献2

沈阳师范大学学报（自然科学版）
2018年 第1期

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部