内存数据仓库集群技术研究

Research on in-memory data warehouse cluster technologies

随着硬件的集成度不断提高,多核处理器和大内存成为当前主流的计算平台,内存计算也成为新兴的高性能数据分析平台.内存数据仓库集群技术面向高性能分析计算,是实现大数据实时分析的基础平台.本文概括地介绍了中国人民大学高性能数据库团队在内存数据仓库集群技术方面的研究工作,包括:以列分布和列计算服务为中心的ScaMMDB内存数据仓库集群,以水平分片、并行计算为中心的ScaMMDBⅡ和reverse-star schema分布、集群向量计算为特征的MiNT-OLAP Cluster等技术的研究发展过程.分析了内存数据仓库集群技术的关键问题及技术挑战,并针对新的内存数据仓库集群应用需求展望未来技术的发展.

With the development of hardware integration techniques, multicore processor andbig memory come to be main stream configuration and in-memory computing comes to be the e-merging high performance data analytical platform. In-memory data warehouse cluster technolo-gies target high performance analytical computing, and it will be the basic platform for big datareal-time analytical processing. This paper briefly introduces the research work on in-memory da-ta warehouse cluster of Renmin University high performance database research group, includingthe developments of column distribution and column computing service oriented ScaMMDB clus-ter, horizontal partition and parallel computing oriented ScaMMDBII, and reverse-star schemadistribution and cluster vector computing oriented MiNT-OLAPCluster technologies. The criticalissues and technical challenges are also presented in this paper. Finally, we give a prospectivediscussion on future technologies for the coming in-memory data warehouse cluster requirements.