基于Tree-lib的大数据实时分析研究

Real Time Analytics Study of Big Data Based on Tree-lib

为提高大数据的存储和并行处理能力,建立了以列存储Infobright与分布式MySQL Cluster为核心的大数据实时并发分析、管理模式,以完成对开源brighthouse引擎的二次开发。利用管理程序Tree-lib对分布式大数据进行可视化监控、维护和管理。实验结果表明,Infobright和Cluster组合具备对大数据的高压缩存储、多并发查询和高效实时分析的能力,Tree-lib完成对树和库的生成、检测、更新、备份和灾难恢复等,实现可视化双向管理和维护的目的。

In order to improve the storage and parallel processing capabilities of big data, the big data of real time con- current analytics and management mode were built with the center of column stored Infobright and distributed MySQL Cluster, to complete to secondary develop of the open source brighthouse engine. The managing procedure Tree-lib was used to visual monitor,maintain and manage the distributed big data. The results of experiments show the combination of Infobright and Cluster have the capability of high compression storage and multiple concurrent inquiries and the effi- cient real time analysis with big data. The Tree-lib accomplishes generation, detection, update, backup and disaster re- cover of tree and library. Finally, the purpose of visual bidirectional management and maintenance is achieved.