基于IndexR的海量数据快速实时检索技术的研究与实现

Research and realization of fast real-time retrieval technology of massive data based on IndexR

由于强调“实时”、“快速”,传统的海量数据检索问题已经变成了一个新的RTOLAP问题。为解决这个传统数据库无能为力的问题,分析比较现有技术后,介绍了基于HDFS的分布式关系型列式数据库IndexR,并提出了一个以IndexR为核心的海量数据快速实时检索解决方案,为复杂信息系统处理海量数据提供了框架性的服务。之后分别与以Hive、MySQL为核心的框架从多个维度进行了对比,说明了该框架在处理RTOLAP场景时相对于传统的OLAP和OLTP框架的优越性,验证了该框架的可行性。

Because of the emphasis of“real-time”and“fast”,the massive data retrieval problem has becoming a new RTOLAP problem.However,the traditional database is not able to figure it out.After analyzing and comparing the existing technology,the distributed relational database which bases on HDFS has been introduced to solve the problem.Meanwhile,a solution of a fast real-time retrieval of massive data which regards the IndexR as the core has also been inferred.Both provide the framework of serving the complex information system which aims at processing massive data.Afterwards,compared with Hive and MySQL as the core framework from multiple dimensions.This shows that the framework is superior to the traditional OLAP and OLTP framework when dealing with RTOLAP scenarios,and verifies the feasibility of the framework.