一种MySQL到HBase的迁移策略的研究与实现

Study and realisation on the migration strategy from MySQL to HBase

随着Web2.0的到来,互联网数据快速增长。大规模数据的采集和处理及应用直接影响着用户体验,决定着企业的发展。比较传统关系型数据库和分布式非关系型数据库两者处理大数据的性能,将本地数据迁移到分布式数据库势在必行。文章分析了现有迁移工具的利弊,基于HBase数据库提出了一种有效的数据迁移策略,并依据提出的策略实现了一种半自动化迁移工具。以美国城市和方言系统CityDetail数据库数据为例,阐述了该迁移工具的工作原理并对迁移后的数据进行多次查询比较,证明了使用该工具进行数据迁移的高效性。

With the coming of Web2.0,the data of Internet increases rapidly.The user experience is directly influenced by collection and processing of large scale data,which determines the development of the enterprise.Comparing RDBMS and distributed database in processing big data,it＇s necessary to migrate data source from RDBMS to distributed database.This paper analyses the advantages and disadvantages of the existing migration tools,proposes an effective data migration strategy based on the HBase and implements a semi-automated migration tool.Finally,this paper takes the City Detail database system as an example,shows the tool＇s operating principles and proves the high effects of this tool by the test result.