基于ElasticSearch的异构数据迁移方法

A Method for Heterogeneous Data Migration Based on ElasticSearch

随着信息技术应用创新和信息系统集约化进程的不断推进,气象部门政务管理信息系统正加速实施国产化自主可控迭代升级,其对历史数据迁移的平滑与否是衡量升级成败的关键因素之一。借助分布式全文检索框架ElasticSearch的动态映射、分布式可扩展以及非结构化数据处理等能力,本文提出一种异构数据迁移方法,能够解决传统方法在系统替换场景中存在的不足。应用该方法对中国气象局电子公文系统进行全国产化替代,取得了良好的效果,不仅满足了历史数据向国产化自主可控生态下的物理迁移需求,还能支持新的系统对历史数据更加灵活的二次使用。

With the development of information technology application innovation and the integrative process of information system,the government management information system of the China Meteorological Administration is accelerating the implementation of nationalization independent and controllable upgrade iteration.Whether the historical data migration can be smoothly realized becomes a pivotal factor in measuring the success or failure of upgrading.This paper proposes a heterogeneous data migration method based on ElasticSearch,a distributed full-text retrieval framework.Using ElasticSearch’s characteristics of dynamic mapping,distributed scalability and unstructured data processing capability,it can crack the limitations of using traditional methods to migrate massive heterogeneous data.This method has been applied to the national industrialized electronic document system of the China Meteorological Administration and has achieved good practical results.It meets the physical migration needs of historical data to the localization independent and controllable ecology and supports the more flexible secondary use of historical data in the new system.