元数据驱动的大型数据库数据迁移工具实现

Implementation of Large Database Migration Tool Based on Metadata-driven

随着数据模型的升级等,企业需要将原来数据库中存储的数据迁移到新的数据库,一般的做法都是通过开发专用迁移软件来实现,而仅仅依靠数据库系统本身提供的工具是不能满足实际数据迁移需求的。通过对ETL模型、元数据驱动技术的研究,结合中国石油数据中心大型数据迁移需求,定义了数据迁移的通用框架模型体系。对其中的关键技术——映射模式的元数据表示进行了重点研究,用元数据对映射模式进行描述与存储。最后,采用元数据来驱动对ETL引擎进行功能实现。该技术与功能已在中国石油数据中心大型数据迁移中得到了应用。

With the upgrade of data model,companies need to migrate the original data stored in the database to the new database,the general practice is through the development of special software to achieve,but simply rely on the tools of database system provided can not meet the actual data migration needs.Through the study of ETL model and metadata-driven technology,in combination with the needs of China＇s oil data centers,large-scale data migration,defined the general framework of the data migration model system.The study was focus on the key technologies-the express of mapping schema,describe and storage with metadata.Finally,the use of metadata is drived the implementation of the ETL engine functions.The technical and functional has been applied in large-scale data migration of China＇s oil data centers.