一种基于插件化异构数据的ETL中间件设计与实现

Design and Implementation of ETL Middleware based on Plug-In Heterogeneous Data

针对在数据仓库的建设中业界端对端的数据迁移工具无法能满足数据库高可扩展性需求,基于数据ETL中间件技术和数据抽取CDC技术分析选型,设计实现一种基于插件化的异构数据ETL中间件,重点介绍其包括业务解析层与数据处理引擎层的总体架构,从而实现数据增量迁移需求,对工业数据仓库建设具有借鉴意义。

Focus on the end-to-end data migration tools in the industry cannot meet the high scalability requirements of the database in the construction of the data warehouse,based on the data ETL middleware technology and data extraction CDC technology analysis and selection,a plug-in-based heterogeneous data is designed and implemented.ETL middleware,focusing on its overall architecture including business analysis layer and data processing engine layer,so as to realize the requirement of incremental data migration,which has reference significance for industrial data warehouse.