面向WDLC的数据质量智能监控研究及应用

Research and Application of Data Quality Intelligent Monitoring for WDLC

数据质量监控是数据质量管理的重要手段,应面向数据全生命周期(TheWholeDateLifeCycle,WDLC)中每一环节,但传统数据质量监控仅聚焦于具体数据环节或业务场景,其监控广度有待提升。针对此问题,结合全生命周期中各数据环节的特点,提出面向数据全生命周期的数据质量智能化监控模型。为了规范数据质量监控过程,从数据的采集、接入、集成、处理、使用、归档等环节建立全面的数据质量监控流程;为了提升数据质量监控自动化和智能化水平,利用人工智能算法建立数据质量智能化检查方法,实现全生命周期的数据质量智能化监控。通过研发面向公共安全的数据质量监控分析平台,验证了该模型的可行性和有效性。

Data quality monitoring is an important means of data quality management.It should be oriented to every link in the whole data life cycle(WDLC).However,traditional data quality monitoring focuses only on specific data links or business sce⁃narios,and its monitoring breadth needs to be improved.To solve this problem,an intelligent data quality monitoring model for WDLC is presented,which combines the characteristics of each data link.In order to standardize the process of data quality moni⁃toring,a comprehensive data quality evaluation process is established from data collection,access,integration,processing,use and archiving.To improve the automation and intelligence level of data quality monitoring,artificial intelligence algorithm is used to es⁃tablish the data quality inspection method,and the WDLC data quality intelligent monitoring is realized.By developing a public security-oriented data quality monitoring and analysis platform,the model is proved to be feasible and effective.