基于简单规则的数据质量检查系统设计与应用

Design and Application of Data Quality Detection System Based on Simple Rules

为了更加全面地对数据存在的质量问题进行检查,并找出其中的问题数据,分析了数据质量评估的一般性指标,从规则约束的角度对关系型数据字段的格式、语法、长度、取值范围,以及字段与字段之间的逻辑关系、函数依赖关系等进行分类描述,设计相应的数据质量检查算法并进行编码实现,形成一套完整的数据质量检查工具。对某单位的设备人员信息数据从完整性、规范性、一致性、有效性等方面进行检查。实验结果表明,这些规则能够有效检出关系型数据中存在的问题。

In order to carry out the overall detection of data quality and locate the incorrect data,analyze the general indexes of data quali-ty assessment. The rules of structured data,which takes the form,grammar,length,range of single field and the logical,functional depend-ency relationship of different fields into account,was classified and programmed by using corresponding algorithm. Design a correspond-ing data quality inspection tools and realize it by programming,forming a set of whole data quality inspection tool. Test on the data of fa-cility and staff information from the angle of integrity,normalization,consistence,effectiveness. The result of experiment shows that such data rules can find out the errors of the data.