低质量数据下的知识获取方案研究

Knowledge Acquisition under the Low-quality Data

数据挖掘已经成为重要的知识来源。随着信息化的深入,企业对数据挖掘有了较大的需求,但数据质量差往往会造成数据挖掘的结论可信度低,从而影响了数据挖掘的应用。本文以可拓学方法,通过系统分析产生脏数据的原因,提出了基于学科链方法的数据清洗方案。通过数据挖掘手段发现数据质量差距,追溯到软件设计、实施、维护等上游环节,提出从来源环节解决问题的系统方法,并提出通过数据挖掘应用咨询促使数据质量改进的措施,从而降低数据清洗的工作量,使数据质量不高的企业也可以实施数据挖掘项目,获取决策支持所需的知识。

The demand for data mining as a new knowledge source has been increasing in enterprise with the development of information system. However, the low-quality data tends to lead to the low credible conclusion on data mining, which affects the application of the data mining. Through the analysis on the reasons based on extension theory, the paper has established a new method called data mining consulting to solve the data quality problem by metasynthesis method including software designing, management and data mining testing, etc. Its application in a web company shows that it has good practicality which can carry out data mining project in low quality data enterprises and increase the data quality.