一种有效的XML数据清洗方法

Efficient Cleaning Approach for XML Data

研究XML格式的重复数据元素的特点,提出对于特定应用领域,在具体的上下文环境中主动学习XML重复元素的识别规则。通过结构转换,将结构不尽相同的XML数据映射成结构一致的数据,并通过学习不同层次数据元素间的依赖关系权重来获得匹配规则。根据学习得到的转换和匹配规则,采用哈希过滤的方法来提高检测重复XML元素的效率。该方法能够有效地解决XML重复检测面临的结构多样性的问题,理论分析和实验表明,该方法有较高的精度和效率。

By studying characteristics of duplicate XML data, this paper proposes an active machine learning method for a specific application, which is applied to glean transformation rules and matching rules, and accurately identify duplicate XML elements. Transfomation rules are used to eliminate the structural diversities among elements and matching rules are used to identify the relationships between parent and child nodes. In turn, during the detection phase an efficient hash filter algorithm is proposed to reduce computational complexity. Theory and experiment shows that the method can solve this problem efficiently and effectively.