基于压缩的海量不完整数据近似查询方法

A Compression-Based Approximate Query Method for Massive Incomplete Data

随着数据的爆炸式增加,不完整数据普遍存在,传统的数据修复方法对于海量数据处理代价过高,且不能彻底修复,在这些不完整的海量数据上进行满足给定需求的近似查询引起了学术界的关注.因此,提出一种基于压缩的海量不完整数据近似查询方法,该方法对属性值缺失字段进行标记,根据频繁查询条件对标记后的数据进行压缩,并建立对应索引;根据属性划分对索引文件再次压缩以节省存储空间,采用编码字典对索引压缩文件进行选择和投影操作,最终获得不完整数据的近似查询结果.实验表明,该方法能够快速定位不完整数据的压缩位置,提高了查询效率,节省了存储空间,并且保证了查询结果的完整性.

With the explosive increase of data,incomplete data are widespread.Traditional methods of data repair will cause high processing cost for mass data,and cannot be fully restored.Thus the approximate querying on these huge amounts of incomplete data for meeting the given requirements attracted greater attention from academics.Therefore,this paper proposes an approximate query method for massive incomplete data based on compression.Tagging the missing attribute value field and finding out the frequent query conditions,this method compresses these data based on the statistical frequent query conditions,and establishes the corresponding indexes.According to the attribute partition rules,index files are compressed again in order to further save storage space.In the stage of query,this method uses encoding dictionary to make selection and projection operations on the index compression files for getting approximate query results of incomplete data in the end.Experimental results show that this method can quickly locate the position of incomplete data compression,improve the query efficiency,save the storage space,and ensure the integrity of the query results.