基于大数据的数据处理方法研究

Data Processing Method Based on Large Date

针对大数据处理效率低问题,该文提出了新的处理办法。其基本思想是利用预处理方法和历史查询结果作为中间结果集,通过对中间集的匹配减少重复处理时间,提高处理效率。最后通过仿真实验对比分析,表明新方法能够一定程度上提高数据处理效率。

Due to the low efficiency in large data processing, this paper proposes a new approach, that basic idea is using preprocessing methods and historical query results as an intermediate result set, matching by the middle set, to reduce the time on duplication processing and improve processing efficiency. Finally, simulation experiments comparative analysis shows that the new method can improve the efficiency of data processing.