并行化的Apriori算法在海量医疗文档数据挖掘中的应用及优化

Optimization and application of Apriori algorithm based on MapReduce in medical big data

针对海量医疗文档数据中巨大潜在价值难以有效挖掘的现状,构建了基于NoSQL和MapReduce的存储与挖掘系统MSPM.通过以键值对形式存储,使复杂异构的医疗文档数据归结为统一的且适于被经典Apriori算法利用的事务数据格式,并通过挖掘MapReduce过程化,一次性全局扫描和兴趣集规约计数等优化策略,有效解决了Apriori算法在医疗大数据应用中开销大、执行速度慢和有效性差的问题.

To solve the problem that values hidden in big medical data cannot be properly mined, an MSPM system based on NoSQL and MapReduce is proposed. By key-value storage, complex and heterogeneous data are summed up in a unified and convenient format of transaction for Apriori. With MapReduce, complete global scanning and interest set counting solved the problem of low speed, high overhead and poor effectiveness of Apriori algorithm in its application to medical data mining.