基于Hadoop的重叠基因对数量的计算方法

A Method for Calculating the Number of Overlapped Gene Pairs Based on Hadoop

基因组间的相似性可以通过重叠基因对的数量进行度量,基因组数据量的指数增长对重叠基因对数量的计算方法提出了更高的要求,业界迫切需要一种能够处理各种数量级别的基因数据库和待查询基因组的计算工具.本文提出一种基于Hadoop的重叠基因对数量的计算方法--DBits.在较小的数据集上,DBits的计算时间少于业界最常用的方法BITS,而在BITS无法处理的海量数据集上,DBits可以进行有效计算.DBits充分发挥分布式计算在大数据处理中的优势,可用于对数据量较大的基因组进行分析,有效地解决了现有技术中存在的重叠基因对数量计算难的问题.

The similarity between genomes can be measured by using the number of overlapping genes pairs. With the exponential growth of genome data,it calls for high-efficiency computational methods to calculate overlapping gene pairs,which is able to process big data on various genome databases. This paper proposes a novel method,DBits,to count the number of overlapping gene pairs based on Hadoop. Comparing to the most common used method,BITS,on the smaller data set,DBits demonstrates greater performance and runs faster. Besides,it is able to run on big data which BITS is unable to achieve results. Hence,DBits not only has the advantage of distributed computational capabilities for big data,but also it is a state-of-art method to effectively calculate overlapping genes pairs.