一种用于大数据分析服务的启发式云爆发算法

A HEURISTIC CLOUD BURSTING ALGORITHM FOR BIG-DATA ANALYTICS SERVICES

基于异构云联合的并行化大数据分析服务可以提升性能。然而由于大数据网络传输存在较大时延，原则上必须在并行化水平和大数据分析性能之间进行折衷。鉴于此，提出一种启发式云爆发算法用于并行化大数据分析服务。首先确定联合云中哪些计算结点应该用于大数据分析并行处理，然后将大数据妥善地分配给这些计算结点，确保处理同步完成且性能最优，最后，确定被分配的不同大小数据块在各个结点的计算次序，确保数据块传输尽量在结点上一数据块计算期间完成。与其他负载均衡算法做了对比，结果表明，使用该算法后性能可提升20％～60％。

Parallelisation big-data analytics services over a federation of heterogeneous clouds are considered to improve the performance. However, principally there is an inherent trade-off between the level of parallelisation and the performance of big-data analytics because a quite significant delay exists when the big-data is transmitted over the network. In view of this, we propose a heuristic cloud bursting algorithm and apply it to parallelisation big-data analytics services. First, the algorithm determines which computing nodes in federated clouds should be used for parallel processing of the big-data analytics ; then it appropriately allocates the big-data to these computing nodes for ensuring the completion of the synchronised processing with best performance; finally, it determines the computation sequence of the allocated big-data chunks with different sizes in each node, so as to guarantee the transmission of a data chunk is to be completed within the computation period of its previous chunk in the node as much as possible. We have compared our algorithm with other load-balancing schemes. Result shows that by using this algorithm the performance can be improved by 20% and up to 60% against other approaches.