面向分布式数据流系统的可扩展负载平衡算法

A Scalable Load-Balancing Algorithm for Distributed Data Stream Management Systems

分布式数据流系统中,数据处理任务由多台计算机相互合作共同完成.当输入数据流速发生较大波动时,会引起部分节点过载,从而影响整个系统的数据处理能力.针对这一实际问题,文中分析了分布式数据流系统的特点以及现有负载平衡算法的局限性,提出了一种适用于由不同性能计算机组成数据流系统的,具有良好扩展性的负载平衡算法.实验结果表明了算法的有效性.

In distributed data stream processing systems, data streams are processed in a number of computers in a collaborative manner. Overload on a part of nodes caused by input data stream fluctuation may weaken data processing ability of the entire system. Aimed at this practical problem and based on the characteristics of distributed data stream processing system and shortcomings of existing load balancing methods, a scalable load-balancing algorithm is proposed. This algorithm can deal with the load-balancing requirement of data stream processing systems composed of computers with different processing ability. Numerical experiments show the effectiveness of the proposed algorithm.