低开销的分布式数据库负载均衡技术

Low overhead load balancing technique for distributed database

针对现有分布式数据库的服务器间负载不均的问题,提出了一种基于数据局部性的两层负载均衡方法。该方法根据数据访问的区域局部性将节点均匀地划分为若干子分块,并由子负载均衡器统筹子块中的负载均衡分配,最终通过中央负载进行全局统筹。该方法一方面利用了数据可能的局部性,另一方面避免了只有一个总负载均衡器的结构中,负载均衡器消息通信拥塞的情况。同时,提出一种低开销的负载监控方法,节点以懒惰的方式向负载均衡器发送自己的负载状态变化情况,可以有效地降低系统通信开销和负载均衡器的队列长度。在此基础上提出一种懒惰负载迁移方法,该方法将数据访问与负载转移相结合,通过跨节点访问的方式间接地进行了负载迁移,通过将两种操作重叠来减少系统的额外开销。基于复杂query算法的Q22测试结果显示该方法在不同线程和方法下的平均响应时间变化不超过10 s,表明此方法能够有效减少负载均衡过程中产生的额外开销,保证系统的高效运行。

Aiming at uneven load between servers in distributed database, a two layer load balancing strategy based on data locality was proposed. The nodes were divided uniformly into several subblocks according to regional locality of data access. A sub-load equalizer was used for load balance, eventually the central load equilizer was selected to complete global co-ordination. The local property of the data was utilized to avoid the congestion of the load equalizer message communication with one total load equalizer. At the same time, a low overhead load monitoring method was presented, in which the node was changed to send its load status to load equalizer in a lazy way, thus communication overhead and load equalizer of the queue length could be effectively reduced. A lazy load transfer method was proposed based on this strategy, combing data access and load transfer. Through accessing to the inter node indirectly the load transfer, the two operations were overlapped to reduce system overhead. The experimental results of Q22 test based on complex query algorithm shows that the proposed method has the average response time of no more than 10 seconds under different threads and methods, which can reduce the overhead of load balancing and ensure the system efficiency.