基于分片一致性哈希负载均衡策略与应用

A Consistent Hashing Load Balancing Strategy Based on Fragmentation and Its Application

采用一致性哈希进行数据分区和负载均衡的分布式键值存储系统具有高可扩展性的特点,但一致性哈希中哈希函数静态负载均衡的特性不能满足日益多样化的应用场景需求。为了适应以上需求,从一致性哈希策略出发,结合动态负载均衡技术,设计了一种基于一致性哈希的动态负载均衡策略。该策略使用与物理节点解耦的分片代替传统的虚拟节点,并利用针对分片的监控信息,从分片级和节点级两个层面对系统负载进行均衡调度,通过更细的调度粒度优化均衡效果。实验结果表明,该策略保留了一致性哈希策略在系统扩展性上的优势,同时优化了一致性哈希策略负载均衡的总体效果。利用基于分片的一致性哈希负载均衡策略,可以有效地均衡系统负载,提高存储系统的效率。

The distributed key-value storage system, which uses consistent hashing for data partitioning and load balancing, has high ex- pansibility. However, the static load balancing strategy in consistent hashing cannot meet the increasingly diverse needs of application. In order to adapt to above needs, a dynamic load balancing strategy is designed based on the consistent hashing and combined with dynamic load balancing. It adopts the fragment decoupled physical nodes instead of traditional virtual nodes and uses the monitoring information of fragments to make decisions for load balancing scheduling from two aspects of fragment level and node level. Experimental results show that it has retained the advantage of consistent hashing strategy in system scalability,while optimizing the overall performance of consistent hashing load balancing. The system load can be effectively balanced and the utilization of the system can be improved.