基于一致性哈希的分布式数据库性能拓展

Performance development of distributed database based on consistent hashing

当对分布式数据库进行设计开发时,通常需要考虑网络设备的异质性和系统内网络之间的可用程度和互操作程度,以及系统网络的规模和开发成本等多个因素。目前一般的分布式数据库通常被设计成一系列网络构成的多层专用网络系统,当衡量网络规模和可用程度等问题时,需要基于每个问题进行单独的设计,由于这种方案缺乏统一的解决途径,导致管理和测试的效率降低。针对传统方式引起的多次设计问题,提出一种新的一致性哈希策略来进行分布式数据库性能动态拓展的方法。首先,通过一个中心管理节点群来控制所有节点的通信过程,并完成处理故障等任务,其中每一个节点均对应一类网络管理方式及协议方式;然后,对相应的异构网络设备进行监控和统计;最后,要保证节点之间则采用统一的协议进行通信。在内网环境下对函数误差、丢失信息量和时延进行评估,测试结果表明这些指标保持稳定,与多层分布式专用网络方法相比,新的方法在节点个数增加接近8倍的情况下,平均误差仅升高了74%,丢失数据量增加了52%,往返时延在毫秒量级上基本没有变化,因此认为这种方案比传统方法能够在保持性能的前提下容纳更多节点的通信进程,表现出一定的易扩展性,且由于对异构设备兼容,使得设计及架设的代价较低。

When designing and developing the distributed database, people usually need to consider the heterogeneity of the network equipment, the availability/interoperability between the network, the size of the system and the development cost. But the general distributed database management system is usually designed as a series of special management system. When measuring the size and availability of the network, it is necessary to design a separate system for each problem. The efficiency of test management reduces for the lack of unified solution. Aiming to the multiple design problems caused by the traditional methods, a new distributed database management system based on consistent hashing strategy was proposed. Firstly, a central management node group was set to control the communication process of all nodes and complete the tasks such as processing failure. Secondly, each node corresponded to a class of network management and protocol, and the corresponding heterogeneous network devices to monitor and statistics. Finally, all nodes used a unified protocol to guarantee their communications. Under inside network conditions, the testing results of evaluation on error function, loss of information and time delay show that these indexes were stable in the case of large proportion of nodes. Compared with the traditional muhilayer distributed network method, the error of new method only increased 74% on average, the missing data increased 52% and the round-trip delay in milliseconds dis not change when the number of nodes increased in a large proportion of nearly 8 times. So it is concluded that this scheme can accommodate more nodes in the premise of maintaining the performance than the traditional method. It shows some expansibility. For the compatibility of heterogeneous equipment, the design and erection has the lower cost.