基于远程直接内存访问的高性能键值存储系统

High performance key-value storage system based on remote direct memory access

随着数据与系统规模的不断扩大,网络传输成为了键值存储系统的性能瓶颈。同时,远程直接内存访问(RDMA)技术能够支持高带宽和低时延的数据传输,为键值存储系统设计提供了新的思路。结合高性能网络中的RDMA技术,设计并实现了高性能、低CPU负载的键值存储系统Chequer;结合RDMA原语的特性,重新设计了键值存储系统的基本操作工作流程;并设计了基于线性探测的共享hash表,解决客户端缓存失效的问题以及提高hash命中率来减少客户端的读取轮数,进一步提高了系统的性能。在小规模集群上实现了Chequer系统,并通过实验验证了其性能。

With the continuous increment of data and system size,network communication becomes a performance bottleneck of key-value storage systems.Meanwhile,Remote Direct Memory Access(RDMA)technique can support high bandwidth,low latency data transmission,which provides a new idea for designing key-value storage systems.Based on RDMA technique in the high performance network,a key-value storage system named Chequer with high performance and low CPU overhead was designed and implemented.By combining the characteristics of RDMA primitives,the basic operation workflow of key-value storage system was redesigned.And a linear probing based shared hash table was designed to reduce the number of client reading rounds by solving the problem of client cache invalidation as well as increasing the hash hit rate,which can further improve the performance of the system.The Chequer system was implemented on the small-scale cluster,and its performance was demonstrated by experiments.