基于增量学习的RocksDB键值系统主动缓存机制

Incremental learning based proactive caching mechanism for RocksDB key-value system

由于分层结构的约束,基于日志结构合并(LSM)树的RocksDB键值存储系统面临着读取性能低下的问题。一种有效的解决方法是对热点数据进行主动缓存,但其面临两个挑战:一是如何在数据分布持续动态变化时对热点数据进行预测,二是如何将主动缓存机制与RocksDB存储结构衔接起来。针对这些挑战,基于预测分析技术,构建了由数据采集、系统交互、系统测试等部分组成的面向RocksDB键值系统的主动缓存框架,能够将热点数据缓存在LSM树的较低层级中;并对数据访问模式进行建模,设计并实现了基于增量学习的热点数据预测分析方法,能够有效减少存储介质的I/O访问次数。实验结果表明该机制能有效提升RocksDB在不同动态工作负载下的数据读取性能。

RocksDB key-value storage system based on Log-Structured Merge(LSM)tree has the problem of low read performance caused by the constraints of its hierarchical structure.One effective solution is to cache hot spot data proactively,but it faces two challenges.One is how to predict the hot spot data when the data distribution keeps on changing constantly,the other is how to integrate the proactive caching mechanism with the RocksDB storage structure.To tackle these challenges,a proactive caching framework for RocksDB key-value system with multiple components including data collection,system interaction and system evaluation was built,which can cache the hot spot data at the low levels of the LSM tree.And with the modeling of data access patterns,an incremental learning based prediction analysis method for hot spot data was designed and implemented,which can reduce the number of I/O operations of storage medium.Experimental results show that the proposed mechanism can effectively improve the read performance of RocksDB under different dynamic workloads.