Apache IoTDB的分布式框架设计

The design of Apache IoTDB distributed framework

Apache IoTDB是一个新型的开源时序数据库管理系统.分布式数据管理系统不仅需要解决数据分区与多副本带来的节点间元数据同步问题,还要支持高效查询请求处理.本文针对节点间元数据同步问题提出了双层粒度元数据管理策略,在此基础上基于一致性哈希分区方法和Raft协议设计了同时支持强一致性查询和最终一致性查询的分布式框架.基于单机版Apache IoTDB进行了系统实现与实验测试,测试结果表明:双层粒度元数据管理策略与单层粒度管理策略相比,其元数据内存资源占用更少且写入性能提升5%~10%,并且分布式Apache IoTDB的读写性能随着集群规模的扩大而线性增长.

Apache Internet of Things Database(IoTDB)is a new open-source timeseries database management system.A distributed data management system not only needs to solve the problem of metadata synchronization between nodes caused by data partition and multiple replicas,but also needs to support efficient query request processing.To solve the problem of metadata synchronization among nodes,we propose a dual-layer granularity metadata management strategy.Based on the consistency hash partitioning method and Raft protocol,we designed a distributed framework that supports both strong consistency query and eventual consistency query.Based on the single-machine version of Apache IoTDB,we carried out the system implementation and experimental test.Compared with the single-level granularity management strategy,the test results showed that the two-level granularity metadata management strategy takes less memory resources and improves the write performance by5%~10%.Also,the results showed that the read and write performance of the distributed Apache IoTDB increases linearly with the extension of cluster size.