基于influxDB的工业时序数据库引擎设计

DESIGN OF INDUSTRIAL TIME SERIES DATABASE ENGINE BASED ON INFLUXDB

工业时序数据具有测点多、采样频率快、读取性能要求高等特点。influxDB提出时间结构合并树TSM,解决了数据的读写性能优化问题,并针对整数、浮点数、布尔、字符串、时间五种数据类型采用不同的压缩算法。它在读写性能、存储空间占用方面取得了较好的效果,在开源时序数据库软件中排名第一。针对influxDB元数据结构相对于工业时序数据过于复杂的问题,提出简化后的TSM文件结构,开发工业时序数据库的引擎,并进行了读写性能测试。结果显示,该引擎一次5万点整型数据写入平均耗时约310 ms,读取1 000点共计100万条数据耗时约626 ms,并且具有很大的性能提升潜力。

Industrial time series data have many characteristics, such as many measuring points, fast sampling frequency and high requirement of reading performance. The TSM proposed by influxDB solves the problem of data read-write performance optimization. Different compression algorithms are adopted for five data types: integer, floating point, Boolean, string and time. It has achieved good results in reading and writing performance and storage space occupancy, ranking first in open source sequential database software. Aiming at the problem that influxDB metadata structure is too complex compared with industrial time series data, this paper proposed a simplified TSM file structure, developed an engine of industrial time series database, and tested its read-write performance. The results show that the engine takes about 310 ms to write 50 000 integer data at a time, and about 626 ms to read 1 million data at 1 000 points, and has great potential for performance improvement.