配电网准实时数据的倒排二级索引集群均衡处理技术

Equalization Treatment Technique for Inverted Secondary Index Cluster of Quasi-real-time Data in Distribution Network

针对配电网的准实时数据体量越来越大、数据节点任务不均衡的问题,在集群框架的基础上建立倒排索引数据结构,提出一种准实时数据的二级索引集群均衡处理技术。以建立的索引集群查询时间模型作为拓扑选择策略依据,综合运用一致性哈希数据分发策略和基于分流轮询策略改进的带权重定向任务分配法,实现集群配置相同情况下对海量配电网准实时数据查询速度的提升。以铁路供电监控系统导出的亿级工程数据为算例,进行集群响应测试和分布式死锁测试。结果表明:拓扑选择策略有助于避免拓扑规划不合理导致的查询耗时增长,带权重定向任务分配法可使并发量小于可用服务线程数5倍时的查询耗时降低50%以上,并显著减小分布式死锁出现的概率。

For the problem of quasi-real-time data volume become bigger and data node task unbalanced in the distribution network, an inverted index data structure were built on the basis of cluster framework and put forward a kind of secondary index cluster equalization treatment technology of quasi-real-time. The topology selective strategy was based on the query time model of index cluster, with consistent Hashing data distributing strategy and weighted directional task allocation method that based on diversion polling strategy, which realizes the improvement of query speed for massive quasi-real-time data in the same situation when the cluster hardware is the same. To set the millions of engineering data in the railway power supply monitoring system as an example, the response test and distributed deadlock test for cluster were curried out. The result shows that topology selective strategy is helpful to avoid query time growth which caused by unreasonable planning, weighted directional task allocation method can reduce the query time by more than 50% when the concurrency is less than 5 times of available service threads, and significantly reduce the probability of distributed deadlock.