一种面向车辆实时数据并行处理的任务调度算法

A Parallel Task Scheduling Algorithm for the Processing of Massive Vehicle Real-time Data

车辆实时监管正面临着不断增长的大规模车辆监测数据的实时处理需求,需要采用分布式的并行计算架构来提升大规模车辆监测数据处理的性能,支撑多样化的车辆监测数据处理任务,应对支撑环境的伸缩性需求。在这种架构下,对系统中不同计算节点间的车辆监测数据处理任务的调度提出了更高的要求。针对这一要求,并结合流式到达及历史积累的车辆监测数据的持续化处理需求以及大规模车辆监测数据实时处理中内存敏感的特征,提出一种基于路由表的并行任务调度算法。该算法基于车辆监测数据时空属性以及各计算节点的内存信息建立路由表,并以路由表的形式来进行任务的并行划分和分配调度,从而使得各计算节点达到负载均衡的状态。实验表明该算法能够使计算节点间的负载差异缩小到12%以内。此外,该算法在某市车辆监管实时系统中的实际应用也证明了其有效性。

Real-time vehicle monitoring is facing the processing requirement of large and growing scale vehicle real-time monitoring data,and it needs a distributed parallel computing architecture to improve the performance of large-scale vehicle monitoring data processing,to support the diverse processing requirement of vehicle monitoring data and to deal with the scalability requirement of supporting environment.In this distributed architecture,it is more important to have a good method to schedule the different types of vehicle monitoring data peocessing tasks among computing nodes.To solve this problem,combined with the continued processing requirement of vehicle monitoring data which reaches in the way of stream and has large-scale historical accumulation,according to the memory sensitive characteristic of the large-scale vehicle real-time monitoring data processing,a parallel task scheduling algorithm is proposed based on the routing table.The algorithm diverses the tasks parallely according to the vehicle monitoring data＇s spatial and temporal properties,and scheduals tasks in the way of routing table exploiting the memory information of the computing nodes,so as to achieve the state of load balancing among computing nodes.The experiments show that the algorithm can reduce the load difference among computing nodes to less than 12%.In addition,the algorithm has been a practical application in a city＇s real-time vehicle monitoring system.