基于分布式文件系统的MPP数据库扫描调度研究

Research on scan scheduling in MPP databases on distributed file systems

基于分布式文件系统的MPP(大规模并行处理)数据库是目前的研究热点,为改善其执行查询扫描操作前调度执行单元读取数据块的过程,提出一种基于节点负载的调度策略NLS。这种策略同时结合数据本地性和节点负载,通过本地读分配保证调度结果满足良好的数据本地性,基于节点的实时工作负载对中间调度结果进行重分配调整,以达到减少数据扫描操作完成时间的目标。实验结果表明,相比连续性调度策略FCS,NLS在保持90%以上数据本地性的同时,在完成时间上的优化最多达到32%,在测试的9种情况中平均优化25%。

MPP（Massive Parallel Processing） database over distributed file systems has become one of research hotspots currently. In order to improve the procedure that schedule execution units to read data blocks before executing query scan operations, a scheduling strategy NLS based on nodes workload is proposed, which combines data locality and nodes workload. On the one hand, the phase of data locality allocating ensures that the scheduling results meet good data locality.On the other hand, reallocating on middle scheduling results based on nodes workload attains the goal of reducing the makespan of scanning data. The experimental results show that compared with the continuity strategy FCS, NLS keeps data locality over 90%. Moreover, the improvement on makespan achieves 32% at most and the average improvement is 25% in all nine test cases.