网格化分布式新安江模型并行计算算法

Parallel computing algorithm of grid-based distributed Xin’anjiang hydrological model

近年来,网格化分布式新安江模型(GXM)在洪水预报中发挥了重大作用,但在进行洪水过程模拟时,模型数据量与计算量巨大,GXM的计算时间随着模型预热期的增加呈指数增长,严重影响GXM的计算效率。因此,提出一种基于网格流向划分与动态优先级有向无环图(DAG)调度的GXM并行算法。首先,对模型参数、模型构件、模型计算过程进行分析;其次,从空间并行性的角度提出了基于网格流向划分的GXM并行算法以提高模型的计算效率;最后,提出一种基于动态优先级的DAG任务调度算法,通过构建网格计算节点的DAG并动态更新计算节点的优先级以实现GXM计算过程中的任务调度,减少模型计算中数据倾斜现象的产生。在陕西省大理河流域与安徽省屯溪流域对提出的算法进行实验,在预热期为30 d、数据分辨率为1 km的情况下,相较于传统的串行算法,所提算法的最大加速比分别达到了4.03和4.11,有效提升了GXM的计算速度与资源利用率。

In recent years,the Grid-based distributed Xin’anjiang hydrological Model(GXM)has played an important role in flood forecasting,but when simulating the flooding process,due to the vast amount of data and calculation of the model,the computing time of GXM increases exponentially with the increase of the model warm-up period,which seriously affects the computational efficiency of GXM.Therefore,a parallel computing algorithm of GXM based on grid flow direction division and dynamic priority Directed Acyclic Graph(DAG)scheduling was proposed.Firstly,the model parameters,model components,and model calculation process were analyzed.Secondly,a parallel algorithm of GXM based on grid flow direction division was proposed from the perspective of spatial parallelism to improve the computational efficiency of the model.Finally,a DAG task scheduling algorithm based on dynamic priority was proposed to reduce the occurrence of data skew in model calculation by constructing the DAG of grid computing nodes and dynamically updating the priorities of computing nodes to achieve task scheduling during GXM computation.Experimental results on Dali River basin of Shaanxi Province and Tunxi basin of Anhui Province show that compared with the traditional serial computing method,the maximum speedup ratio of the proposed algorithm reaches 4.03 and 4.11,respectively,the computing speed and resource utilization of GXM were effectively improved when the warm-up period is 30 days and the data resolution is 1 km.