基于异构GPU集群的并行分布式编程解决方案

AN PARALLEL AND DISTRIBUTED PROGRAMMING SOLUTION BASED ON HETEROGENEOUS GPU CLUSTER

由于超强的计算能力、高速访存带宽、支持大规模数据级并行程序设计等特点,GPU已经成为超级计算机和高性能计算(HPC)集群的主流加速器。随着处理单元的发展和集群节点的拓展,GPU集群不仅在节点层面呈现异构化,节点内也趋于异构化,大大提高了在GPU集群中编程的复杂度。主流GPU异构集群系统大多采用针对GPU的异构计算编程模型与面向分布式内存的消息传递模型的简单结合方式,这种方式使得GPU集群程序设计缺乏确定的准则,往往是低效而且易错的。为了提高在GPU集群中编程的效率,降低编程复杂度,以及实现平台无关性,提出一套异构GPU集群的并行分布式编程的解决方案。该方案通过采用扩展语言方法提出了编程框架DISPAR,并实现了预处理器系统StreamCC。实验证明了其可行性。

Due to its characteristics of super powerful computing capability, high-speed memory access bandwidth and supporting large- scale data-level parallel programming, GPUs have become the mainstream accelerators for supercomputers and high performance computing field. GPU-enhanced clusters are showed to be heterogeneous in both node layer and intra-node as the evolvement of the processing elements and the expansion of the cluster nodes, which greatly increases the complexity of GPU cluster programming. Mainstream heterogeneous GPU clusters mostly adopt the way of simple combination of the heterogeneous computing programming model for GPU and the distributed memory- oriented message passing mode], such approach makes the GPU cluster programming lack of determinate criteria and often be inefficient and error-prone. In order to improve the efficiency of programming GPU cluster and to reduce programming complexity, as well as to realise the platform independence, we present a set of parallel and distributed programming solution for heterogeneous GPU cluster. The scheme presents the programming framework DISPAR through the use of the extended language method, and realises the preprocessor system StreamCC. Experiment proves its feasibility.