神威太湖之光上OpenFOAM的移植与优化

Porting and Optimizing OpenFOAM on Sunway TaihuLight System

神威太湖之光是最新一期Top500榜单上排名第一的超级计算机,峰值性能为125.4PFlops,其计算能力主要归功于国产SW26010众核处理器。OpenFOAM(Open Source Field Operation and Manipulation)是计算流体力学领域使用最广泛的开源软件包,但是由于其基于C++实现,与神威太湖之光上的异构众核处理器SW26010的编译器不兼容,因此无法直接在该架构上有效运行。基于SW26010的主核/从核的体系架构移植了OpenFOAM的核心计算代码,并采用混合语言编程实现的方式来解决编译不兼容的问题。此外,通过寄存器通信、向量化和双缓冲等优化手段,单核组的性能较优化后的主核代码提高了8.03倍,较Intel(R)Xeon(R)CPU E5-2695v3的串行执行性能提高了1.18倍。同时,将单核组的实现扩展到了神威太湖之光的大规模集群上,并进行了强可扩展性测试,256个核组上实现了184.9倍的加速。采用的移植方式和优化手段也可以为其他复杂C++程序在神威太湖之光上的应用提供借鉴。

The Sunway TaihuLight supercomputer based on the Chinese-designed many-core processors is the world＇s fastest system with a peak performance of 125.4PFlops.OpenFOAM（open source field operation and manipulation）is one of the most popular open source computational fluid dynamics（CFD）software which is written in C＋＋ and not fully compatible with compilers on the heterogeneous many-core processor SW26010.This paper ported OpenFOAM based on SW26010＇s MPE（management processing element）/CPE（computing processing element）cluster architecture.To overcome the compilation incompatibility problem,we adopted the mixed-language application design.We also applied several SW26010＇s feature-specific optimizations on the hotspot of OpenFOAM to deliver high performance,such as the register communication,vectorization,and double buffering.The experiments on SW26010 using real datasets show that the single-CG（core group）code runs 8.03 x faster than the well-tuned version on the MPE,and the performance of single-CG is 1.18 x higher than the serial implementation of Intel（R）Xeon（R）CPU E5-2695v3.We also optimized the parallel implementation of OpenFOAM and yielded speedups of 184.9x on 256 CGs.The porting methods and optimizations presented can also be referenced for other complex C＋＋ programs to achieve high performance on SW26010.