面向新一代神威超级计算机的高效内存分配器

Efficient memory allocator for the New Generation Sunway supercomputer

随着应用程序规模的增大,应用程序对计算资源的需求也日益增加,超级计算机为满足这一需求提供了良好的平台。传统的超级计算机主要面向科学计算程序,而近年来应用的多样化对超级计算机的软硬件设计提出了新要求。该文在新一代神威超级计算机上发现了在动态运行模式下内存分配的性能问题,并针对神威的体系结构特征和应用特征,设计了高效的内存分配器——SWAlloc。实验结果表明:SWAlloc可以将超大规模机器学习训练框架八卦炉的内存分配速度提升至多75 839倍;对随机生成的内存分配记录和标准测试程序集PARSEC中的内存分配记录的测试结果,验证了SWAlloc在不同应用上的通用性和高效性,可将神威超级计算机上PARSEC的内存分配效率提升至多51倍(平均提升36%)。SWAlloc已经布署于新一代神威超级计算机上,并用于SWPytorch、 SWTensorFlow等超大规模应用。

Supercomputers provide enormous computing power for large applications. Traditional supercomputers have mainly targeted scientific computing problems. However, other applications have new requirements for the both supercomputer software and hardware designs. The New Generation Sunway supercomputer has an inefficient memory allocator when running in the dynamic mode. This study develops an efficient memory allocator, SWAlloc, that reduces the memory allocation time of the brain scale pretrained model training framework, BaGuaLu, by up to 75 839 times. Evaluations using PARSEC also show that SWAlloc can speed up the memory allocation by up to 51 times(36% on average). SWAlloc has been deployed on the New Generation Sunway supercomputer for use by various large applications, including SWPytorch and SWTensorFlow.