使用Spack软件包管理器构建高性能计算软件环境

Building a Software Environment with Spack on HPC Platform

为科研人员提供一个稳定易用的计算环境是高性能计算平台的重要任务。现有主流的基于手动编译的软件环境构建方法,由于缺少自动编译和多版本共存等特性,难以有效管理日益增多的软件。文章介绍一个面向高性能计算的软件包管理器Spack,包括其主要用法、内部设计、在编译耗时和运行效率上的优势。实验结果表明,Spack有效降低了软件编译时间,且编译出的软件运行效率优于Docker,与手工编译相同。上海交通大学高性能计算平台基于Spack实现了分角色的软件管理方法,除了管理员提供超过200种常用软件,用户可以很方便地自行构建专属的软件环境。新软件上线时间从5天缩短到2天,极大提高了系统管理员的维护效率和用户的使用效率。

Maintaining a powerful yet easy-to-use High high Performance performance Computing computing(HPC)software environment is a critical task for HPC centers.We find that current existing approaches of building software,including manual compiling and Autotools autotools compiling,failed to manage the complexity raised by increasing number of softwares.To resolve HPC software management issues,we present Spack,an open source package manager designed for HPC.Basic usage and internal implementations of Spack are discussed,along with experiments and role-based Spack usage on Shanghai Jiao Tong University Supercomputer.Experiential results show that,Spack outperforms the manual approach in compiling speed,and outperforms Docker in software efficiency.With the help of Spack,more than 200 software packages are provided on SJTU HPC platform,and time taken to install a new software is shorten from 5 days to 2 days.Besides that,users are enable to build their private software environments alongside the global one provided by administrators.