面向非易失内存的MPI-IO接口优化

An optimization of MPI-IO interface for non-volatile memory

在高性能计算环境中,MPI应用多个计算节点同时访问底层存储系统文件时,其I/O开销受到访问模式和外存设备性能的影响。针对MPI应用访问文件的特征,利用非易失内存高带宽、低时延、可字节寻址、数据可持久化等优势,提出面向非易失内存的MPI-IO接口优化方案;对文件数据建立分布式的缓存并维护持久性的元数据、对进程间数据传输策略进行优化,使应用可以有效管理、利用非易失内存设备,保持缓存数据一致有效。实验结果证明,所提系统为应用带来数十倍的读写性能提升。未来将进一步优化本方案的并行性。

In an HPC system where multiple computation nodes of an MPI application simultaneously access files in underlying storage systems,the I/O overhead is affected by the access mode and the properties of external storage devices.Based on the patterns of MPI applications to access files,an optimization for MPI-IO interface for persistent memories was introduced on high-bandwidth,low-latency,byte-addressable,data-persistent memories.By constructing distributed data cache,maintaining persistent metadata and leveraging optimizations on data movements among processes,applications were enabled to efficiently manage and utilize persistent memories with data consistency guaranteed,resulting in tens of times improvement on read/write bandwidth.Further optimizations on parallelism were set for future work.