应用透明的超算多层存储加速技术研究

Research on Application-Transparent Supercomputing Multi-tier Storage Acceleration Technology

在E级计算时代,超算系统一般使用多层存储架构以满足应用数据访问的容量和性能需求,这种架构中不同层次的存储介质差异较大,难以实现统一名字空间管理,往往需要应用修改数据访问流程才能最大程度利用到多层存储的性能和容量优势。针对多层存储统一名字空间的问题,提出针对非易失性双列存储模块(NVDIMM)的块级缓存和针对突发缓冲存储(BB)的文件级缓存技术。基于NVDIMM的块级缓存技术对缓存窗口灵活控制,以支持数据块粒度的异步读写,实现NVDIMM与BB层统一名字空间管理;基于BB的文件级缓存技术将数据缓存在BB层中,并动态迁移和管理文件副本,实现BB层与传统磁盘文件系统统一名字空间管理。在神威E级原型验证系统中的测试结果表明,所提出的两种技术较好地解决了多层存储的透明加速难题,NVDIMM块级缓存与BB相比,在缓存窗口16 MB时128 KB顺序读写带宽分别提升27%和36%,8 KB随机读写带宽分别提升20%和37%;基于BB的文件缓存技术利用BB的高带宽支撑数据访问,与全局文件系统相比,128 KB顺序读写带宽分别提升55%和141%,8 KB随机读写带宽分别提升163%和209%。此外,实际应用的测试也表明以上两种缓存技术具有透明的存储加速效果。

Supercomputing systems have entered the exascale era and to meet the growing demand for data access and performance,supercomputing systems typically require multi-tier storage architecture.Storage media typically vary between tiers,making it difficult to achieve unified namespace management.This paper proposes to address this problem by developing blocklevel caching based on Non-Volatile Dual In-line Memory Module(NVDIMM)and file-level cache based on Burst Buffer(BB)technologies.The NVDIMM-based block-level cache supports asynchronous read/write of data block granularity through flexible control of the cache window and realizes the unified namespace of NVDIMM and BB.The file-level cache is based on BB and supports data caching in it,dynamically migrates and manages file copies,and realizes the unified namespace between BB and the traditional disk file system.Evaluation result of the verification systems in the Sunway E-class prototype shows that both technologies proposed in this paper can achieve transparent acceleration.Compared with BB,the NVDIMM block-level cache increases the sequential read/write bandwidth of a 128 KB block by 27%and 36%,respectively,and the random read/write bandwidth of an 8 KB block by 20%and 37%,respectively,given a cache window size of 16 MB.The BB-based file cache allows applications to exploit the high bandwidth of BB.Compared with the Global File System(GFS),the 128 KB block sequential read/write bandwidth increases by 55%and 141%,respectively,and the 8 KB random read/write bandwidth increases by 163%and 209%,respectively.In addition,the practical application presented in this study shows that above two cache technologies have transparent storage acceleration effects.