Hybrid hierarchy storage system in MilkyWay-2 supercomputer

With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabytes. To reduce this gap, large-scale storage systems need to be designed and implemented with high performance and scalability. MilkyWay-2 （TH-2） supercomputer system with peak performance 54.9 Props, definitely has this kind of requirement for storage system. This paper mainly introduces the storage system in MilkyWay-2 supercomputer, including the hardware architecture and the parallel file system. The storage system in MilkyWay-2 supercomputer exploits a novel hybrid hierarchy storage architecture to enable high scalability of I/O clients, I/O bandwidth and storage capacity. To fit this architecture, a user level virtualized file system, named H^2FS, is designed and implemented which can cooperate local storage and shared storage together into a dynamic single namespace to optimize I/O performance in IO-intensive applications. The evaluation results show that the storage system in MilkyWay-2 supercomputer can satisfy the critical requirements in large scale supercomputer, such as performance and scalability.

Pinpointing and scheduling access conflicts to improve internal resource utilization in solid-state drives

Modern solid-state drives (SSDs)are integrating more internal resources to achieve higher capacity.Parallelizing accesses across internal resources can potentially enhance the performance of SSDs.However,exploiting parallelism inside SSDs is challenging owing to real-time access conflicts.In this paper,we propose a highly parallelizable I/O scheduler (PIOS)to improve internal resource utilization in SSDs from the perspective of I/O scheduling.Specifically, we first pinpoint the conflicting flash requests with precision during the address translation in the Flash Translation Layer (FTL).Then,we introduce conflict eliminated requests (CERs)to reorganize the I/O requests in the device-level queue by dispatching conflicting flash requests to different CERs.Owing to the significant performance discrepancy between flash read and write operations,PIOS employs differentiated scheduling schemes for read and write CER queues to always allocate internal resources to the conflicting CERs that are more valuable.The small dominant size prioritized scheduling policy for the write queue significantly decreases the average write latency.The high parallelism density prioritized scheduling policy for the read queue better utilizes resources by exploiting internal parallelism aggressively.Our evaluation results show that the paralle/izable I/O scheduler (PIOS)can accomplish better SSD performance than existing I/O schedulers implemented in both SSD devices and operating systems.