Redbud并行文件系统的可扩展存储管理机制

High Scalable Storage Management in Redbud Parallel File System

数据规模和并发访问的需求日益增长,可扩展能力成为并行文件系统的重要需求之一.文中提出了一种基于非对称并行文件系统Redbud的高可扩展资源管理机制.该管理机制根据数据的访问特征,使用不同的树形结构管理不同类型的数据,满足了文件数据和元数据的并发检索需求;该管理机制还使用文件级的数据分布机制,允许用户利用各种策略进行目录和文件的管理,能满足文件级的数据访问性能、目录级数据可靠性等实际应用需求.多个基准测试程序和实际应用程序的测试结果表明,文件的独占访问能达到磁盘95%的性能;同时,随着设备和应用节点的增加,数据和元数据的并发访问性能线性增长.

With ever-increasing storage scale and parallel file access pattern, the high scalability becomes one of important requirement of parallel file system. In this paper, we present a scalable storage management mechanism for an asymmetric parallel file system, Redbud. According to the files access pattern, the management mechanism leverages multiple tree structures to manage different types of data, satisfying parallel file index requirement of users~ we also introduce a file-level configurable data placement mechanism, allowing users to distribute files and directory with scripted policy, satisfying diverse users~ demands, including file-level performance and directory-level security. Measurements of both a wide range of benchmarks and realistic workloads demonstrate that exclusive file access performance achieves 95% peak performance of disks; meanwhile, both metadata and data parallel access performance scale as either devices or clients increase.