面向天河二号Lustre文件系统优化实践

Optimization Practice of Lustre File System for Tianhe No. 2

近年来,随着高性能计算技术的飞速发展,Lustre文件系统作为高性能计算重要组成部分越来越受到重视,对于Lustre文件系统元数据服务的高可靠性一直是研究的重点,针对Lustre文件系统元数据高可靠性的研究,本文设计了一种双MDT冗余的架构保证了元数据服务的高可靠性;同时随着应用复杂度的提高和计算规模的增加,对Lustre文件系统的性能要求也越来越高,特别是在处理海量小文件和I/O密集型应用时,对Lustre文件系统元数据的I/O性能提出了更高的要求,为了提高元数据的I/O性能,本文通过升级MDT元数据底层硬件设备来提升元数据的I/O性能和提升整体文件系统的I/O性能,满足新的应用对文件系统IO的要求。

In recent years, with the rapid development of high-performance computing technology, the Lustre file system has been paid more and more attention as an important component of high-performance computing. The high reliability of the metadata service of the Lustre file system has always been the focus of research. This paper has designed a dual MDT redundant architecture to ensure the high reliability of metadata services;at the same time, as the application complexity increases and the calculation scale increases, the performance requirements of the Lustre file system are also getting higher and higher, especially when dealing with massive small files and I/O intensive applications. I/O performance of the metadata of the Lustre file system has raised higher requirements. In order to improve the I/O performance of metadata, this article upgrades the underlying hardware equipment of MDT metadata to improve the metadata I/O performance and to improve the overall file system I/O performance to meet the new application requirements for file system IO.