一种性能优化的小文件存储访问策略的研究

A Strategy of Small File Storage Access with Performance Optimization

在分布式文件系统中,小文件的管理一般存在访问性能较差和存储空间浪费较大等缺点.为了解决这些问题,提出了一种性能优化的小文件存储访问(SFSA)策略.SFSA将逻辑上连续的数据尽可能存储在物理磁盘的连续空间,使用Cache充当元数据服务器的角色并通过简化的文件信息节点提高Cache利用率,提高了小文件访问性能;写数据时聚合更新数据及其文件夹域中的相关数据为一次I/O请求写入,减少了文件碎片数量,提高了存储空间利用率;文件传输时利用局部性原理,提前发送批量的高访问率的小文件,降低了建立网络连接开销,提升了文件传输性能.理论分析和实验证明,SFSA的设计思想和方法能有效地优化小文件的存储访问性能.

In distributed file system, the management of small file storage access has encountered some problems, such as poor access performance, low disk space utilization rate, high file transfer delay, etc. To solve these problems, this paper proposes a strategy of small file storage access （SFSA） with performance optimization. SFSA can try to store logical continuous data on continuous space of physical disks as far as possible, and use a cache to act as metadata server and improve utilization rate of cache by using simplified file information node. Therefore it can improve the performance of small file storage access. In order to solve the problem of low disk space utilization rate, SFSA still uses a method of writing optimization which combines the dirty data with its related data in file folder domain into a single I/O request, so, it can reduce the number of file fragments. In addition, according to the principle of data locality, we also propose a method which sends the highly accessed small flies ahead of time. It reduces the overhead of network connection and improves the file transfer performance. Theoretical analysis and experimental results show that the design idea and method of SFSA strategy can improve the performance of small file storage access effectively.