跨语言用户态文件系统框架读写性能优化

Reading and Writing Performance Optimization of Cross-Language FUSE Framework

以深度学习为代表的数据分析应用越来越多依赖分布式文件系统存储管理大规模数据集.为了增强数据访问的兼容性,现有分布式文件存储系统通常需提供标准POSIX接口,以支持深度学习等应用的无缝对接.然而,以内核模块形态开发提供POSIX接口的文件系统非常复杂耗时.近年来,用户态文件系统(Filesystem in Userspace,FUSE)框架大幅简化了文件系统的开发工作,已被Alluxio和Ceph等诸多知名分布式文件系统使用.目前常用的用户态FUSE库libfuse仅提供C语言编程接口,但现有大数据分布式文件系统基本都是基于Java语言开发的(例如HDFS和Alluxio等),为了使基于Java语言开发的分布式文件系统可以对接C语言开发的FUSE库,需采用跨语言FUSE框架作为中介.跨语言FUSE框架利用跨编程语言的函数回调机制,使操作系统FUSE库的C语言函数可以跨语言的调用分布式文件系统提供的Java语言编程接口,从而为大数据分布式文件系统提供标准POSIX接口的访问能力.但在数据密集型应用中,现有跨语言FUSE框架的执行效率低,导致数据密集型作业(深度学习、大数据分析等)中数据I/O耗时占据了显著的性能开销,成为新的潜在性能瓶颈.针对此问题,本文首先评估分析了重要且广为使用的跨语言FUSE框架JNR-FUSE的性能,发现并定位其在高并发和小文件场景下存在的性能瓶颈;接着从多方面剖析性能瓶颈根因,进而总结出高效跨语言FUSE框架的性能优化方向,并面向Java语言设计实现了跨语言FUSE框架JNI-FUSE.JNI-FUSE利用延迟分离和元信息缓存等优化技术降低跨语言函数回调开销,从而提升跨语言FUSE框架的性能.实验结果表明,对比当前性能最好的Java FUSE框架JNR-FUSE,本文提出的JNI-FUSE带来了1.15~6.04倍的FUSE框架性能提升和1.90~2.71倍的文件系统端到端性能提升,并为上层深度学习训练任务带来了1.06~1.73倍的训练加速.本文设计提出的JNI-FUSE(Java Native Interface-Filesystem in User SpacE)因性能优势,已被知名开源分布式文件系统Alluxio官方接受集成.

Big data analytical applications like deep learning-based AI applications more and more rely on distributed file system to store and manage large scale data sets.File systems often need to provide standard POSIX interfaces to en⁃hance their access compatibility with upper-layer applications.However,it is complicated to develop POSIX-compatible file systems in kernel space.In recent years,FUSE(File System in User Space)has been used by many well-known file sys⁃tems,including Alluxio,Ceph,etc.,because it significantly simplifies the file system development.The popular FUSE li⁃brary libfuse is developed in the C language.However,the popular distributed file systems(like HDFS and Alluxio)for big data applications are developed with the Java language.To make the Java-based distributed file systems use the FUSE mechanism,the cross-language FUSE frameworks are needed to bridge the gap,which becomes a potential performance bot⁃tleneck.The cross-language FUSE framework uses a cross-programming language function callback mechanism to enable the C functions of the FUSE library to call the programming interface provided by the distributed file system in Java.In this way,we can provide the access to the standard POSIX interface for the Java-based distributed file systems.However,the existing cross-language FUSE frameworks are inefficient in performance.It makes the data I/O in data-intensive appli⁃cations(like deep learning and big data analysis)occupy noticeable proportion of their execution costs.To address the prob⁃lem,we first systematically evaluate the performance of the widely-used cross-language FUSE framework,and find the bot⁃tlenecks of throughput performance in high concurrency and small file scenarios.We then analyze the bottlenecks of the cross-language FUSE framework from multiple perspectives,and propose several directions for optimizing the cross-lan⁃guage FUSE framework.According to the optimization directions,we design and implement JNI-FUSE(Java Native Inter⁃face-Filesystem in User SpacE),an efficient cross-language FUSE framework.In JNI-FUSE,we propose the defer detach and meta cache techniques to reduce execution costs of cross-programming language function callbacks.Experiment results show that JNI-FUSE improves the average framework performance from 1.15 times to 6.04 times compared to the cuttingedge cross-language FUSE framework JNR-FUSE.JNI-FUSE improves the end-to-end performance by 1.90 time to 2.71 times,and accelerates the deep learning training by 1.06 times to 1.73 times compared to JNR-FUSE.JNI-FUSE has been accepted and integrated by the well-known open-source distributed file system Alluxio due to its good performance.