摘要
如何为上层应用和计算提供稳定高效的文件I/O性能,是分布式文件系统性能研究的热点。文中分析分布式文件系统在设计机理上的共同特征,基于此提出一种通用型的启发式文件预取模型,并选取HDFS平台进行系统实现。启发式文件预取对上层应用透明,采用在文件系统内部建立预取线程池的方法,以组成文件块的数据存储文件为预取单位,在分布式文件系统内部实现。这种设计思路具有一定的普适性,适合推广应用于多种分布式文件系统。实验结果表明,所述的启发式文件预取,能够有效提升分布式文件系统的I/O性能。
How to provide a stable and efficient file I/O performance for the upper application and computing, is the performance research hotspot oriented to distributed file system. This paper analyzed the mechanism in the design of the distributed file system on the common features, presented a general-purpose file prefetching heuristic module, and se- lected HDFS platform system to implement. The heuristic file prefetching module services the upper application and ac- complishes the implementation in the internal of distributed file system, using the method of establishing prefetching thread pool within the file system,and the data not block as prefetching unit. This idea has certain universality, and is suitable for a variety of distributed file systems. Experimental results show that the heuristic file prefetching method can enhance the distributed file system I/O performance effectively.
出处
《计算机科学》
CSCD
北大核心
2014年第7期30-35,共6页
Computer Science
基金
国家“十二五”863计划信息技术领域重大项目“云计算关键技术与系统”课题:以公众汉语服务为主的搜索引擎研制(2011AA01A205)资助
