一种面向低延迟的内存HDFS数据存储策略

A Data Storage Strategy for Low-latency in Memory HDFS

从底层存储介质的选择和上层分布式文件系统效率的角度出发,提出基于HDFS的内存分布式文件系统架构Mem-HDFS,同时利用集群DataNode的内存和磁盘存储数据,以提高基于HDFS应用的读取性能.并对其进行性能优化,提出一种适应的同分布数据存储策略以及并行读取访问算法.对比实验表明:提出的算法能够较好地降低读取访问延迟.

In this paper,we present a storage system,Mem-HDFS,to enable low-latency read accesses.Mem-HDFS integrates the available memory resources in an HDFS clusters to form a cloud storage system.We also present an adaptive identically distributed storage strategy and concurrently read access algorithm.By serving data from memory instead of disks,Mem-HDFS can yield high I/O performance with low latency.