基于云存储的二阶段动态优化调度机制

Two-stage Dynamic Optimized Scheduling Mechanism Based on Cloud Storage

在分布式存储的研究中,如何高效地利用存储空间是个热点问题。存储集群中,每个数据节点存储容量不可能完全一致,由于主节点选择数据节点的随机性,被选中数据节点磁盘可能接近满额,此时主节点会自动做存储负载均衡,占用数据传输带宽,不仅影响数据传输的性能,而且会引起传输数据的不可靠。论文提出一种基于云存储的二阶段动态优化调度机制:第一阶段通过计算副本存储优选比率,采用基于贪心算法的局部优化存储方案,选择存储节点,均衡副本放置空间;第二阶段采用实时监控存储集群,动态调整副本放置节点,达到存储资源的高效利用。最后通过实验,验证了该调度机制可有效地放置副本,减少节点间的数据传输,并提高文件访问效率。

How to use storage space effectively in distributed storage cluster is a hot issue. The storage capacity can＇t be completely consistent in the cluster. Due to the randomness selection of the masternode, the selected datanode is likely to be close to full in disk, then the master automatically does storage load balance. It will not only affect the performance of data transimission, also can lead to less reliability of the data. In this paper, a two-stage dynamic optimal scheduling mechanism based on cloud storage is presented： the first stage uses the local optimal storage scheme based on greedy algorithm, by calculating copy storage optimal ratio, chooses the storage node and balance the placed space; the second stage use the realtime monitor to get the information of the cluster, dynamically adjust the placed nodes of the replicas. Finally, the experiment demonstrates that the dynamic scheduling mechanism can effectively place replicas, reduce the data transimission between datanodes and improve the efficiency of the data access.