摘要
HDFS(Hadoop Distributed File System)默认随机选择副本存储位置,未充分考虑数据节点的差异性。文章提出了一种改进的数据存放策略,使用多目标优化方法选取机架,通过计算集群和节点的使用率,择优选择存放节点。实验结果表明,改进后的策略优化了集群中节点的负载均衡,提高了数据读取性能。
HDFS(Hadoop distributed file system)randomly selects the replica storage location by default,without fully considering the differences of data nodes.This paper proposes an improved data storage strategy,which uses the multi-objective optimization method to select the rack,and selects the storage node by calculating the utilization rate of the cluster and nodes.The experimental results show that the improved strategy optimizes the load balancing of nodes in the cluster and improves the data reading performance.
作者
袁爱平
邓河
陈为满
YUAN Ai-ping;DENG he;CHEN Wei-man(Department of Software,Changsha Social Work College,Changsha 410004,China)
出处
《电脑与信息技术》
2022年第5期49-51,共3页
Computer and Information Technology
基金
湖南省教育厅科学研究项目“云计算环境下大数据分布式存储策略研究”(项目编号:19C0103)。
关键词
HDFS
副本放置
多目标优化
负载均衡
读取性能
HDFS
replica placement
multi-objective optimization
load balancing
reading performance
