摘要
城市监控视频数量呈现出爆炸式增长,为应对快速增长的存储压力,Hadoop分布式文件系统(HDFS)已被许多视频服务网站广泛使用,但其负载平衡工具并未考虑视频文件在线播放的带宽消耗特性和NameNode异构性能差异.基于此,本文提出一种基于HDFS的动态负载均衡方法.此方法建立了新的负载评估模型,通过对元数据的多副本异构节点进行动态自适应备份,实现了元数据的动态分配,并保证了元数据服务器集群的性能.实验结果表明,本方法可以有效避免高带宽消耗数据块的聚集,在90%的场景中优于原有的负载均衡方法,可以将数据节点集群中瓶颈节点的带宽峰值降低20%.
The number of surveillance videos in cities has exploded.In order to cope with the rapid increase in storage pressure,Hadoop Distributed File System(HDFS)has been widely used by many video service websites,but its load balancing tool does not consider the bandwidth consumption of online video files.Features and NameNode heterogeneous performance differences.Based on this,this paper proposes a dynamic load balancing method based on HDFS.This method establishes a new load evaluation model,and realizes the dynamic allocation of metadata and ensures the performance of the metadata server cluster through dynamic adaptive backup of heterogeneous nodes with multiple copies of metadata.Experimental results show that this method can effectively avoid the aggregation of high bandwidth consumption data blocks,is better than the original load balancing method in 90%of the scenarios,and can reduce the bandwidth peak of the bottleneck node in the data node cluster by 20%.
作者
李玥
LI Yue(Jincheng College of Sichuan University,Chengdu Sichuan 611731,China)
出处
《信息与电脑》
2021年第3期68-72,共5页
Information & Computer
