基于加权队列的多源大数据跨源调度仿真

Simulation of Multi-Source Big Data Cross-Source Scheduling Based on Weighted Queue

针对传统的大数据跨源调度方法存在延时较长、开销率较高和数据利用率较低等问题,提出了一种基于加权队列的多源大数据跨源调度方法。方法中引用服务概率函数,分别计算不同数据流的权重,并结合队列管理机制构建队列管理器。采用不同的加权方法分别计算各队列管理器中数据流传输节点的阈值,从中选取最佳阈值,通过对阈值的调节实现多源大数据跨源调度。实验结果表明,所提方法与传统方法相比,能够有效提高数据利用率,降低调度延时以及通信开销。

Due to high time delay,high overhead rate and low data utilization in traditional methods,this article presented a cross-origin scheduling method for multi-source big data based on weighted queue.In this method,the service probability function was used to calculate the weights of different data streams.The queue manager was constructed based on queue management mechanism.In addition,different weighting methods were used to calculate the thresholds of data stream transmission nodes in all queue managers,and the best thresholds were selected.By adjusting theses thresholds,the cross-source scheduling of multi-source big data was achieved.Simulation results show that,compared with traditional methods,the proposed method can effectively improve data utilization and reduce the scheduling delay and communication overhead.