基于全级C阶矩模型并行流数预测的广域大数据吞吐量优化

Wide Area Big Data Throughput Optimization Based on Full C-order Moment Model with Parallel Flow Prediction

针对传统大数据密集型的可扩展计算系统在数据源利用和数据传输方面效率不高的问题,提出基于并行流数预测的应用层吞吐量优化模型。为提高并行流数预测精度,以提高瓶颈链路的利用效率为目的,设计等效并行流数选取方式。借鉴部分C阶矩模型和完全二阶矩模型,构建全级C阶矩模型,并且设计低采样吞吐量优化框架,降低计算复杂度。在不同大小数据集上的实验结果表明,全级C阶矩并行流数的预测模型更适合大数据传输,且效率更高。

In order to solve the problem of low efficiency in the use of data sources and data transmission of the traditional data intensive scalable computing systems, this paper proposes a big data throughput optimization algorithm based on full C-order moment model with parallel flow prediction. To improve the prediction accuracy of parallel flow, it takes the utilization efficiency improvement of the bottleneck link for the purpose, and designs the method of equivalent parallel flow algorithm. According to the partial C-order moment model and full second-order moment model, it constructs the full C-order moment model, and designs the low sample throughput optimization framework, which can reduce the computational complexity. Experimental results on the data set with different size show that the full C-order moment model with parallel flow is more suitable for the transmission of big data, and more efficient.