摘要
为在已有硬件资源配置条件下提高运算效率,提出了基于HDFS分布式存储环境、考虑数据依赖关系的任务调度和资源部署算法。建立的任务调度和资源部署模型包含HDFS集群、数据节点磁盘输入输出负载、网络负载、磁盘输入输出负载的矩阵描述,任务调度和资源部署流程通过设计的任务排程器进行实现。将节点资源评价的函数值作为对算法的评价指标,测试比较了几种算法的性能。在节点资源评价结果方面,与离请求者最近算法、随机算法相比,提出的任务调度和资源部署算法具有较大优势。
In order to improve the operation efficiency under the condition of existing hardware resource configuration,the task scheduling and resource deployment algorithm is proposed,considering data dependencies and distributed storage environment based on HDFS.The task scheduling and resource deployment model includes HDFS cluster,data node disk input and output load,network load,disk input and output matrix description.The process is implemented by task scheduler.Taking the function value of node resource evaluation as the evaluation index of the algorithm,the performance of several algorithms is tested and compared.At last,the experimental results show that the task scheduling and resource deployment algorithm in this paper has more advantages than the nearest algorithm and the stochastic algorithm in resource evaluation.
作者
蔡尊煌
CAI Zunhuang(Fujian Forestry Vocational and Technical College,Nanping Fujian 353000,China)
出处
《重庆科技学院学报(自然科学版)》
CAS
2020年第1期56-59,共4页
Journal of Chongqing University of Science and Technology:Natural Sciences Edition
基金
福建省教育厅中青年教师教育科研(A类)项目“逆向工程技术在高精度模具开发中的应用研究”(JAT170984)。