为解决Hadoop现有调度器调度任务时不能根据任务的紧迫程度分配资源的问题,研究YARN中的资源调度机制,改进原调度器(Capacity Scheduler),提出一种基于优先级权重的Hadoop YARN(Yet Another Resource Negotiator)调度算法(Weight Schedu...为解决Hadoop现有调度器调度任务时不能根据任务的紧迫程度分配资源的问题,研究YARN中的资源调度机制,改进原调度器(Capacity Scheduler),提出一种基于优先级权重的Hadoop YARN(Yet Another Resource Negotiator)调度算法(Weight Scheduler Based on Priority)。为叶子队列设置队列优先级,结合队列资源利用率和队列优先级选择队列;将应用程序的初始权重设置为应用程序优先级的大小,通过等待时间判断是否更新权重,根据权重对队列中的应用程序进行排序,调度时优先为权重高的应用程序分配资源。实验结果表明,与原有调度算法相比,改进算法平均任务执行时间约减少141 s,平均等待时间减少34.5%,保证了用户执行任务的相对公平,提高了用户总体满意度。展开更多
The paper follows possible specification of a control algorithm of a WS （water management system） during floods using the procedures of AI （artificial intelligence）. The issue of minimizing negative impacts of flo...The paper follows possible specification of a control algorithm of a WS （water management system） during floods using the procedures of AI （artificial intelligence）. The issue of minimizing negative impacts of floods represents influencing and controlling a dynamic process of the system where the main regulation elements are water reservoirs. Control of water outflow from reservoirs is implicitly based on the used model （titled BW） based on FR （fuzzy regulation）. Specification of a control algorithm means dealing with the issue of preparing a knowledge base for the process of tuning fuzzy regulators based on an I/O （input/output） matrix obtained by optimization of the target behaviour of WS. Partial results can be compared with the regulation outputs when specialized tuning was used for the fuzzy regulator of the control algorithm. Basic approaches follow from the narrow relation on BW model use to simulate floods, without any connection to real water management system. A generally introduced model allows description of an outflow dynamic system with stochastic inputs using submodels of robust regression in the outflow module. The submodels are constructed on data of historical FS （flood situations）.展开更多
文摘为解决Hadoop现有调度器调度任务时不能根据任务的紧迫程度分配资源的问题,研究YARN中的资源调度机制,改进原调度器(Capacity Scheduler),提出一种基于优先级权重的Hadoop YARN(Yet Another Resource Negotiator)调度算法(Weight Scheduler Based on Priority)。为叶子队列设置队列优先级,结合队列资源利用率和队列优先级选择队列;将应用程序的初始权重设置为应用程序优先级的大小,通过等待时间判断是否更新权重,根据权重对队列中的应用程序进行排序,调度时优先为权重高的应用程序分配资源。实验结果表明,与原有调度算法相比,改进算法平均任务执行时间约减少141 s,平均等待时间减少34.5%,保证了用户执行任务的相对公平,提高了用户总体满意度。
文摘The paper follows possible specification of a control algorithm of a WS （water management system） during floods using the procedures of AI （artificial intelligence）. The issue of minimizing negative impacts of floods represents influencing and controlling a dynamic process of the system where the main regulation elements are water reservoirs. Control of water outflow from reservoirs is implicitly based on the used model （titled BW） based on FR （fuzzy regulation）. Specification of a control algorithm means dealing with the issue of preparing a knowledge base for the process of tuning fuzzy regulators based on an I/O （input/output） matrix obtained by optimization of the target behaviour of WS. Partial results can be compared with the regulation outputs when specialized tuning was used for the fuzzy regulator of the control algorithm. Basic approaches follow from the narrow relation on BW model use to simulate floods, without any connection to real water management system. A generally introduced model allows description of an outflow dynamic system with stochastic inputs using submodels of robust regression in the outflow module. The submodels are constructed on data of historical FS （flood situations）.
