MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a hea...MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a heartbeat mechanism for MapReduce Task Scheduler using Dynamic Calibration （HMTS- DC） is proposed to address the unbalanced node computation capacity problem in a het- erogeneous MapReduce environment. HMTS- DC uses two mechanisms to dynamically adapt and balance tasks assigned to each com- pute node： 1） using heartbeat to dynamically estimate the capacity of the compute nodes, and 2） using data locality of replicated data blocks to reduce data transfer between nodes. With the first mechanism, based on the heart- beats received during the early state of the job, the task scheduler can dynamically estimate the computational capacity of each node. Us- ing the second mechanism, unprocessed Tasks local to each compute node are reassigned and reserved to allow nodes with greater capacities to reserve more local tasks than their weaker counterparts. Experimental results show that HMTS-DC performs better than Hadoop and Dynamic Data Placement Strategy （DDP） in a dynamic environment. Furthermore, an en- hanced HMTS-DC （EHMTS-DC） is proposed bv incorporatin historical data. In contrastto the ＂slow start＂ property of HMTS-DC, EHMTS-DC relies on the historical computation capacity of the slave machines. The experimental results show that EHMTS-DC outperforms HMTS-DC in a dynamic environment.展开更多
This study evaluates the performance of the Grid-point Atmospheric Model of IAP LASG,version 3(GAMIL3),in simulating the Madden–Julian Oscillation(MJO),based on the CMIP6(phase 6 of the Coupled Model Intercomparison ...This study evaluates the performance of the Grid-point Atmospheric Model of IAP LASG,version 3(GAMIL3),in simulating the Madden–Julian Oscillation(MJO),based on the CMIP6(phase 6 of the Coupled Model Intercomparison Project)AMIP(Atmospheric Model Intercomparison Project)simulation.Results show that GAMIL3 reasonably captures the main features of the MJO,such as the eastward-propagating signal in the MJO frequency band,the symmetric and asymmetric structures of the MJO,several convectively coupled equatorial waves,and the MJO life cycle.However,GAMIL3 underestimates the MJO amplitude,especially for outgoing longwave radiation,as do most CMIP5 models,and simulates slow eastward propagation.展开更多
文摘MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a heartbeat mechanism for MapReduce Task Scheduler using Dynamic Calibration （HMTS- DC） is proposed to address the unbalanced node computation capacity problem in a het- erogeneous MapReduce environment. HMTS- DC uses two mechanisms to dynamically adapt and balance tasks assigned to each com- pute node： 1） using heartbeat to dynamically estimate the capacity of the compute nodes, and 2） using data locality of replicated data blocks to reduce data transfer between nodes. With the first mechanism, based on the heart- beats received during the early state of the job, the task scheduler can dynamically estimate the computational capacity of each node. Us- ing the second mechanism, unprocessed Tasks local to each compute node are reassigned and reserved to allow nodes with greater capacities to reserve more local tasks than their weaker counterparts. Experimental results show that HMTS-DC performs better than Hadoop and Dynamic Data Placement Strategy （DDP） in a dynamic environment. Furthermore, an en- hanced HMTS-DC （EHMTS-DC） is proposed bv incorporatin historical data. In contrastto the ＂slow start＂ property of HMTS-DC, EHMTS-DC relies on the historical computation capacity of the slave machines. The experimental results show that EHMTS-DC outperforms HMTS-DC in a dynamic environment.
基金jointly supported by the National Key Research and Development Program of China grant number 2017YFA0603903the National Natural Science Foundation of China grant numbers 41622503 and 41775101。
文摘This study evaluates the performance of the Grid-point Atmospheric Model of IAP LASG,version 3(GAMIL3),in simulating the Madden–Julian Oscillation(MJO),based on the CMIP6(phase 6 of the Coupled Model Intercomparison Project)AMIP(Atmospheric Model Intercomparison Project)simulation.Results show that GAMIL3 reasonably captures the main features of the MJO,such as the eastward-propagating signal in the MJO frequency band,the symmetric and asymmetric structures of the MJO,several convectively coupled equatorial waves,and the MJO life cycle.However,GAMIL3 underestimates the MJO amplitude,especially for outgoing longwave radiation,as do most CMIP5 models,and simulates slow eastward propagation.
