MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a hea...MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a heartbeat mechanism for MapReduce Task Scheduler using Dynamic Calibration （HMTS- DC） is proposed to address the unbalanced node computation capacity problem in a het- erogeneous MapReduce environment. HMTS- DC uses two mechanisms to dynamically adapt and balance tasks assigned to each com- pute node： 1） using heartbeat to dynamically estimate the capacity of the compute nodes, and 2） using data locality of replicated data blocks to reduce data transfer between nodes. With the first mechanism, based on the heart- beats received during the early state of the job, the task scheduler can dynamically estimate the computational capacity of each node. Us- ing the second mechanism, unprocessed Tasks local to each compute node are reassigned and reserved to allow nodes with greater capacities to reserve more local tasks than their weaker counterparts. Experimental results show that HMTS-DC performs better than Hadoop and Dynamic Data Placement Strategy （DDP） in a dynamic environment. Furthermore, an en- hanced HMTS-DC （EHMTS-DC） is proposed bv incorporatin historical data. In contrastto the ＂slow start＂ property of HMTS-DC, EHMTS-DC relies on the historical computation capacity of the slave machines. The experimental results show that EHMTS-DC outperforms HMTS-DC in a dynamic environment.展开更多
Programs take on changing behavior at nmtime in a simultaneous multithreading （SMT） environment. How reasonably common resources are distributed among the threads significantly determines the throughput and fairness...Programs take on changing behavior at nmtime in a simultaneous multithreading （SMT） environment. How reasonably common resources are distributed among the threads significantly determines the throughput and fairness performance in SMT processors. Existing resource distribution methods either mainly rely on the front-end fetch policy, or make distribution decisions according to the limited information from the pipeline. It is difficult for them to efficiently catch the various resource requirements of the threads. This work presents a spatially triggered dissipative resource distribution （SDRD） policy for SMT processors, its two parts, the self-organization mechanism that is driven by the real-time instructions per cycle （IPC） performance and the introduction of chaos that tries to control the diversity Of trial resource distributions, work together to supply sustaining resource distribution optimization for changing program behavior. Simulation results show that SDRD with fine-grained diversity controlling is more effective than that with a coarse-grained one. And SDRD benefits much from its two well-coordinated parts, providing potential fairness gains as well as good throughput gains. Meanings and settings of important SDRD parameters are also discussed.展开更多
In order to improve the concurrent access performance of the web-based spatial computing system in cluster,a parallel scheduling strategy based on the multi-core environment is proposed,which includes two levels of pa...In order to improve the concurrent access performance of the web-based spatial computing system in cluster,a parallel scheduling strategy based on the multi-core environment is proposed,which includes two levels of parallel processing mechanisms.One is that it can evenly allocate tasks to each server node in the cluster and the other is that it can implement the load balancing inside a server node.Based on the strategy,a new web-based spatial computing model is designed in this paper,in which,a task response ratio calculation method,a request queue buffer mechanism and a thread scheduling strategy are focused on.Experimental results show that the new model can fully use the multi-core computing advantage of each server node in the concurrent access environment and improve the average hits per second,average I/O Hits,CPU utilization and throughput.Using speed-up ratio to analyze the traditional model and the new one,the result shows that the new model has the best performance.The performance of the multi-core server nodes in the cluster is optimized;the resource utilization and the parallel processing capabilities are enhanced.The more CPU cores you have,the higher parallel processing capabilities will be obtained.展开更多
Systems that exhibit complex behaviours are often found in a particular dynamical con- dition, poised between order and disorder. This observation is at the core of the so-called criticality hypothesis, which states t...Systems that exhibit complex behaviours are often found in a particular dynamical con- dition, poised between order and disorder. This observation is at the core of the so-called criticality hypothesis, which states that systems in a dynamical regime between order and disorder attain the highest level of computational capabilities and achieve an optimal trade-off between robustness and flexibility. Recent results in cellular and evolutionary biology, ueuroscience and computer science have revitalised the interest in the criticality hypothesis, emphasising its role as a viable candidate general law in adaptive complex systems. This paper provides an overview of the works on dynamical criticality that are -- To the best of our knowledge -- Particularly relevant for the criticality hypothesis. The authors review the main contributions concerning dynamics and information processing at the edge of chaos, and illustrate the main achievements in the study of critical dynamics in biological systems. Finally, the authors discuss open questions and propose an agenda for future work.展开更多
文摘MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a heartbeat mechanism for MapReduce Task Scheduler using Dynamic Calibration （HMTS- DC） is proposed to address the unbalanced node computation capacity problem in a het- erogeneous MapReduce environment. HMTS- DC uses two mechanisms to dynamically adapt and balance tasks assigned to each com- pute node： 1） using heartbeat to dynamically estimate the capacity of the compute nodes, and 2） using data locality of replicated data blocks to reduce data transfer between nodes. With the first mechanism, based on the heart- beats received during the early state of the job, the task scheduler can dynamically estimate the computational capacity of each node. Us- ing the second mechanism, unprocessed Tasks local to each compute node are reassigned and reserved to allow nodes with greater capacities to reserve more local tasks than their weaker counterparts. Experimental results show that HMTS-DC performs better than Hadoop and Dynamic Data Placement Strategy （DDP） in a dynamic environment. Furthermore, an en- hanced HMTS-DC （EHMTS-DC） is proposed bv incorporatin historical data. In contrastto the ＂slow start＂ property of HMTS-DC, EHMTS-DC relies on the historical computation capacity of the slave machines. The experimental results show that EHMTS-DC outperforms HMTS-DC in a dynamic environment.
基金the Hi-Tech Research and Development Pro-gram (863) of China (No. 2006AA01Z431) the Key Science andTechnology Program of Zhejiang Province (Nos. 2007C11068 and2007C11088), China
文摘Programs take on changing behavior at nmtime in a simultaneous multithreading （SMT） environment. How reasonably common resources are distributed among the threads significantly determines the throughput and fairness performance in SMT processors. Existing resource distribution methods either mainly rely on the front-end fetch policy, or make distribution decisions according to the limited information from the pipeline. It is difficult for them to efficiently catch the various resource requirements of the threads. This work presents a spatially triggered dissipative resource distribution （SDRD） policy for SMT processors, its two parts, the self-organization mechanism that is driven by the real-time instructions per cycle （IPC） performance and the introduction of chaos that tries to control the diversity Of trial resource distributions, work together to supply sustaining resource distribution optimization for changing program behavior. Simulation results show that SDRD with fine-grained diversity controlling is more effective than that with a coarse-grained one. And SDRD benefits much from its two well-coordinated parts, providing potential fairness gains as well as good throughput gains. Meanings and settings of important SDRD parameters are also discussed.
基金Supported by the China Postdoctoral Science Foundation(No.2014M552115)the Fundamental Research Funds for the Central Universities,ChinaUniversity of Geosciences(Wuhan)(No.CUGL140833)the National Key Technology Support Program of China(No.2011BAH06B04)
文摘In order to improve the concurrent access performance of the web-based spatial computing system in cluster,a parallel scheduling strategy based on the multi-core environment is proposed,which includes two levels of parallel processing mechanisms.One is that it can evenly allocate tasks to each server node in the cluster and the other is that it can implement the load balancing inside a server node.Based on the strategy,a new web-based spatial computing model is designed in this paper,in which,a task response ratio calculation method,a request queue buffer mechanism and a thread scheduling strategy are focused on.Experimental results show that the new model can fully use the multi-core computing advantage of each server node in the concurrent access environment and improve the average hits per second,average I/O Hits,CPU utilization and throughput.Using speed-up ratio to analyze the traditional model and the new one,the result shows that the new model has the best performance.The performance of the multi-core server nodes in the cluster is optimized;the resource utilization and the parallel processing capabilities are enhanced.The more CPU cores you have,the higher parallel processing capabilities will be obtained.
文摘Systems that exhibit complex behaviours are often found in a particular dynamical con- dition, poised between order and disorder. This observation is at the core of the so-called criticality hypothesis, which states that systems in a dynamical regime between order and disorder attain the highest level of computational capabilities and achieve an optimal trade-off between robustness and flexibility. Recent results in cellular and evolutionary biology, ueuroscience and computer science have revitalised the interest in the criticality hypothesis, emphasising its role as a viable candidate general law in adaptive complex systems. This paper provides an overview of the works on dynamical criticality that are -- To the best of our knowledge -- Particularly relevant for the criticality hypothesis. The authors review the main contributions concerning dynamics and information processing at the edge of chaos, and illustrate the main achievements in the study of critical dynamics in biological systems. Finally, the authors discuss open questions and propose an agenda for future work.
