MapReduce Scheduler by Characterizing Performance Interference

Recently, virtualization has become more and more important in the cloud computing to support efficient flexible resource provisioning. However, performance interference among virtual machines(VMs) has become a challenge which may affect the effectiveness of resource provisioning. In a virtual cluster which runs the Map Reduce applications, the performance interference can also affect the performance of the Map and Reduce tasks and thus cause a performance degradation of the Map Reduce job. Accordingly, this paper presents a Map Reduce scheduling framework to mitigate this performance degradation caused by the performance interference. The framework includes a performance interference prediction module and an interference aware scheduling algorithm. To verify its effectiveness, we have done a set of experiments on a 24-node virtual Map Reduce cluster. The experiments illustrate that the proposed framework can achieve a performance improvement in the virtualized environment compared with other Map Reduce schedulers.