Method for improving MapReduce performance by prefetching before scheduling

In this paper, a prefetching technique is proposed to solve the performance problem caused by remote data access delay. In the technique, the map tasks which will cause the delay are predicted first and then the input data of these tasks will be preloaded before the tasks are scheduled. During the execution, the input data can be read from local nodes. Therefore, the delay can be hidden. The technique has been implemented in Hadoop-0. 20.1. The experiment results have shown that the technique reduces map tasks causing delay, and improves the performance of Hadoop MapRe- duce by 20%.

大规模关系图增量计算框架研究

随着大数据时代的到来,关系图的规模日益扩大,现有的系统已经无法满足应用程序对图数据处理的及时性要求。提出了增量图计算原理,然后将关系图快照,增量数据计算以及数据延迟加载等机制加入到图计算过程中,并在此基础上提出了支持大规模关系图增量计算的框架,并对框架中的实现技术和核心模块进行阐述。