基于云平台的MapReduce性能优化策略

MapReduce performance optimization strategy based on a cloud platform

设计了基于云平台架构的M印Reduce性能优化策略,全面考虑MapReduce作业过程中的数据传输与数据处理流程,将虚拟网络拓扑结构的设计描述成一个优化问题,并构建模型实现了通信代理数量、通信代理的放置位置以及虚拟机与通信代理之间的映射关系,以解决目前大多数研究只单方面考虑平台的数据处理或数据传输性能的缺陷.实验结果表明,与随机匹配策略和贪心策略相比,本方案优化了云计算系统的虚拟网络拓扑结构,减少了数据传输与处理的时间总开销,显著地提高了大数据处理的整体性能.

A MapReduce performance optimization strategy bassd on cloud platform architecture was designed,and both data transmission and data processing in the process of MapReduce were considered.The design of virtual network topology was described as an optimization problem,and the model was constructed to realize the optimal number of communication agents,optimal placement of each communication agent and the optimal matching between virtual machines and communication agents in order to solve the problem that only the performance of data processing or data transmission has been considered in current studies.The experimental results show that,compared with random matching strategy and greedy strategy,our topology optimization mechanism can optimize virtual network topology of cloud computing systems,reduce the cost of the total time for data communication and data processing and improve the overall performance of cloud-based big data applications dramatically.