摘要
MapReduce编程模型是广泛应用于云计算环境下处理海量数据的一种并行计算框架。然而该框架下的面向数据密集型计算,集群节点间的数据传输依赖性较强,造成节点间的消息处理负载过重。提出基于消息代理机制的MapReduce改进模型,优化数据流。经实验数据表明,基于消息代理机制的MapReduce框架能提高数据密集型应用上的负载均衡。
MapReduce programming model is a kind of parallel computing framework which is distributed under the environ- ment of mass data processing system. Currently, the MapReduce applications are widely used for commercial data intensive computing, the data transmission between the nodes on cluster has a large extent dependence. It causes that the load of message handling between the nodes is heavy. This paper puts forward an improved model of MapReduce based on message broker mechanism, to optimize the MapReduce data flow. The experimental data indicates that based on message broker mechanism the MapReduce framework can improve the load balance in data intensive applications.
出处
《计算机工程与应用》
CSCD
2013年第5期120-122,262,共4页
Computer Engineering and Applications
