ECLHadoop:基于Hadoop的有效电子商务物流大数据处理策略

ECLHadoop:efficient big data processing strategy based on Hadoop for electronic commerce logistics

随着云计算的快速发展,越来越多的电子商务服务应用面临处理大数据的要求,例如电子商务物流服务中顾客通过社会媒体发布而产生的大量数据。为提高电子商务物流大数据的处理效率,基于Hadoop设计了一种称为ECLHadoop的有效电子商务物流大数据处理策略,通过将相关的数据块放入相同的数据节点,进而达到降低MapReduce I/O代价的目的,尤其是降低shuffling阶段的I/O代价。仿真实验结果显示,基于Hadoop的ECLHadoop大数据处理策略能够较好地进行电子商务物流服务中的数据密集型分析,提高电子商务物流大数据计算效率。

With the rapid development of cloud computing,more and more electronic commerce applications are confronted with the problems of processing big data,such as big data from the social media posted by the customers of electronic commerce logistics.In order to improve the big data processing efficiency in electronic commerce logistics,an efficient big data processing strategy based on Hadoop is designed,which is named ECLHadoop.In ECLHadoop,those closely related data blocks are placed at the same nodes,which can help to reduce the MapReduce I/O cost,especially the I/O cost at the shuffling stage.The simulation experiment results show that,based on Hadoop,the ECLHadoop can improve the big data computing efficiency for data-intensive analysis in the electronic commerce logistics service.