大数据技术研究综述

Research overview of big data technology

大数据的产生给海量信息处理技术带来新的挑战.为了更全面深入地了解大数据的内涵,从大数据的概念特征、一般处理流程、关键技术三个方面进行详细阐述.分析了大数据的产生背景,简述了大数据的基本概念、典型的4"V"特征以及重点应用领域;归纳总结了大数据处理的一般流程,针对其中的关键技术,如MapReduce、GFS、BigTable、Hadoop以及数据可视化等,介绍了基本的处理过程和组织结构;具体分析指出了大数据时代所面临的问题与挑战.

The emergence of ＂big data＂ has brought new challenges to mass information processing technology. This comprehensive overview was intended to elaborate on big data from three aspects： the concept and characteristics, general data processing framework and key techniques. The background of big data was explained, and the basic concepts, typical 4＂V＂ characteristics as well as related application fields were sketched. Then, the general procedures of big data processing were summarized, and fundamental analysis and description of the key techniques, such as MapReduce, GFS, BigTable, Hadoop and data visualization, were given as well. Finally, the new issues and challenges in the Big Data Era were pointed out.