基于Hadoop的大数据论坛日志系统的分析与设计

Analysis and design of big data BBS logging system based on Hadoop

大数据时代的诞生,为解决挖掘海量数据存储的信息带来了技术上的革命,通过网站点击存留的日志数据,分析这些数据,可获知许多对网站运营至关重要的信息。本文采用Hadoop分布式平台,使用HDFS数据存储,利用Hive来分析日志数据,设计一种Web日志分析系统,并阐述了系统的设计过程。

The birth of the era of big data has brought a technological revolution to solve the problem of mining the information stored in massive data. By clicking on the retained log data and analyzing these data, we can get a lot of information which is very important to the operation of the website. This paper uses Hadoop distributed platform, HDFS data storage, Hive to analyze log data, designs a Web log analysis system, and describes the design process of the system.