基于Hadoop架构的电信离线数据综合处理的设计与实现

Design and implementation of Telecom Offline data integrated processing based on Hadoop Architecture

本文研究的系统搭建在Hadoop平台上,通过Flume-Kafka技术实现对每日数百亿的数据进行数据清洗、数据分析以及数据挖掘等。完成数据消费后,对分析后的数据进行入库操作,通过Webserver技术实现建立仿真的BI前端系统,按照手机号、时间、通话时长等维度进行数据展示。为电信运营商从多个角度定义用户,形成用户肖像,为决策系统的建立提供数据支持。

The system studied in this paper is built on the Hadoop platform.Through Flume-Kafka technology,it can carry out data cleaning,data analysis and data mining of tens of billions of data every day.After data consumption is completed,the analyzed data is put into the database,and a simulated BI front-end system is established through Webserver technology,and data is displayed according to the dimensions of mobile phone number,time,call duration,etc.To define users from multiple angles for telecom operators,to form user portrait,to provide data support for the establishment of decision system.