高校网络日志大数据分析平台研究

Research on big data analysis platform of massive campus network logs

许多高校面临着用户快速增长、网络数据庞大、信息安全隐患等多方面的问题,校园网每天生成NAT、DNS、Web、VPN等海量网络日志,对网络IT运维管理和大数据处理提出挑战。通过介绍一种基于大数据的海量网络日志分析平台,研究大数据分析平台的技术架构与总线模块,实现了对校园网海量日志进行统一收集与大数据分析。

Many colleges and universities are faced with the problems of rapid growth of users, massive network data and information security risks etc., the campus network generates massive network logs of NAT, DNS, Web, VPN and so on, it challenges the network IT Operations Management （ITOM） and big data processing. This paper introduces a massive network logs analysis platform based on the big data technology, studies the technology architecture and bus module of big data analysis platform, and realizes the collection and big data analysis of the campus massive network logs.