基于Prometheus的油田数据泄漏防护大数据系统运维方法

Operation and Maintenance Method of Big Data System for Oilfield Data Leakage Prevention based on Prometheus

数据泄漏防护系统可以阻止企业的重要数据从网络出口违规外发。针对油田数据泄漏防护大数据平台对各个区域、不同类型的数据采集和设备、业务监测问题,本文首先提出依托大数据Flume(数据采集系统)组件,采用抓取外泄事件日志JSON(一种轻量级的数据交换格式)和解析后的违规外发文件txt文本的方法,实现数据采集;其次依托Prometheus服务监管系统Flume_exporter、Node_exporter组件和Grafana(一个跨平台的、开源的度量分析和可视化工具),实现对数据泄漏防护设备信息和应用数据的监管;最后指出搭建采集系统和监管平台对数据泄漏防护系统运维的作用。

Data leakage protection system can prevent important data of enterprises from being illegally exported from networks.This paper aims to solve problems of acquiring data of different types in different regions,and monitoring equipment and business on big data system for oilfield data leakage prevention.First,based on big data Flume(data acquisition system)component,this paper proposes to adopt the method of capturing JSON(JavaScript Object Notation,a lightweight data exchange format)of the leakage event log and the parsed illegally exported TXT file to achieve data acquisition.Second,Flume_exporter(Prometheus service supervision system),Node_exporter component and Grafana(a cross-platform open source measurement analysis and visualization tool)are used to supervise equipment information and application data of data leakage prevention.Finally,this paper points out the effect of building an acquisition system and supervision platform in operation and maintenance of data leakage prevention system.