集群环境下Syslog管理系统设计

Design of Syslog Management System in Cluster Environment

日志是系统安全和计算机管理的重要依据,通过分析日志可以迅速定位和解决问题。但一般情况下,日志分散保存在不同的设备,并且具有数据量大的特点。在集群环境下,为了实现日志的集中化储存和管理,在Syslog日志管理系统的基础上,通过对开源方案ELK的裁剪和定制开发,实现了日志的采集、存储、筛选和分析。重点基于配置规则实现了基于多元组规则和日志msg内容正则表达式的日志二次过滤,再对日志进行封装,上报机器备份。系统测试结果表明,二次过滤可以自动过滤掉大量日志,从而更快定位问题。可视化的界面更加方便管理。日志接收能力大于每秒100条,日志检索时间在秒级,能够保证长期稳定运行,在企业的运维管理中具有很高的应用价值。

This paper based on the syslog log management system realizes the collection,storage,filtering and analysis of log data by cutting and customizing the open source tool ELK.Through the configuration rules,the secondary log filtering based on the multivariate rules and the regular expression of log msg content is realized,then the log is encapsulated and transmitted to the higher server.The experimental results of this system show that the secondary filter can automatically filter out a large number of logs,so as to locate the problem faster.