基于rabbitmq的海量日志的分布式处理

Base on the rabbitmq massive distributed processing of logs

近年来,随着科学技术的发展,各种设备日志层出不穷,然而传统的基于单节点的集中式日志分析已经无法满足日志海量、异构、多样等要求,何有效的利用分布式计算资源解决海量日志分析问题成为当下日志研究的一个热点问题。本文提出了基于消息队列rabbitmq的集群方案,并在此基础上测试了海量数据的分布式计算.对比实验结果说明集群优势可以有效的解决日志分析海量数据的问题。

With the development of science and techonlogy, there are kinds of equipment log, however the tranditional based on single node centralized log analysis is already can not satisfy the massive, heterogeneous and diverse requirements of log analysis. And how to effectively use the distributed computing resources to solve the massive log analysis problem is becoming a hotspot in the current research of log. based on the message queues rabbitmq cluster solution is proposed in this paper, and a lot of data distributed computing is achieved. Experiment results show that the advantages of cluster can solve the problem of huge amounts of data log analysis effectively.