高效率多线程网络流量采集算法研究及实践

On the High Efficiency Multithreading Net Flow Collection Algorithm and Relevant Practice

介绍了采用SNMP协议采集CISCO路由器网络流量的方法,对流量采集算法进行了分析,提出一种高效率的流量采集算法.创建四个线程进行数据采集和处理:发送线程、接收线程、预处理线程和存储线程.发送线程负责发送SNMP请求报文,接收线程负责接收SNMP响应报文,预处理线程过滤及整合采集到的流量信息,存储线程负责将预处理后的数据持久地存入数据库中.由于采用并行方式进行数据采集,该算法较串行采集效率高,并在实践中验证其可行性.

Methods of using SNMP Protocol to collect net flow by addressing CISCO router are introduced. Net flow collection algorithms are carefully analyzed. An efficient net flow collection algorithm is proposed. Four threads are created to handle data collection and data processing whose functions are sending, receiving, pre - processing, and storing up the thread. Sending thread takes responsibility of sending SNMP request messages, while receiving thread takes charge of receiving SNMP response messages. Due to the fact of parallel data collection, the algorithm is sure of high efficiency. Pre - processing thread filtrates and processes raw data, and stores thread feeds data into databases. The approach has already been verified in practice.