利用深度学习的高速网络流基数估计算法

High-speed Network Flow Cardinality Estimation Algorithm Using Deep Learning

实时准确地测量高速网络中每条流的基数,对流量工程、异常检测及网络安全等领域来说至关重要.但网络处理器芯片中能够匹配流速处理需求的高速存储资源极度有限,无法满足流信息直接存储的需求,需要不同流共享同一段存储空间,利用紧凑数据摘要(Sketch)实时处理和存储流的基数信息.这种存储共享的方式使得不同流的信息混杂在一起,并在流基数估计的过程中引入了难以过滤的噪声.针对上述现有研究中普遍存在的问题,提出了一种基于深度学习的高速网络流基数估计算法.所设计的算法改进了已有研究的实时数据包处理和存储更新规则,设计了一种更为高效的编码方法在存储共享的基础上尽可能地减少噪声,并利用深度学习模型来学习每条流编码数据中的潜在模式以提高基数估计的性能.实验结果表明,所提出的方法相较于已有最新研究成果具有更高的精度和更低的存储开销.

Accurately measuring per-flow cardinality in high-speed networks plays an essential role in traffic engineering,anomaly detection,and network security.However,on-chip memory space in network processor chips is extremely limited,which cannot meet the requirements of recording the network traffic information directly.It is necessary to record distinct flows in the same space and use compact data summary(called Sketch)to process and save flows information in real-time.This way of processing makes different flows intermixed together and leads to difficult-to-filter noise in per-flow cardinality estimation.We propose a learning-based enhanced per-flow cardinality estimation algorithm to address the common problems in the above research.The proposed algorithm improves the existing research on real-time packet processing and memory update rules,develops a more efficient encoding method to reduce noise as much as possible based on memory sharing.In addition,our algorithm uses deep learning models to learn latent patterns from per-flow encoded data to improve the performance of per-flow cardinality estimation.Experimental results show that our solution has higher accuracy and lower memory overhead than vHLL.