前置包级别采样网络流基数估计算法的研究

Research on preposition packet-level sampling for high-speed network flow cardinality estimation

在网络管理中,高速网络流量测量中的网络流基数测量可为扫描攻击检测、性能诊断和异常检测等重要网络功能提供分析依据,具有十分重要的地位。然而,随着网络设备数量和链路速率的急剧增长,实时测量数百万甚至更高量级的流量是十分困难的。由于缺乏昂贵的专用硬件支持(如智能网卡),交换机上严重受限的计算和存储资源将导致数据包的处理速度下降,进而使得现有测量方法的吞吐量过低。针对该问题,有研究者采用元素级别采样技术以缩减原始流量数据。但现有的元素级别采样技术需要解析数据包并进行哈希计算,而这两种操作会消耗大量的时间,导致吞吐量的改善效果不显著。对此,文章提出了前置包级别采样网络流基数估计算法,该算法仅利用一个计数器即可完成包级别采样判断,无需进行包解析等复杂计算,有效缩短了平均采样处理时间,大幅度提高了吞吐量。同时,该算法利用概率分析对原估计结果进行了修正,有效提高了估计精度。实验结果表明,相较于其他测量方法,该算法可在保证估计精度的前提下使吞吐量提高一倍。

In network management,the flow cardinality estimation in high-speed network traffic measurement can provide analytical basis for important network functions such as scanning attack detection,performance diagnosis,and anomaly detection,and has a very important position.However,with the rapid increase in the number of network devices and link rates,real-time measurement of millions or even higher orders of traffic is very difficult.Due to the lack of expensive dedicated hardware support(such as smart network cards),severely limited computing and storage resources on switches will lead to a decrease in packet processing speed,resulting in low throughput of existing measurement methods.In response to this issue,some researchers have adopted element level sampling techniques to reduce the original traffic data.However,existing element level sampling techniques require parsing data packets and performing hash calculations,which can consume a significant amount of time and result in insignificant improvement in throughput.In this regard,the article proposes a pre packet-level sampling network flow cardinality estimation algorithm,which can complete packet level sampling judgment with only one counter and does not require complex calculations such as packet parsing.This algorithm effectively shortens the average sampling processing time and significantly improves throughput.At the same time,the algorithm utilizes probability analysis to modify the original estimation results,effectively improving the estimation accuracy.The experimental results show that compared to other measurement methods,this algorithm can double the throughput while ensuring estimation accuracy.