申威平台高速网络数据处理框架的设计与实现

Design and Implementation of High Speed Network Data Processing Framework Based on Domestic Sunway Processors

随着大数据时代网络流量的激增,传统内核网络协议栈由于内核切换开销占比高等原因导致现有基于内核的网络数据处理系统无法充分利用10 Gb乃至100 Gb的高速网卡收发能力。为了降低内核切换开销,开源DPDK用户态网络开发套件被提出以支持高速网络流量处理,并在x86平台得到大规模应用和部署。为了满足国产化信创和网络安全的要求,面向国产申威处理器平台设计并实现了一套基于DPDK的网络流量组包解析框架,充分利用DPDK的大页内存、无锁队列等机制,设计多线程并行以发挥申威处理器多核性能,支持常见基于TCP/UDP的多种应用层协议解析,并具有轻量化和可扩展特点。基于真实硬件平台实验结果表明,该框架性能比现有主流软件提高10%左右,为基于国产处理器平台的高速网络数据处理做了初步探索。

With the surge of network traffic in the era of big data,the traditional kernel-based network stack cannot fully utilize the high-speed NIC transceiver capability of 10 Gb or even 100 Gb due to the high kernel switch overhead of traditional kernel network stack.To reduce the software overhead,the open-source DPDK userspace network development kit is developed to support high-speed network application,which has been massively applied and deployed on x86 platforms.To meet the requirements of localization and national network security,we design and implement a DPDK-based IP packet parsing framework for the domestic Sunway processor platform,which make full use of DPDK’s large-page memory,lock-free queue and other advance mechanisms and take advantage of the multi-core performance of the Sunway processor,which supports parsing of multiple user layer protocols based on TCP/IP stack,and has lightweight and scalable features.According to the experimental results on the real hardware platform,the performance of the proposed framework is about 10%higher than that of the existing mainstream software,which is a preliminary exploration for the application of the DPDK framework in domestic processors.