面向网络数据实时检测的多线程内存管理技术

Multithreaded memory management in real-time network data detection

针对目前内存管理算法在多线程环境下存在的问题,提出了一种新的面向网络数据实时检测系统的多线程内存管理算法 MLMalloc。MLMalloc 算法采用线程局部内存区的方式解决了在多线程环境下内存锁的频繁竞争问题;预先在线程局部区和全局内存区分配大量内存,减少了在线程局部内存区、全局内存区和系统内存区之间的内存获取和归还抖动;线程局部内存区和全局内存区由固定大小的内存堆构成,在两种内存区之间采用预测的方式,一次锁操作分配或归还多个堆,减少了对内存区锁的竞争次数。实验结果表明,MLMalloc 算法显著地提高了多线程环境下大规模网络数据实时检测系统内存操作的效率。MLMalloc 算法的性能要优于 PTFMalloc 算法和 TCMalloc 算法,并且运行线程的数目越多,性能提高的幅度越大。

The paper proposes a new algorithm named MLMalloc （multithread local malloc） for the real-time network data detection. MLMalloe adopts the thread-local memory to avoid the contention for multithread memory lock. To significantly reduce the thrashing times in requesting and releasing the memory among thread-local memory, global memory and system memory, MLMalloe pre-allocate plenty of memory resouree in thread-local memory and global memory. The thread-local memory and global memory consist of fixed - size memory heaps. Multiple heaps will be got or returned once, so the times of operating memory lock decrease drastically. The experimental result shows that MLMalloe can greatly improve the performance in memory operation and perform better than the famous PTMalloc and TCMalloc, especially when more threads are running.