NUMA架构内多个节点间访存延时平衡的内存分配策略

A Memory Allocation Policy for the Balance of Access Latency Among Multiple Memory Nodes in NUMA Architecture

随着多核架构的发展和普及,NUMA多核架构凭借其本地访存低延时的优势,被各大商业数据中心以及科学计算集群广泛采用.NUMA架构通过增加多个内存控制器,缓解了多核架构下对同一个内存控制器的争用问题,但同时也增加了内存管理的负担.Linux的系统开发者为了实现充分利用NUMA本地访存低延时的特点,在为进程分配内存时,选择进程当前正在运行的NUMA节点作为分配内存的目标节点.这种分配会导致进/线程之间共享内存的不公平.例如,一个在当前本地节点被分配很多内存的进程,可能被调度到远端节点运行,这样会导致进程的性能波动.针对这一问题,该文设计了一种保证NUMA架构内各内存节点间访存延时平衡的内存分配策略,并在Linux系统中实现和验证.延时的获取方法依赖平台,但是系统内核的策略是通用的.实验结果表明,与Linux默认的内存分配策略相比,进/线程间的不公平性平均降低了16%(最多34%),并且各进/线程的性能没有较大抖动.

Non-uniform memory access （NUMA） is a computer memory design used in multipro- cessing, where the memory access time depends on the memory location relative to the processor. Modern data centers and the scientific computing clusters widely adapt the NUMA architecture due to its low latency of local memory accessing, which is achieved by partitioning the whole memory into multiple nodes and each memory node is connected to a processors memory controller. However, this partitioning make the memory management much complicated. To achieve a low memory access latency, the default Linux memory policy chooses to allocate the physical memory pages from the memory node where the process is running. Unfortunately, this allocation policy may potentially result in the unfairness among the processes, e. g. , due to the process scheduling, one process with many local memory access may execute on remote node, which leads to the performance variation. To solve this problem, this paper proposes a node access latency indicator. Based on the indicator, we design a memory allocation policy assuring the balance of access latency among memory nodes in NUMA architecture, and implement it in the Linux kernel. To obtain the latency is platform dependent, but the policy in system kernel could be universal. The real system based evaluation results show that our proposal can reduce the unfairness up to 34% （with an average of 16%） while keeping a stable performance, compared to Linux＇s default memory allocation algorithm.