Optimizing Memory Access Efficiency in CUDA Kernel via Data Layout Technique

Over the past decade, Graphics Processing Units (GPUs) have revolutionized high-performance computing, playing pivotal roles in advancing fields like IoT, autonomous vehicles, and exascale computing. Despite these advancements, efficiently programming GPUs remains a daunting challenge, often relying on trial-and-error optimization methods. This paper introduces an optimization technique for CUDA programs through a novel Data Layout strategy, aimed at restructuring memory data arrangement to significantly enhance data access locality. Focusing on the dynamic programming algorithm for chained matrix multiplication—a critical operation across various domains including artificial intelligence (AI), high-performance computing (HPC), and the Internet of Things (IoT)—this technique facilitates more localized access. We specifically illustrate the importance of efficient matrix multiplication in these areas, underscoring the technique’s broader applicability and its potential to address some of the most pressing computational challenges in GPU-accelerated applications. Our findings reveal a remarkable reduction in memory consumption and a substantial 50% decrease in execution time for CUDA programs utilizing this technique, thereby setting a new benchmark for optimization in GPU computing.

Research on optimization of virtual machine memory access based on NUMA architecture

With the rapid development of big data and artificial intelligence(AI),the cloud platform architecture system is constantly developing,optimizing,and improving.As such,new applications,like deep computing and high-performance computing,require enhanced computing power.To meet this requirement,a non-uniform memory access(NUMA)configuration method is proposed for the cloud computing system according to the affinity,adaptability,and availability of the NUMA architecture processor platform.The proposed method is verified based on the test environment of a domestic central processing unit(CPU).

Optimized CUDA Implementation to Improve the Performance of Bundle Adjustment Algorithm on GPUs

The 3D reconstruction pipeline uses the Bundle Adjustment algorithm to refine the camera and point parameters. The Bundle Adjustment algorithm is a compute-intensive algorithm, and many researchers have improved its performance by implementing the algorithm on GPUs. In the previous research work, “Improving Accuracy and Computational Burden of Bundle Adjustment Algorithm using GPUs,” the authors demonstrated first the Bundle Adjustment algorithmic performance improvement by reducing the mean square error using an additional radial distorting parameter and explicitly computed analytical derivatives and reducing the computational burden of the Bundle Adjustment algorithm using GPUs. The naïve implementation of the CUDA code, a speedup of 10× for the largest dataset of 13,678 cameras, 4,455,747 points, and 28,975,571 projections was achieved. In this paper, we present the optimization of the Bundle Adjustment algorithm CUDA code on GPUs to achieve higher speedup. We propose a new data memory layout for the parameters in the Bundle Adjustment algorithm, resulting in contiguous memory access. We demonstrate that it improves the memory throughput on the GPUs, thereby improving the overall performance. We also demonstrate an increase in the computational throughput of the algorithm by optimizing the CUDA kernels to utilize the GPU resources effectively. A comparative performance study of explicitly computing an algorithm parameter versus using the Jacobians instead is presented. In the previous work, the Bundle Adjustment algorithm failed to converge for certain datasets due to several block matrices of the cameras in the augmented normal equation, resulting in rank-deficient matrices. In this work, we identify the cameras that cause rank-deficient matrices and preprocess the datasets to ensure the convergence of the BA algorithm. Our optimized CUDA implementation achieves convergence of the Bundle Adjustment algorithm in around 22 seconds for the largest dataset compared to 654 seconds for the sequential implementation, resulting in a speedup of 30×. Our optimized CUDA implementation presented in this paper has achieved a 3× speedup for the largest dataset compared to the previous naïve CUDA implementation.

Memory Access Optimization of Molecular Dynamics Simulation Software Crystal-MD on Sunway Taihu Light

The radiation damage effect of key structural materials is one of the main research subjects of the numerical reactor.From the perspective of experimental safety and feasibility,Molecular Dynamics(MD)in the materials field is an ideal method for simulating the radiation damage of structural materials.The Crystal-MD represents a massive parallel MD simulation software based on the key material characteristics of reactors.Compared with the Large-scale Atomic/Molecurlar Massively Parallel Simulator(LAMMPS)and ITAP Molecular Dynamics(IMD)software,the Crystal-MD reduces the memory required for software operation to a certain extent,but it is very time-consuming.Moreover,the calculation results of the Crystal-MD have large deviations,and there are also some problems,such as memory limitation and frequent communication during its migration and optimization.In this paper,in order to solve the above problems,the memory access mode of the Crystal-MD software is studied.Based on the memory access mode,a memory access optimization strategy is proposed for a unique architecture of China’s supercomputer Sunway Taihu Light.The proposed optimization strategy is verified by the experiments,and experimental results show that the running speed of the Crystal-MD is increased significantly by using the proposed optimization strategy.

Memory access optimization for particle operations in computational fluid dynamics-discrete element method simulations

Computational Fluid Dynamics-Discrete Element Method is used to model gas-solid systems in several applications in energy,pharmaceutical and petrochemical industries.Computational performance bot-tlenecks often limit the problem sizes that can be simulated at industrial scale.The data structures used to store several millions of particles in such large-scale simulations have a large memory footprint that does not fit into the processor cache hierarchies on current high-performance-computing platforms,leading to reduced computational performance.This paper specifically addresses this aspect of memory access bottlenecks in industrial scale simulations.The use of space-flling curves to improve memory access patterns is described and their impact on computational performance is quantified in both shared and distributed memory parallelization paradigms.The Morton space flling curve applied to uniform grids and k-dimensional tree partitions are used to reorder the particle data-structure thus improving spatial and temporal locality in memory.The performance impact of these techniques when applied to two benchmark problems,namely the homogeneous-cooling-system and a fluidized-bed,are presented.These optimization techniques lead to approximately two-fold performance improvement in particle focused operations such as neighbor-list creation and data-exchange,with~1.5 times overall improvement in a fluidization simulation with 1.27 million particles.

基于重用距离的访存指令优化

随着机器学习、推荐系统和社交网络等数据驱动类技术的发展,数据正在以流的形式呈现.传统的缓存替换算法无法有效适应应用程序的流式访问行为,导致数据流程序带来了大量的缓存未命中与严重的缓存污染问题.本文依据数据流程序变化带来的新的局部性优化挑战,提出了一种基于重用距离和非时态访存指令的优化方法RDNT.该方法首先筛选内存访问指令,然后计算重用距离,最后用非时态内存访问指令替换重用距离过大的常规内存访问指令.在SPEC CPU 2017测试集的实验结果表明,RDNT能够有效提高程序性能,与常规访存方式相比产生了8%的加速比,降低了程序的运行时间.

swLLVM:面向神威新一代超级计算机的优化编译器

异构众核架构具有超高的能效比,已成为超级计算机体系结构的重要发展方向.然而,异构系统的复杂性给应用开发和优化提出了更高要求,其在发展过程中面临好用性和可编程性等众多技术挑战.我国自主研制的神威新一代超级计算机采用了国产申威异构众核处理器SW26010Pro.为了发挥新一代众核处理器的性能优势,支撑新兴科学计算应用的开发和优化,设计并实现面向SW26010Pro平台的优化编译器swLLVM.该编译器支持Athread和SDAA双模态异构编程模型,提供多级存储层次描述及向量操作扩展,并且针对SW26010Pro架构特点实现控制流向量化、基于代价的节点合并以及针对多级存储层次的编译优化.测试结果表明,所设计并实现的编译优化效果显著,其中,控制流向量化和节点合并优化的平均加速比分别为1.23和1.11,而访存相关优化最高可获得2.49倍的性能提升.最后,使用SPEC CPU2006标准测试集从多个维度对swLLVM进行了综合评估,相较于SWGCC的相同优化级别,swLLVM整型课题性能平均下降0.12%,浮点型课题性能平均提升9.04%,整体性能平均提升5.25%,编译速度平均提升79.1%,代码尺寸平均减少1.15%.

A Study on Modeling and Optimization of Memory Systems

Accesses Per Cycle(APC),Concurrent Average Memory Access Time(C-AMAT),and Layered Performance Matching(LPM)are three memory performance models that consider both data locality and memory assess concurrency.The APC model measures the throughput of a memory architecture and therefore reflects the quality of service(QoS)of a memory system.The C-AMAT model provides a recursive expression for the memory access delay and therefore can be used for identifying the potential bottlenecks in a memory hierarchy.The LPM method transforms a global memory system optimization into localized optimizations at each memory layer by matching the data access demands of the applications with the underlying memory system design.These three models have been proposed separately through prior efforts.This paper reexamines the three models under one coherent mathematical framework.More specifically,we present a new memorycentric view of data accesses.We divide the memory cycles at each memory layer into four distinct categories and use them to recursively define the memory access latency and concurrency along the memory hierarchy.This new perspective offers new insights with a clear formulation of the memory performance considering both locality and concurrency.Consequently,the performance model can be easily understood and applied in engineering practices.As such,the memory-centric approach helps establish a unified mathematical foundation for model-driven performance analysis and optimization of contemporary and future memory systems.

CPU和GPU协同处理的光学卫星遥感影像正射校正方法

系统地探讨基于CPU和GPU协同处理的光学卫星遥感影像正射校正方法。首先使用"层次性分块"策略设计基于CPU和GPU协同处理的正射校正方法,然后通过配置选择优化和存储层次性访问等手段进一步提高方法执行效率。在Tesla M2050 GPU上对资源三号卫星下视全色影像进行正射校正的试验结果表明,本文方法可大幅提高光学卫星遥感影像正射校正效率,与传统串行正射校正算法相比,加速比最高达到110倍以上,相应的处理时间压缩至5 s以内,可满足对海量数据光学卫星遥感影像进行快速正射校正的要求。

用于DDR3访存优化的数据缓冲机制

为提高DDR3控制器访存效率,设计了基于DDR3存储器预取访问数据长度的数据缓冲机制,将访存请求分为三种基本类型并分别排队处理,降低数据丢弃和实际动态随机访问存储器访问发生次数。针对图像和视频类应用程序的实验结果表明,相对于传统先到先服务的DDR3访存控制器,该机制取得了平均21.3%、最好51.3%的性能提升,硬件开销在可接受范围内。

一种支持多种访存技术的CBEA片上多核MPI并行编程模型

现有的CBEA（Cell Broadband Engine Architecture）编程模型多侧重于支持类似于流处理的“批量访存”（Bulk Data Transfer）应用,传统非规则访存应用性能较低.文中基于Cell架构提出了一种同时支持“批量访存”与非规则访存应用的MPI并行编程模型,将通信分解在PPE（PowerPC Processing Element）上,拓宽模型的适用范围;在统一访存接口下,通过运行时访存剖分信息指导选择和优化访存以提高计算效率.实验结果表明,文中提出的编程模型支持多种访存模式并具有很好的并行加速比,可获得较同类相关技术30%-50%左右的性能提升.

面向异构众核从核的数学函数库访存优化方法

数学库函数算法的特性致使函数存在大量的访存,而当前异构众核的从核结构采用共享主存的方式实现数据访问,从而严重影响了从核的访存速度,因此异构众核结构中数学库函数的性能无法满足高性能计算的要求。为了有效解决此问题,提出了一种基于访存指令的调度策略,亦即将访存延迟有效地隐藏于计算延迟中,以提高基于汇编实现的数学函数库的函数性能;结合动态调用方式,利用从核本地局部数据存储空间LDM(local data memory),提出了一种提高访存速度的ldm_call算法。两种优化技术在共享存储结构下具有普遍适用性,并能够有效减少函数访存开销,提高访存速度。实验表明,两种技术分别能够平均提高函数性能16.08%和37.32%。

NVIDIA Tegra K1异构计算平台访存优化研究

在异构计算平台的移植和优化过程中,数字图像处理算法的访存性能已成为制约系统性能的主要因素。为此,结合NVIDIA Tegra K1硬件架构特征和具体算法特性,从合并与向量化访存优化、全局访存bank和channel冲突消除等方面,对矩阵转置算法和拉普拉斯滤波算法在NVIDIA Tegra K1异构计算平台上的实现和访存性能优化进行研究。实验结果表明,采用优化方法后的矩阵转置算法和拉普拉斯滤波算法在NVIDIA Tegra K1异构计算平台上取得了较大的访存性能提升,并且具有较好的实时性。

一种异构多核处理器的并行流存储结构

异构多核处理器可结合多种处理器体系结构的优势,既保留传统通用体系结构的灵活性,又拥有大量计算资源,可提供更高的峰值计算性能.YHFT64-3异构多核处理器中浮点处理部件18套,峰值计算能力强大,设计与之相匹配的存储系统是一项重大挑战.针对YHFT64-3处理器,本文提出了一种并行流层次存储结构,深入阐述了如何体现应用特点、支持并行数据流处理的存储系统的设计思想和方法,从多个层次实现对并行数据流的挖掘或捕获.测试结果表明,这种存储结构体现了应用特点,能够较好地发挥YHFT64-3处理器的性能,同频情况下(500MHz),YHFT64-3比YHFT64-2性能高2—3个数量级,与1.6GHz的Itanium2性能相当,但代价更低.

多核龙芯3A上二级BLAS库的优化

针对龙芯3A体系结构以及二级BLAS库函数的特点,在指令级、存储级和线程级抽取并行方案,总结了一些合适的优化方法,并对其进行了定量的分析。实验表明,这些优化可以将二级BLAS函数单线程的性能提升20%以上,多线程下也可以得到2.5倍左右的加速比,这对今后多核龙芯上的系统软件优化工作有着一定的帮助。

EPIC微体系结构的存储级并行执行模型的研究

描述了一种可以有效提高存储级并行(Memory Level Parallelism,MLP)的指令优化锁步执行模型———OLSM(Opti mized Lock-Step execution Model)执行模型,并建立了一种能体现OLSM模型思想的层次存储结构.OLSM允许显示并行指令计算(Explicit Parallel Instruction Computing,EPIC)微处理器实现一定程度的乱序执行,解决了传统超长指令字(Very Long Instruction Word,VLI W)锁步执行的缺陷,可以充分利用结构中的大量计算和存储资源,最大化隐藏存储延迟、提高MLP.

基于复杂网络理论的内存访问功耗优化研究

近年来随着移动处理、嵌入式应用的大量涌现,以及通用微处理器工艺水平和主频的不断提升,双核乃至四核的出现,功耗日益成为设计者必须关心的问题,功耗问题也已成为制约处理器发展的一个重要瓶颈。制约着嵌入式设备广泛应用的障碍不再是处理器的速度、芯片的工艺,而是设备功耗,如何在相同电池能量的供给情况下工作时间最长,成为嵌入式领域备受国内外关注的研究方向。因此,嵌入式的功耗优化问题就成为嵌入式发展亟待解决的重要问题。提出用复杂网络理论来降低存储器访问的功耗。

Canny边缘检测算法在飞腾平台上的实现与优化

为实现国产飞腾DSP平台对底层图像库的支持,针对原始Canny边缘检测算法计算时间过长的问题,设计一种面向FT-M7002平台的Canny梯度计算并行算法。基于FT-M7002高性能处理架构,采用单指令流多数据流向量化方式增强DSP内核指令的并行处理能力,根据FT-M7002平台向量存储器的层次结构特征,分析Canny梯度计算并行算法的访存模式,通过首地址偏移取址解决不连续访存问题,并结合双缓冲方式完成数据传输与数据计算。实验结果表明,在与原始Canny算法具有相同检测精度的情况下,该算法在卷积核大小为3×3、5×5、7×7时整体运行速度提升了1.490~2.112倍,缩小了与主流加速器件在数字图像处理领域的性能差距。

功耗受限情况下多核处理器能效优化方案

将处理器功耗控制在预算以下有助于降低散热成本和提升系统稳定性,但现有功耗优化方案大多依赖线下分析得到的先验知识,影响实用性,而集中式搜索最优策略的算法也存在复杂度过高的问题。为此,提出功耗优化方案PPCM。利用动态电压频率调整(DVFS)技术控制CPU功耗在预算内以提高处理器能效。同时,将功耗控制和功耗分配解耦合以提高灵活性。采用动态调整的线性模型估计功耗,通过反馈控制技术对其进行调节。以计算访存比为指标在应用间分配功耗,并考虑多线程应用特征进行线程间功耗分配。实验结果表明,PPCM比Priority算法速度平均提高10.7%,能耗平均降低5.1%,能量-延迟积平均降低14.3%。与PCM CA算法相比,其速度平均提高4.5%,能量-延迟积平均降低5.0%。

卷积神经网络训练访存优化

虽然批归一化算法能有效加速深度卷积网络模型的收敛速度,但其数据依赖性复杂,训练时会导致严重的“存储墙”瓶颈。故对使用批归一化算法的卷积神经网络,提出多层融合且重构批归一化层的训练方法,减少模型训练过程中的访存量。首先,通过分析训练时批归一化层的数据依赖、访存特征及模型训练时的访存特征,分析访存瓶颈的关键因素;其次,使用“计算换访存”思想,提出融合“卷积层+批归一化层+激活层”结构的方法,并基于批归一化层的计算访存特征,将其重构为两个子层,分别与相邻层融合,进一步减少训练时对主存的读写,并构建了训练时的访存量模型与计算量模型。实验结果表明,使用NVIDIA TESLA V100 GPU训练ResNet-50、Inception V3及DenseNet模型时,同原始训练方法相比,其访存数据量分别降低了33%,22%及31%,V100的实际计算效率分别提升了20.5%,18.5%以及18.1%。这种优化方法利用了网络结构与模型训练时的访存特点,可与其他访存优化方法协同使用,进一步降低模型训练时的访存量。