共性数学库PETSc(Portable,Extensible Toolkit for Scientific Computation)是高性能计算的基础模块,是超级计算机计算环境的基础算法库之一,其性能直接影响调用数学库的高性能数值计算应用的效率.面向国际上首台100P神威·太湖之...共性数学库PETSc(Portable,Extensible Toolkit for Scientific Computation)是高性能计算的基础模块,是超级计算机计算环境的基础算法库之一,其性能直接影响调用数学库的高性能数值计算应用的效率.面向国际上首台100P神威·太湖之光异构超级计算机,根据实际研究需要选取PETSc中两个典型用例ex5(单节点线性求解方程组问题)和ex19(多节点求解2D驱动腔问题)进行实验探究.对运行结果分析找到的热点函数主要为PETSc函数库中7个核心函数,针对这7个核心函数(主要包括向量运算与矩阵运算),提出和实现了其异构并行算法,并结合机器的异构体系结构提出了相应的性能优化方法.在超级计算机上的实验结果为:核心函数并行算法在4主核、256从核的单节点上加速比最大可达到16.4;多节点情况下,当输入规模为16 384时,8192个节点相对于256节点的加速比为32,且加速比随着异构处理器数目的增加接近线性增加,表明PETSc核心函数并行算法在神威·太湖之光超级计算机上具有良好的可扩展性.展开更多
虽然批归一化算法能有效加速深度卷积网络模型的收敛速度,但其数据依赖性复杂,训练时会导致严重的“存储墙”瓶颈。故对使用批归一化算法的卷积神经网络,提出多层融合且重构批归一化层的训练方法,减少模型训练过程中的访存量。首先,通...虽然批归一化算法能有效加速深度卷积网络模型的收敛速度,但其数据依赖性复杂,训练时会导致严重的“存储墙”瓶颈。故对使用批归一化算法的卷积神经网络,提出多层融合且重构批归一化层的训练方法,减少模型训练过程中的访存量。首先,通过分析训练时批归一化层的数据依赖、访存特征及模型训练时的访存特征,分析访存瓶颈的关键因素;其次,使用“计算换访存”思想,提出融合“卷积层+批归一化层+激活层”结构的方法,并基于批归一化层的计算访存特征,将其重构为两个子层,分别与相邻层融合,进一步减少训练时对主存的读写,并构建了训练时的访存量模型与计算量模型。实验结果表明,使用NVIDIA TESLA V100 GPU训练ResNet-50、Inception V3及DenseNet模型时,同原始训练方法相比,其访存数据量分别降低了33%,22%及31%,V100的实际计算效率分别提升了20.5%,18.5%以及18.1%。这种优化方法利用了网络结构与模型训练时的访存特点,可与其他访存优化方法协同使用,进一步降低模型训练时的访存量。展开更多
快速傅里叶变换(fast Fourier transform,FFT)在数字信号处理中占据核心地位.随着高性能超长点数FFT需求的增长,数字信号处理器(digital signal processor,DSP)的计算能力越来越难以满足需求,集成FFT加速器成为重要的发展趋势.为了支持...快速傅里叶变换(fast Fourier transform,FFT)在数字信号处理中占据核心地位.随着高性能超长点数FFT需求的增长,数字信号处理器(digital signal processor,DSP)的计算能力越来越难以满足需求,集成FFT加速器成为重要的发展趋势.为了支持超长点数FFT,将2维分解算法推广到多维,提出一种可集成于DSP的高性能超长点数FFT加速器结构.该结构通过基于素数个存储体的无冲突体编址方法实现了3维转置运算;通过递推算法实现了高效铰链因子生成;使用单精度浮点二项融合点积运算和融合加-减运算,对FFT运算电路进行了精细化设计.实现了对4G点数单精度浮点FFT计算的支持.综合结果表明:FFT加速器运行频率能够达到1GHz以上,性能达到640Gflop/s.在支持的点数和性能方面都较已有研究成果取得大幅提升.展开更多
High performance computer(HPC) is a complex huge system,of which the architecture design meets increasing difficulties and risks.Traditional methods,such as theoretical analysis,component-level simulation and sequenti...High performance computer(HPC) is a complex huge system,of which the architecture design meets increasing difficulties and risks.Traditional methods,such as theoretical analysis,component-level simulation and sequential simulation, are not applicable to system-level simulations of HPC systems.Even the parallel simulation using large-scale parallel machines also have many difficulties in scalability,reliability,generality,as well as efficiency.According to the current needs of HPC architecture design,this paper proposes a system-level parallel simulation platform:ArchSim.We first introduce the architecture of ArchSim simulation platform which is composed of a global server(GS),local server agents(LSA) and entities.Secondly,we emphasize some key techniques of ArchSim,including the synchronization protocol,the communication mechanism and the distributed checkpointing/restart mechanism.We then make a synthesized test of some main performance indices of ArchSim with the phold benchmark and analyze the extra overhead generated by ArchSim.Finally,based on ArchSim,we construct a parallel event-driven interconnection network simulator and a system-level simulator for a small scale HPC system with 256 processors.The results of the performance test and HPC system simulations demonstrate that ArchSim can achieve high speedup ratio and high scalability on parallel host machine and support system-level simulations for the architecture design of HPC systems.展开更多
文摘虽然批归一化算法能有效加速深度卷积网络模型的收敛速度,但其数据依赖性复杂,训练时会导致严重的“存储墙”瓶颈。故对使用批归一化算法的卷积神经网络,提出多层融合且重构批归一化层的训练方法,减少模型训练过程中的访存量。首先,通过分析训练时批归一化层的数据依赖、访存特征及模型训练时的访存特征,分析访存瓶颈的关键因素;其次,使用“计算换访存”思想,提出融合“卷积层+批归一化层+激活层”结构的方法,并基于批归一化层的计算访存特征,将其重构为两个子层,分别与相邻层融合,进一步减少训练时对主存的读写,并构建了训练时的访存量模型与计算量模型。实验结果表明,使用NVIDIA TESLA V100 GPU训练ResNet-50、Inception V3及DenseNet模型时,同原始训练方法相比,其访存数据量分别降低了33%,22%及31%,V100的实际计算效率分别提升了20.5%,18.5%以及18.1%。这种优化方法利用了网络结构与模型训练时的访存特点,可与其他访存优化方法协同使用,进一步降低模型训练时的访存量。
文摘快速傅里叶变换(fast Fourier transform,FFT)在数字信号处理中占据核心地位.随着高性能超长点数FFT需求的增长,数字信号处理器(digital signal processor,DSP)的计算能力越来越难以满足需求,集成FFT加速器成为重要的发展趋势.为了支持超长点数FFT,将2维分解算法推广到多维,提出一种可集成于DSP的高性能超长点数FFT加速器结构.该结构通过基于素数个存储体的无冲突体编址方法实现了3维转置运算;通过递推算法实现了高效铰链因子生成;使用单精度浮点二项融合点积运算和融合加-减运算,对FFT运算电路进行了精细化设计.实现了对4G点数单精度浮点FFT计算的支持.综合结果表明:FFT加速器运行频率能够达到1GHz以上,性能达到640Gflop/s.在支持的点数和性能方面都较已有研究成果取得大幅提升.
基金supported by the National High Technology Research and Development 863 Program of China under Grant No. 2007AA01Z117the National Basic Research 973 Program of China under Grant No.2007CB310900
文摘High performance computer(HPC) is a complex huge system,of which the architecture design meets increasing difficulties and risks.Traditional methods,such as theoretical analysis,component-level simulation and sequential simulation, are not applicable to system-level simulations of HPC systems.Even the parallel simulation using large-scale parallel machines also have many difficulties in scalability,reliability,generality,as well as efficiency.According to the current needs of HPC architecture design,this paper proposes a system-level parallel simulation platform:ArchSim.We first introduce the architecture of ArchSim simulation platform which is composed of a global server(GS),local server agents(LSA) and entities.Secondly,we emphasize some key techniques of ArchSim,including the synchronization protocol,the communication mechanism and the distributed checkpointing/restart mechanism.We then make a synthesized test of some main performance indices of ArchSim with the phold benchmark and analyze the extra overhead generated by ArchSim.Finally,based on ArchSim,we construct a parallel event-driven interconnection network simulator and a system-level simulator for a small scale HPC system with 256 processors.The results of the performance test and HPC system simulations demonstrate that ArchSim can achieve high speedup ratio and high scalability on parallel host machine and support system-level simulations for the architecture design of HPC systems.
