As an important branch of information technology, high-performance computing has expanded its application field and its influence has been expanding. High-performance computing is always a key area of application in m...As an important branch of information technology, high-performance computing has expanded its application field and its influence has been expanding. High-performance computing is always a key area of application in meteorology. We used field research and literature review methods to study the application of high performance computing in China’s meteorological department, and obtained the following results: 1) China Meteorological Department gradually established the first high-performance computer system since 1978. High-performance computing services can support operational numerical weather prediction models. 2) The Chinese meteorological department has always used the relatively advanced high-performance computing technology, and the business system capability has been continuously improved. The computing power has become an important symbol of the level of meteorological modernization. 3) High-performance computing technology and meteorological numerical forecasting applications are increasingly integrated, and continue to innovate and develop. 4) In the future, high-performance computing resource management will gradually transit from the current local pre-allocation mode to the local remote unified scheduling and shared use. In summary, we have come to the conclusion that the performance calculation business of the meteorological department will usher in a better tomorrow.展开更多
Cloud computing is expanding widely in the world of IT infrastructure. This is due partly to the cost-saving effect of economies of scale. Fair market conditions can in theory provide a healthy environment to reflect ...Cloud computing is expanding widely in the world of IT infrastructure. This is due partly to the cost-saving effect of economies of scale. Fair market conditions can in theory provide a healthy environment to reflect the most reasonable costs of computations. While fixed cloud pricing provides an attractive low entry barrier for compute-intensive applications, both the consumer and supplier of computing resources can see high efficiency for their investments by participating in auction-based exchanges. There are huge incentives for the cloud provider to offer auctioned resources. However, from the consumer perspective, using these resources is a sparsely discussed challenge. This paper reports a methodology and framework designed to address the challenges of using HPC （High Performance Computing） applications on auction-based cloud clusters. The authors focus on HPC applications and describe a method for determining bid-aware checkpointing intervals. They extend a theoretical model for determining checkpoint intervals using statistical analysis of pricing histories. Also the latest developments in the SpotHPC framework are introduced which aim at facilitating the managed execution of real MPI applications on auction-based cloud environments. The authors use their model to simulate a set of algorithms with different computing and communication densities. The results show the complex interactions between optimal bidding strategies and parallel applications performance.展开更多
A supercomputer with 1.0 Petaflops peak performance in single precision, designed and established by Institute of Process Engineering, Chinese Academy of Sciences, is introduced in this brief communication. A designin...A supercomputer with 1.0 Petaflops peak performance in single precision, designed and established by Institute of Process Engineering, Chinese Academy of Sciences, is introduced in this brief communication. A designing philosophy utilizing the similarity between hardware, software and the problems to be solved is embodied, based on the multi-scale method and discrete simulation approaches developed at Institute of Process Engineering （IPE） and implemented in a graphic processing unit （GPU）-based hybrid computing mode. The preliminary applications of this machine in areas of multi-phase flow, molecular dynamics and so on are reported, demonstrating the supercomputer as a paradigm of green computation in new architecture.展开更多
High performance computer （HPC） is a complex huge system, of which the architecture design meets increasing difficulties and risks. Traditional methods, such as theoretical analysis, component-level simulation and s...High performance computer （HPC） is a complex huge system, of which the architecture design meets increasing difficulties and risks. Traditional methods, such as theoretical analysis, component-level simulation and sequential simulation, are not applicable to system-level simulations of HPC systems. Even the parallel simulation using large-scale parallel machines also have many difficulties in scalability, reliability, generality, as well as efficiency. According to the current needs of HPC architecture design, this paper proposes a system-level parallel simulation platform： ArchSim. We first introduce the architecture of ArchSim simulation platform which is composed of a global server （GS）, local server agents （LSA） and entities. Secondly, we emphasize some key techniques of ArchSim, including the synchronization protocol, the communication mechanism and the distributed checkpointing/restart mechanism. We then make a synthesized test of some main performance indices of ArchSim with the phold benchmark and analyze the extra overhead generated by ArchSim. Finally, based on ArchSim, we construct a parallel event-driven interconnection network simulator and a system-level simulator for a small scale HPC system with 256 processors. The results of the performance test and HPC system simulations demonstrate that ArchSim can achieve high speedup ratio and high scalability on parallel host machine and support system-level simulations for the architecture design of HPC systems.展开更多
Objective:As a high computation cost discipline,nuclear science and engineering still relies heavily on traditional high performance computing(HPC)clusters.However,the usage of traditional HPC for nuclear science and ...Objective:As a high computation cost discipline,nuclear science and engineering still relies heavily on traditional high performance computing(HPC)clusters.However,the usage of traditional HPC for nuclear science and engineering has been limited due to the poor flexibility,the software compatibility and the poor user interfaces.Virtualized/virtual HPC(vHPC)can mimic an HPC by using a cloud computing platform.In this work,we designed and developed a vHPC system for employment in nuclear engineering.Methods:The system is tested using the computation of the numberπby Monte Carlo and an X-ray digital imaging system simulation.The performance of the vHPC system is compared with that of the traditional HPCs.Results:As the number of the simulated particles increases,the virtual cluster computing time grows propor-tionally.The time used for the simulation of the X-ray imaging was about 21.1 h over a 12 kernels virtual server.Experimental results show that the performance of virtual cluster computing and the actual physical machine is almost the same.Conclusions:From these tests,it is concluded that vHPC is a good alternative for employing in nuclear engineering.The proposed vHPC in this paper will make HPC flexible and easy to deploy.展开更多
HPCC(high performance computing challenge)基准是由DARPA的HPCS(high productivity computing system)项目所发布的评价高性能计算系统的测试基准程序,自推出至今,受到工业界和学术界的广泛关注.但是,HPCC仍有不尽如人意之处,主要表...HPCC(high performance computing challenge)基准是由DARPA的HPCS(high productivity computing system)项目所发布的评价高性能计算系统的测试基准程序,自推出至今,受到工业界和学术界的广泛关注.但是,HPCC仍有不尽如人意之处,主要表现在其测试结果是若干个指标项,需要测试者和决策者根据这些测试指标项进行分析和评估,缺少一个整体的、直观而统一的评价结果.提出一种基于HPCC和层次分析法的高性能计算系统评价模型——AHPCC(a high performance computer system evaluation model based on HPCC),当系统通过运行HPCC得到测试结果后,使用AHPCC模型对这些测试参数按系统应用目标建立层次结构图,并最终计算得到各系统关于特定应用目标的单一分数.以12个已测出HPCC性能参数的系统为例,使用AHPCC模型计算并分析了系统评价结果.实验结果表明,AHPCC模型提供了实际系统的统一而直观的评价指标,其评价结果符合高性能系统的设计和应用特点.展开更多
In this paper we present a 2D/3D high order accurate finite volume scheme in the context of direct Arbitrary-Lagrangian-Eulerian algorithms for general hyperbolic systems of partial differential equations with non-con...In this paper we present a 2D/3D high order accurate finite volume scheme in the context of direct Arbitrary-Lagrangian-Eulerian algorithms for general hyperbolic systems of partial differential equations with non-conservative products and stiff source terms.This scheme is constructed with a single stencil polynomial reconstruction operator,a one-step space-time ADER integration which is suitably designed for dealing even with stiff sources,a nodal solver with relaxation to determine the mesh motion,a path-conservative integration technique for the treatment of non-conservative products and an a posteriori stabilization procedure derived from the so-called Multidimensional Optimal Order Detection(MOOD)paradigm.In this work we consider the seven equation Baer-Nunziato model of compressible multi-phase flows as a representative model involving non-conservative products as well as relaxation source terms which are allowed to become stiff.The new scheme is validated against a set of test cases on 2D/3D unstructured moving meshes on parallel machines and the high order of accuracy achieved by the method is demonstrated by performing a numerical convergence study.Classical Riemann problems and explosion problems with exact solutions are simulated in 2D and 3D.The overall numerical code is also profiled to provide an estimate of the computational cost required by each component of the whole algorithm.展开更多
Managing software packages in a scientific computing environment is a challenging task, especially in the case of heterogeneous systems. It is error prone when installing and updating software packages in a sophistica...Managing software packages in a scientific computing environment is a challenging task, especially in the case of heterogeneous systems. It is error prone when installing and updating software packages in a sophisticated computing environment. Testing and performance evaluation in an on-the-fly manner is also a troublesome task for a production system. In this paper, we discuss a package management scheme based on containers. The newly developed method can ease the maintenance complexity and reduce human mistakes. We can benefit from the self-containing and isolation features of container technologies for maintaining the software packages among intricately connected clusters. By deploying the Super Computing application Strore（SCStore） over the WAN connected world-largest clusters, it proved that it can greatly reduce the effort for maintaining the consistency of software environment and bring benefit to achieve automation.展开更多
Hyperparameter tuning is a key step in developing high-performing machine learning models, but searching large hyperparameter spaces requires extensive computation using standard sequential methods. This work analyzes...Hyperparameter tuning is a key step in developing high-performing machine learning models, but searching large hyperparameter spaces requires extensive computation using standard sequential methods. This work analyzes the performance gains from parallel versus sequential hyperparameter optimization. Using scikit-learn’s Randomized SearchCV, this project tuned a Random Forest classifier for fake news detection via randomized grid search. Setting n_jobs to -1 enabled full parallelization across CPU cores. Results show the parallel implementation achieved over 5× faster CPU times and 3× faster total run times compared to sequential tuning. However, test accuracy slightly dropped from 99.26% sequentially to 99.15% with parallelism, indicating a trade-off between evaluation efficiency and model performance. Still, the significant computational gains allow more extensive hyperparameter exploration within reasonable timeframes, outweighing the small accuracy decrease. Further analysis could better quantify this trade-off across different models, tuning techniques, tasks, and hardware.展开更多
文摘As an important branch of information technology, high-performance computing has expanded its application field and its influence has been expanding. High-performance computing is always a key area of application in meteorology. We used field research and literature review methods to study the application of high performance computing in China’s meteorological department, and obtained the following results: 1) China Meteorological Department gradually established the first high-performance computer system since 1978. High-performance computing services can support operational numerical weather prediction models. 2) The Chinese meteorological department has always used the relatively advanced high-performance computing technology, and the business system capability has been continuously improved. The computing power has become an important symbol of the level of meteorological modernization. 3) High-performance computing technology and meteorological numerical forecasting applications are increasingly integrated, and continue to innovate and develop. 4) In the future, high-performance computing resource management will gradually transit from the current local pre-allocation mode to the local remote unified scheduling and shared use. In summary, we have come to the conclusion that the performance calculation business of the meteorological department will usher in a better tomorrow.
基金＂This paper is an extended version of ＂SpotMPl： a framework for auction-based HPC computing using amazon spot instances＂ published in the International Symposium on Advances of Distributed Computing and Networking （ADCN 2011）.Acknowledgment This research is supported in part by the National Science Foundation grant CNS 0958854 and educational resource grants from Amazon.com.
文摘Cloud computing is expanding widely in the world of IT infrastructure. This is due partly to the cost-saving effect of economies of scale. Fair market conditions can in theory provide a healthy environment to reflect the most reasonable costs of computations. While fixed cloud pricing provides an attractive low entry barrier for compute-intensive applications, both the consumer and supplier of computing resources can see high efficiency for their investments by participating in auction-based exchanges. There are huge incentives for the cloud provider to offer auctioned resources. However, from the consumer perspective, using these resources is a sparsely discussed challenge. This paper reports a methodology and framework designed to address the challenges of using HPC （High Performance Computing） applications on auction-based cloud clusters. The authors focus on HPC applications and describe a method for determining bid-aware checkpointing intervals. They extend a theoretical model for determining checkpoint intervals using statistical analysis of pricing histories. Also the latest developments in the SpotHPC framework are introduced which aim at facilitating the managed execution of real MPI applications on auction-based cloud environments. The authors use their model to simulate a set of algorithms with different computing and communication densities. The results show the complex interactions between optimal bidding strategies and parallel applications performance.
文摘A supercomputer with 1.0 Petaflops peak performance in single precision, designed and established by Institute of Process Engineering, Chinese Academy of Sciences, is introduced in this brief communication. A designing philosophy utilizing the similarity between hardware, software and the problems to be solved is embodied, based on the multi-scale method and discrete simulation approaches developed at Institute of Process Engineering （IPE） and implemented in a graphic processing unit （GPU）-based hybrid computing mode. The preliminary applications of this machine in areas of multi-phase flow, molecular dynamics and so on are reported, demonstrating the supercomputer as a paradigm of green computation in new architecture.
基金supported by the National High Technology Research and Development 863 Program of China under Grant No. 2007AA01Z117the National Basic Research 973 Program of China under Grant No.2007CB310900
文摘High performance computer （HPC） is a complex huge system, of which the architecture design meets increasing difficulties and risks. Traditional methods, such as theoretical analysis, component-level simulation and sequential simulation, are not applicable to system-level simulations of HPC systems. Even the parallel simulation using large-scale parallel machines also have many difficulties in scalability, reliability, generality, as well as efficiency. According to the current needs of HPC architecture design, this paper proposes a system-level parallel simulation platform： ArchSim. We first introduce the architecture of ArchSim simulation platform which is composed of a global server （GS）, local server agents （LSA） and entities. Secondly, we emphasize some key techniques of ArchSim, including the synchronization protocol, the communication mechanism and the distributed checkpointing/restart mechanism. We then make a synthesized test of some main performance indices of ArchSim with the phold benchmark and analyze the extra overhead generated by ArchSim. Finally, based on ArchSim, we construct a parallel event-driven interconnection network simulator and a system-level simulator for a small scale HPC system with 256 processors. The results of the performance test and HPC system simulations demonstrate that ArchSim can achieve high speedup ratio and high scalability on parallel host machine and support system-level simulations for the architecture design of HPC systems.
基金supported by National Key Research and Development Program 2016YFC0105406National Natural Science Foundation of China(11575095,61571262)。
文摘Objective:As a high computation cost discipline,nuclear science and engineering still relies heavily on traditional high performance computing(HPC)clusters.However,the usage of traditional HPC for nuclear science and engineering has been limited due to the poor flexibility,the software compatibility and the poor user interfaces.Virtualized/virtual HPC(vHPC)can mimic an HPC by using a cloud computing platform.In this work,we designed and developed a vHPC system for employment in nuclear engineering.Methods:The system is tested using the computation of the numberπby Monte Carlo and an X-ray digital imaging system simulation.The performance of the vHPC system is compared with that of the traditional HPCs.Results:As the number of the simulated particles increases,the virtual cluster computing time grows propor-tionally.The time used for the simulation of the X-ray imaging was about 21.1 h over a 12 kernels virtual server.Experimental results show that the performance of virtual cluster computing and the actual physical machine is almost the same.Conclusions:From these tests,it is concluded that vHPC is a good alternative for employing in nuclear engineering.The proposed vHPC in this paper will make HPC flexible and easy to deploy.
文摘HPCC(high performance computing challenge)基准是由DARPA的HPCS(high productivity computing system)项目所发布的评价高性能计算系统的测试基准程序,自推出至今,受到工业界和学术界的广泛关注.但是,HPCC仍有不尽如人意之处,主要表现在其测试结果是若干个指标项,需要测试者和决策者根据这些测试指标项进行分析和评估,缺少一个整体的、直观而统一的评价结果.提出一种基于HPCC和层次分析法的高性能计算系统评价模型——AHPCC(a high performance computer system evaluation model based on HPCC),当系统通过运行HPCC得到测试结果后,使用AHPCC模型对这些测试参数按系统应用目标建立层次结构图,并最终计算得到各系统关于特定应用目标的单一分数.以12个已测出HPCC性能参数的系统为例,使用AHPCC模型计算并分析了系统评价结果.实验结果表明,AHPCC模型提供了实际系统的统一而直观的评价指标,其评价结果符合高性能系统的设计和应用特点.
基金W.B.has been financed by the European Research Council(ERC)under the European Union’s Seventh Framework Programme(FP7/2007-2013)with the research project STiMulUs,ERC Grant agreement no.278267R.L.has been partially funded by the ANR under the JCJC project“ALE INC(ubator)3D”JS01-012-01the“International Centre for Mathematics and Computer Science in Toulouse”(CIMI)partially supported by ANR-11-LABX-0040-CIMI within the program ANR-11-IDEX-0002-02.The authors would like to acknowledge PRACE for awarding access to the SuperMUC supercomputer based in Munich,Germany at the Leibniz Rechenzentrum(LRZ).Parts of thematerial contained in this work have been elaborated,gathered and tested while W.B.visited the Mathematical Institute of Toulouse for three months and R.L.visited the Dipartimento di Ingegneria Civile Ambientale e Meccanica in Trento for three months.
文摘In this paper we present a 2D/3D high order accurate finite volume scheme in the context of direct Arbitrary-Lagrangian-Eulerian algorithms for general hyperbolic systems of partial differential equations with non-conservative products and stiff source terms.This scheme is constructed with a single stencil polynomial reconstruction operator,a one-step space-time ADER integration which is suitably designed for dealing even with stiff sources,a nodal solver with relaxation to determine the mesh motion,a path-conservative integration technique for the treatment of non-conservative products and an a posteriori stabilization procedure derived from the so-called Multidimensional Optimal Order Detection(MOOD)paradigm.In this work we consider the seven equation Baer-Nunziato model of compressible multi-phase flows as a representative model involving non-conservative products as well as relaxation source terms which are allowed to become stiff.The new scheme is validated against a set of test cases on 2D/3D unstructured moving meshes on parallel machines and the high order of accuracy achieved by the method is demonstrated by performing a numerical convergence study.Classical Riemann problems and explosion problems with exact solutions are simulated in 2D and 3D.The overall numerical code is also profiled to provide an estimate of the computational cost required by each component of the whole algorithm.
基金supported by the National Key R&D Program of China(No.2016YFA0602100)the National Natural Science Foundation of China(No.91530323)Open Fund of Key Laboratory of Data Analysis and Applications,SOA(No.LDAA-2014-03)
文摘Managing software packages in a scientific computing environment is a challenging task, especially in the case of heterogeneous systems. It is error prone when installing and updating software packages in a sophisticated computing environment. Testing and performance evaluation in an on-the-fly manner is also a troublesome task for a production system. In this paper, we discuss a package management scheme based on containers. The newly developed method can ease the maintenance complexity and reduce human mistakes. We can benefit from the self-containing and isolation features of container technologies for maintaining the software packages among intricately connected clusters. By deploying the Super Computing application Strore（SCStore） over the WAN connected world-largest clusters, it proved that it can greatly reduce the effort for maintaining the consistency of software environment and bring benefit to achieve automation.
文摘Hyperparameter tuning is a key step in developing high-performing machine learning models, but searching large hyperparameter spaces requires extensive computation using standard sequential methods. This work analyzes the performance gains from parallel versus sequential hyperparameter optimization. Using scikit-learn’s Randomized SearchCV, this project tuned a Random Forest classifier for fake news detection via randomized grid search. Setting n_jobs to -1 enabled full parallelization across CPU cores. Results show the parallel implementation achieved over 5× faster CPU times and 3× faster total run times compared to sequential tuning. However, test accuracy slightly dropped from 99.26% sequentially to 99.15% with parallelism, indicating a trade-off between evaluation efficiency and model performance. Still, the significant computational gains allow more extensive hyperparameter exploration within reasonable timeframes, outweighing the small accuracy decrease. Further analysis could better quantify this trade-off across different models, tuning techniques, tasks, and hardware.
