Analyzing time-dimension communication characterizations for representative scientific applications on supercomputer systems

Exascale computing is one of the major challenges of this decade,and several studies have shown that communications are becoming one of the bottlenecks for scaling parallel applications.The analysis on the characteristics of communications can effectively aid to improve the performance of scientific applications.In this paper,we focus on the statistical regularity in time-dimension communication characteristics for representative scientific applications on supercomputer systems,and then prove that the distribution of communication-event intervals has a power-law decay,which is common in scientific interests and human activities.We verify the distribution of communication-event intervals has really a power-law decay on the Tianhe-2 supercomputer,and also on the other six parallel systems with three different network topologies and two routing policies.In order to do a quantitative study on the power-law distribution,we exploit two groups of statistics:bursty vs.memory and periodicity vs.dispersion.Our results indicate that the communication events show a“strong-bursty and weak-memory”characteristic and the communication event intervals show the periodicity and the dispersion.Finally,our research provides an insight into the relationship between communication optimizations and time-dimension communication characteristics.

More Super Supercomputers

In June 2018, the United States claimed the No. 1 position in supercomputing according to TOP500, which ranks the top 500 most powerful computer systems in the world [1]. The US Department of Energy’s Summit machine (Fig. 1)[1] claimed this distinction, which previously had been held by China’s Sunway TaihuLight supercomputer.

The Chinese Supercomputer

The first in China 10~9 sparallel supercomputer, named as Yinhe-Ⅱ, had been manufac-tured by Science-technological University of National Defence. The main feature of thesupercomputer are: 4-processor system, the principle frequence 50 MHz, the word length 64 byte,the main memory 256 Mb, two individual input / output subsystems, > 10~9 operations per sec-

World’s Fastest Supercomputer Picks COVID-19 Drug

We have demonstrated the application of the world’s fastest supercomputer Fugaku located in Japan to select the COVID-19 drugs and stopping the pandemic spread. Using computer simulation out of 2128 potential drug candidates, the world’s fastest supercomputer picked 30 most effective and potential drugs. Twelve of them are under clinical trials outside Japan;some are being tested in Japan. The computer reduced the computation time from one year to 10 days when compared to second superfast computer of the world. Fugaku supercomputer was employed to know the behavior of airborne aerosol COVID-19 virus. 3Cs were suggested: avoid closed and crowded spaces and contacts to stop the pandemic spread. The progress in vaccine development and proper use and type of mask has also been described in this article. The article will benefit greatly to stop spreading and treating the pandemic COVID-19.

Typhoon Case Comparison Analysis Between Heterogeneous Many-Core and Homogenous Multicore Supercomputing Platforms

In this paper,a typical experiment is carried out based on a high-resolution air-sea coupled model,namely,the coupled ocean-atmosphere-wave-sediment transport(COAWST)model,on both heterogeneous many-core(SW)and homogenous multicore(Intel)supercomputing platforms.We construct a hindcast of Typhoon Lekima on both the SW and Intel platforms,compare the simulation results between these two platforms and compare the key elements of the atmospheric and ocean modules to reanalysis data.The comparative experiment in this typhoon case indicates that the domestic many-core computing platform and general cluster yield almost no differences in the simulated typhoon path and intensity,and the differences in surface pressure(PSFC)in the WRF model and sea surface temperature(SST)in the short-range forecast are very small,whereas a major difference can be identified at high latitudes after the first 10 days.Further heat budget analysis verifies that the differences in SST after 10 days are mainly caused by shortwave radiation variations,as influenced by subsequently generated typhoons in the system.These typhoons generated in the hindcast after the first 10 days attain obviously different trajectories between the two platforms.

MilkyWay-2 supercomputer： system and application

On June 17, 2013, MilkyWay-2 （Tianhe-2） supercomputer was crowned as the fastest supercomputer in the world on the 41th TOP500 list. This paper provides an overview of the MilkyWay-2 project and describes the design of hardware and software systems. The key architecture features of MilkyWay-2 are highlighted, including neo-heterogeneous compute nodes integrating commodity- off-the-shelf processors and accelerators that share similar instruction set architecture, powerful networks that employ proprietary interconnection chips to support the massively parallel message-passing communications, proprietary 16- core processor designed for scientific computing, efficient software stacks that provide high performance file system, emerging programming model for heterogeneous systems, and intelligent system administration. We perform extensive evaluation with wide-ranging applications from LINPACK and Graph500 benchmarks to massively parallel software deployed in the system.

Hybrid hierarchy storage system in MilkyWay-2 supercomputer

With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabytes. To reduce this gap, large-scale storage systems need to be designed and implemented with high performance and scalability. MilkyWay-2 （TH-2） supercomputer system with peak performance 54.9 Props, definitely has this kind of requirement for storage system. This paper mainly introduces the storage system in MilkyWay-2 supercomputer, including the hardware architecture and the parallel file system. The storage system in MilkyWay-2 supercomputer exploits a novel hybrid hierarchy storage architecture to enable high scalability of I/O clients, I/O bandwidth and storage capacity. To fit this architecture, a user level virtualized file system, named H^2FS, is designed and implemented which can cooperate local storage and shared storage together into a dynamic single namespace to optimize I/O performance in IO-intensive applications. The evaluation results show that the storage system in MilkyWay-2 supercomputer can satisfy the critical requirements in large scale supercomputer, such as performance and scalability.

Iaso： an autonomous fault-tolerant management system for supercomputers

With the increase of system scale, the inherent reliability of supercomputers becomes lower and lower. The cost of fault handling and task recovery increases so rapidly that the reliability issue will soon harm the usability of supercomputers. This issue is referred to as the ＂reliability wall＂, which is regarded as a critical problem for current and future supercomputers. To address this problem, we propose an autonomous fault-tolerant system, named Iaso, in MilkyWay- 2 system. Iaso introduces the concept of autonomous management in supercomputers. By autonomous management, the computer itself, rather than manpower, takes charge of the fault management work. Iaso automatically manage the whole lifecycle of faults, including fault detection, fault diagnosis, fault isolation, and task recovery. Iaso endows the autonomous features with MilkyWay-2 system, such as self-awareness, self-diagnosis, self-healing, and self-protection. With the help of Iaso, the cost of fault handling in supercomputers reduces from several hours to a few seconds. Iaso greatly improves the usability and reliability of MilkyWay-2 system.

AquaSee: Predict Load and Cooling System Faults of Supercomputers Using Chilled Water Data

An analysis of real-world operational data of Tianhe-1A(TH-1A)supercomputer system shows that chilled water data not only can reflect the status of a chiller system but also are related to supercomputer load.This study proposes AquaSee,a method that can predict the load and cooling system faults of supercomputers by using chilled water pressure and temperature data.This method is validated on the basis of real-world operational data of the TH-1A supercomputer system at the National Supercomputer Center in Tianjin.Datasets with various compositions are used to construct the prediction model,which is also established using different prediction sequence lengths.Experimental results show that the method that uses a combination of pressure and temperature data performs more effectively than that only consisting of either pressure or temperature data.The best inference sequence length is two points.Furthermore,an anomaly monitoring system is set up by using chilled water data to help engineers detect chiller system anomalies.

Development of Operational Technology for Meteorological High Performance Computing

As an important branch of information technology, high-performance computing has expanded its application field and its influence has been expanding. High-performance computing is always a key area of application in meteorology. We used field research and literature review methods to study the application of high performance computing in China’s meteorological department, and obtained the following results: 1) China Meteorological Department gradually established the first high-performance computer system since 1978. High-performance computing services can support operational numerical weather prediction models. 2) The Chinese meteorological department has always used the relatively advanced high-performance computing technology, and the business system capability has been continuously improved. The computing power has become an important symbol of the level of meteorological modernization. 3) High-performance computing technology and meteorological numerical forecasting applications are increasingly integrated, and continue to innovate and develop. 4) In the future, high-performance computing resource management will gradually transit from the current local pre-allocation mode to the local remote unified scheduling and shared use. In summary, we have come to the conclusion that the performance calculation business of the meteorological department will usher in a better tomorrow.

A survey of brain-inspired artificial intelligence and its engineering

Exploring the human brain is perhaps the most challenging and fascinating scientific issue in the 21st century.It will facilitate the development of various aspects of the society,including economics,education,health care,national defense and daily life.The artificial intelligence techniques are becoming useful as an alternate method of classical techniques or as a component of an integrated system.They are used to solve complicated problems in various fields and becoming increasingly popular nowadays.Especially,the investigation of human brain will promote the artificial intelligence techniques,utilizing the accumulating knowledge of neuroscience,brain-machine interface techniques,algorithms of spiking neural networks and neuromorphic supercomputers.Consequently,we provide a comprehensive survey of the research and motivations for brain-inspired artificial intelligence and its engineering over its history.The goals of this work are to provide a brief review of the research associated with brain-inspired artificial intelligence and its related engineering techniques,and to motivate further work by elucidating challenges in the field where new researches are required.

Scalability and efficiency challenges for the exascale supercomputing system:practice of a parallel supporting environment on the Sunway exascale prototype system

With the continuous improvement of supercomputer performance and the integration of artificial intelligence with traditional scientific computing,the scale of applications is gradually increasing,from millions to tens of millions of computing cores,which raises great challenges to achieve high scalability and efficiency of parallel applications on super-large-scale systems.Taking the Sunway exascale prototype system as an example,in this paper we first analyze the challenges of high scalability and high efficiency for parallel applications in the exascale era.To overcome these challenges,the optimization technologies used in the parallel supporting environment software on the Sunway exascale prototype system are highlighted,including the parallel operating system,input/output(I/O)optimization technology,ultra-large-scale parallel debugging technology,10-million-core parallel algorithm,and mixed-precision method.Parallel operating systems and I/O optimization technology mainly support largescale system scaling,while the ultra-large-scale parallel debugging technology,10-million-core parallel algorithm,and mixed-precision method mainly enhance the efficiency of large-scale applications.Finally,the contributions to various applications running on the Sunway exascale prototype system are introduced,verifying the effectiveness of the parallel supporting environment design.

High Performance Interconnect Network for Tianhe System

In this paper, we present the Tianhe-2 interconnect network and message passing services. We describe the architecture of the router and network interface chips, and highlight a set of hardware and software features effectively supporting high performance communications, ranging over remote direct memory access, collective optimization, hardwareenable reliable end-to-end communication, user-level message passing services, etc. Measured hardware performance results are also presented.

The TianHe-1A Supercomputer: Its Hardware and Software

This paper presents an overview of TianHe-lA （TH-1A） supercomputer, which is built by National University of Defense Technology of China （NUDT）. TH-1A adopts a hybrid architecture by integrating CPUs and GPUs, and its interconnect network is a proprietary high-speed communication network. The theoretical peak performance of TH-1A is 4700TFlops, and its LINPACK test result is 2566TFlops. It was ranked the No. 1 on the TOP500 List released in November, 2010. TH-1A is now deployed in National Supercomputer Center in Tianjin and provides high performance computing services. TH-1A has played an important role in many applications, such as oil exploration, weather forecast, bio-medical research.

Optimizing Linpack Benchmark on GPU-Accelerated Petascale Supercomputer

In this paper we present the programming of the Linpack benchmark on TianHe-1 system,the first petascale supercomputer system of China,and the largest GPU-accelerated heterogeneous system ever attempted before.A hybrid programming model consisting of MPI,OpenMP and streaming computing is described to explore the task parallel,thread parallel and data parallel of the Linpack.We explain how we optimized the load distribution across the CPUs and GPUs using the two-level adaptive method and describe the implementation in details.To overcome the low-bandwidth between the CPU and GPU communication,we present a software pipelining technique to hide the communication overhead.Combined with other traditional optimizations,the Linpack we developed achieved 196.7 GFLOPS on a single compute element of TianHe-1.This result is 70.1% of the peak compute capability,3.3 times faster than the result by using the vendor＇s library.On the full configuration of TianHe-1 our optimizations resulted in a Linpack performance of 0.563 PFLOPS,which made TianHe-1 the 5th fastest supercomputer on the Top500 list in November,2009.

Dawning Nebulae:A PetaFLOPS Supercomputer with a Heterogeneous Structure

Dawning Nebulae is a heterogeneous system composed of 9280 multi-core x86 CPUs and 4640 NVIDIA Fermi GPUs. With a Linpack performance of 1.271 petaFLOPS, it was ranked the second in the TOP500 List released in June 2010. In this paper, key issues in the system design of Dawning Nebulae are introduced. System tuning methodologies aiming at petaFLOPS Linpack result are presented, including algorithmic optimization and communication improvement. The design of its file I/O subsystem, including HVFS and the underlying DCFS3, is also described. Performance evaluations show that the Linpack efficiency of each node reaches 69.89%, and 1024-node aggregate read and write bandwidths exceed 100 GB/s and 70 GB/s respectively. The success of Dawning Nebulae has demonstrated the viability of CPU/GPU heterogeneous structure for future designs of supercomputers.

A Large-Scale Study of Failures on Petascale Supercomputers

With the rapid development of supercomputers, the scale and complexity are ever increasing, and the reliability and resilience are faced with larger challenges. There are many important technologies in fault tolerance, such as proacrive failure avoidance technologies based on fault prediction, reactive fault tolerance based on checkpoint, and scheduling technologies to improve reliability. Both qualitative and quantitative descriptions on characteristics of system faults are very critical for these technologies, This study analyzes the source of failures on two typical petascale supercomputers called Sunway BlueLight （based on multi-core CPUs） and Sunway TaihuLight （based on heterogeneous manycore CPUs）. It uncovers some interesting fault characteristics and finds unknown correlation relationship among main components＇ faults. Finally the paper analyzes the failure time of the two supercomputers in various grains of resource and different time spans, and builds a uniform multi-dimensional failure time model for petascale supereomputers.

OpenMC: Towards Simplifying Programming for TianHe Supercomputers

Modern petascale and future exascale systems are massively heterogeneous architectures. Developing productive intra-node programming models is crucial toward addressing their programming challenge. We introduce a directive- based intra-node programming model, OpenMC, and show that this new model can achieve ease of programming, high performance, and the degree of portability desired for heterogeneous nodes, especially those in TianHe supercomputers. While existing models are geared towards oifloading computations to accelerators （typically one）, OpenMC alms to more uniformly and adequately exploit the potential offered by multiple CPUs and accelerators in a compute node. OpenMC achieves this by providing a unified abstraction of hardware resources as workers and facilitating the exploitation of asynchronous task parallelism on the workers. We present an overview of OpenMC, a prototyping implementation, and results from some initial comparisons with OpenMP and hand-written code in developing six applications on two types of nodes from TianHe supercomputers.

A visual analytics system for optimizing the performance of large-scale networks in supercomputing systems

The overall efficiency of an extreme-scale supercomputer largely relies on the performance of its network interconnects.Several of the state of the art supercomputers use networks based on the increasingly popular Dragonfly topology.It is crucial to study the behavior and performance of different parallel applications running on Dragonfly networks in order to make optimal system configurations and design choices,such as job scheduling and routing strategies.However,in order to study these temporal network behavior,we would need a tool to analyze and correlate numerous sets of multivariate time-series data collected from the Dragonfly's multi-level hierarchies.This paper presents such a tool-a visual analytics system-that uses the Dragonfly network to investigate the temporal behavior and optimize the communication performance of a supercomputer.We coupled interactive visualization with time-series analysis methods to help reveal hidden patterns in the network behavior with respect to different parallel applications and system configurations.Our system also provides multiple coordinated views for connecting behaviors observed at different levels of the network hierarchies,which effectively helps visual analysis tasks.We demonstrate the effectiveness of the system with a set of case studies.Our system and findings can not only help improve the communication performance of supercomputing applications,but also the network performance of next-generation supercomputers.

Unified Programming Models for Heterogeneous High-Performance Computers

Unified programming models can effectively improve program portability on various heterogeneous high-performance computers.Existing unified programming models put a lot of effort to code portability but are still far from achieving good performance portability.In this paper,we present a preliminary design of a performance-portable unified programming model including four aspects:programming language,programming abstraction,compilation optimization,and scheduling system.Specifically,domain-specific languages introduce domain knowledge to decouple the optimizations for different applications and architectures.The unified programming abstraction unifies the common features of different architectures to support common optimizations.Multi-level compilation optimization enables comprehensive performance optimization based on multi-level intermediate representations.Resource-aware lightweight runtime scheduling system improves the resource utilization of heterogeneous computers.This is a perspective paper to show our viewpoints on programming models for emerging heterogeneous systems.