Cloud computing is expanding widely in the world of IT infrastructure. This is due partly to the cost-saving effect of economies of scale. Fair market conditions can in theory provide a healthy environment to reflect ...Cloud computing is expanding widely in the world of IT infrastructure. This is due partly to the cost-saving effect of economies of scale. Fair market conditions can in theory provide a healthy environment to reflect the most reasonable costs of computations. While fixed cloud pricing provides an attractive low entry barrier for compute-intensive applications, both the consumer and supplier of computing resources can see high efficiency for their investments by participating in auction-based exchanges. There are huge incentives for the cloud provider to offer auctioned resources. However, from the consumer perspective, using these resources is a sparsely discussed challenge. This paper reports a methodology and framework designed to address the challenges of using HPC （High Performance Computing） applications on auction-based cloud clusters. The authors focus on HPC applications and describe a method for determining bid-aware checkpointing intervals. They extend a theoretical model for determining checkpoint intervals using statistical analysis of pricing histories. Also the latest developments in the SpotHPC framework are introduced which aim at facilitating the managed execution of real MPI applications on auction-based cloud environments. The authors use their model to simulate a set of algorithms with different computing and communication densities. The results show the complex interactions between optimal bidding strategies and parallel applications performance.展开更多
We present techniques for characterization, modeling and generation of workloads for cloud computing applications. Methods for capturing the workloads of cloud computing applications in two different models - benchmar...We present techniques for characterization, modeling and generation of workloads for cloud computing applications. Methods for capturing the workloads of cloud computing applications in two different models - benchmark application and workload models are described. We give the design and implementation of a synthetic workload generator that accepts the benchmark and workload model specifications generated by the characterization and modeling of workloads of cloud computing applications. We propose the Georgia Tech Cloud Workload Specification Language (GT-CWSL) that provides a structured way for specification of application workloads. The GT-CWSL combines the specifications of benchmark and workload models to create workload specifications that are used by a synthetic workload generator to generate synthetic workloads for performance evaluation of cloud computing applications.展开更多
Traditional HPC （High Performance Computing） cluster is built on top of physical machines. It is usually not practical to reassign these machines to other tasks due to the fact that software installation is time con...Traditional HPC （High Performance Computing） cluster is built on top of physical machines. It is usually not practical to reassign these machines to other tasks due to the fact that software installation is time consuming. As a result, these machines are usually dedicated for the cluster usage. Virtualization technology provides an abstract layer which allows several different operating systems （with different software packages） running on top of one physical machine. Cloud computing provides an easy way for the user to manage and interact with the computing resources （the virtual machines in this case）. In this work, we demonstrate the feasibility of building a cloud-based cluster for HPC on top of a set of desktop computers that are interconnected by means of Fast Ethernet. Our cluster has several advantages. For instance, the deployment time of the cluster is quite fast： We need only 5 min to deploy a cluster of 30 machines, Besides, several performance benchmarks have been carried out. As expected, the embarrassingly parallel problem has the linear relationship between the performance and the cluster size.展开更多
This work proposes ARS(FaaS) serverless framework scheduling and provisioning resources for streaming applications autonomously, which ensures real-time response on unpredictable and fluctuating streaming data. A HPC ...This work proposes ARS(FaaS) serverless framework scheduling and provisioning resources for streaming applications autonomously, which ensures real-time response on unpredictable and fluctuating streaming data. A HPC cloud platform is used as a de facto platform, on which serverless computing for stream analytic is explored. This work enables application developers to build and run steaming applications without worrying about servers, which means that the developers are able to focus on application features instead of scheduling and provisioning resources of the infrastructure. The serverless computing framework, ARS(FaaS), provides function-as-a-service to make the developers write code in discrete event-driven functions. ARS(FaaS) is capable of running and scaling the developer's code automatically, according to the throughput of streaming events. The major contribution of this serverless framework is effective and efficient autonomous resource scheduling for real-time streaming analytic, which enables the developers to build applications faster with autonomous resource scheduling. ARS(FaaS) framework is appropriate for real-time and stream analytic on event-driven data with spiky and variable compute requirements.展开更多
Complex multi-tier applications deployed in cloud computing environments can experience rapid changes in their workloads. To ensure market readiness of such applications, adequate resources need to be provisioned so t...Complex multi-tier applications deployed in cloud computing environments can experience rapid changes in their workloads. To ensure market readiness of such applications, adequate resources need to be provisioned so that the applications can meet the demands of specified workload levels and at the same time ensure that service level agreements are met. Multi-tier cloud applications can have complex deployment configurations with load balancers, web servers, application servers and database servers. Complex dependencies may exist between servers in various tiers. To support provisioning and capacity planning decisions, performance testing approaches with synthetic workloads are used. Accuracy of a performance testing approach is determined by how closely the generated synthetic workloads mimic the realistic workloads. Since multi-tier applications can have varied deployment configurations and characteristic workloads, there is a need for a generic performance testing methodology that allows accurately modeling the performance of applications. We propose a methodology for performance testing of complex multi-tier applications. The workloads of multi-tier cloud applications are captured in two different models-benchmark application and workload models. An architecture model captures the deployment configurations of multi-tier applications. We propose a rapid deployment prototyping methodology that can help in choosing the best and most cost effective deployments for multi-tier applications that meet the specified performance requirements. We also describe a system bottleneck detection approach based on experimental evaluation of multi-tier applications.展开更多
Objective:As a high computation cost discipline,nuclear science and engineering still relies heavily on traditional high performance computing(HPC)clusters.However,the usage of traditional HPC for nuclear science and ...Objective:As a high computation cost discipline,nuclear science and engineering still relies heavily on traditional high performance computing(HPC)clusters.However,the usage of traditional HPC for nuclear science and engineering has been limited due to the poor flexibility,the software compatibility and the poor user interfaces.Virtualized/virtual HPC(vHPC)can mimic an HPC by using a cloud computing platform.In this work,we designed and developed a vHPC system for employment in nuclear engineering.Methods:The system is tested using the computation of the numberπby Monte Carlo and an X-ray digital imaging system simulation.The performance of the vHPC system is compared with that of the traditional HPCs.Results:As the number of the simulated particles increases,the virtual cluster computing time grows propor-tionally.The time used for the simulation of the X-ray imaging was about 21.1 h over a 12 kernels virtual server.Experimental results show that the performance of virtual cluster computing and the actual physical machine is almost the same.Conclusions:From these tests,it is concluded that vHPC is a good alternative for employing in nuclear engineering.The proposed vHPC in this paper will make HPC flexible and easy to deploy.展开更多
MapReduce is a programming model for processing large data sets, and Hadoop is the most popular open-source implementation of MapReduce. To achieve high performance, up to 190 Hadoop configuration parameters must be m...MapReduce is a programming model for processing large data sets, and Hadoop is the most popular open-source implementation of MapReduce. To achieve high performance, up to 190 Hadoop configuration parameters must be manually tunned. This is not only time-consuming but also error-pron. In this paper, we propose a new performance model based on random forest, a recently devel- oped machine-learning algorithm. The model, called RFMS, is used to predict the performance of a Hadoop system according to the system＇ s configuration parameters. RFMS is created from 2000 distinct fine-grained performance observations with different Hadoop configurations. We test RFMS against the measured performance of representative workloads from the Hadoop Micro-benchmark suite. The results show that the prediction accuracy of RFMS achieves 95% on average and up to 99%. This new, highly accurate prediction model can be used to automatically optimize the performance of Hadoop systems.展开更多
随着云计算技术的发展,高性能计算云(HPC in the Cloud)已得到学术界和产业界的关注。由于虚拟化技术带来的性能开销,高性能计算云面临着一些挑战。针对"高性能计算+云"的计算模式,分析了高性能计算云的优势,深入介绍了国内...随着云计算技术的发展,高性能计算云(HPC in the Cloud)已得到学术界和产业界的关注。由于虚拟化技术带来的性能开销,高性能计算云面临着一些挑战。针对"高性能计算+云"的计算模式,分析了高性能计算云的优势,深入介绍了国内外基于基准测试的高性能计算云的性能评测、性能优化、能耗和成本效益等关键问题,得出了针对基准测试的高性能计算云研究的基本思路,并对当前面临的问题和今后的发展趋势进行了总结和展望。展开更多
基金＂This paper is an extended version of ＂SpotMPl： a framework for auction-based HPC computing using amazon spot instances＂ published in the International Symposium on Advances of Distributed Computing and Networking （ADCN 2011）.Acknowledgment This research is supported in part by the National Science Foundation grant CNS 0958854 and educational resource grants from Amazon.com.
文摘Cloud computing is expanding widely in the world of IT infrastructure. This is due partly to the cost-saving effect of economies of scale. Fair market conditions can in theory provide a healthy environment to reflect the most reasonable costs of computations. While fixed cloud pricing provides an attractive low entry barrier for compute-intensive applications, both the consumer and supplier of computing resources can see high efficiency for their investments by participating in auction-based exchanges. There are huge incentives for the cloud provider to offer auctioned resources. However, from the consumer perspective, using these resources is a sparsely discussed challenge. This paper reports a methodology and framework designed to address the challenges of using HPC （High Performance Computing） applications on auction-based cloud clusters. The authors focus on HPC applications and describe a method for determining bid-aware checkpointing intervals. They extend a theoretical model for determining checkpoint intervals using statistical analysis of pricing histories. Also the latest developments in the SpotHPC framework are introduced which aim at facilitating the managed execution of real MPI applications on auction-based cloud environments. The authors use their model to simulate a set of algorithms with different computing and communication densities. The results show the complex interactions between optimal bidding strategies and parallel applications performance.
文摘We present techniques for characterization, modeling and generation of workloads for cloud computing applications. Methods for capturing the workloads of cloud computing applications in two different models - benchmark application and workload models are described. We give the design and implementation of a synthetic workload generator that accepts the benchmark and workload model specifications generated by the characterization and modeling of workloads of cloud computing applications. We propose the Georgia Tech Cloud Workload Specification Language (GT-CWSL) that provides a structured way for specification of application workloads. The GT-CWSL combines the specifications of benchmark and workload models to create workload specifications that are used by a synthetic workload generator to generate synthetic workloads for performance evaluation of cloud computing applications.
文摘Traditional HPC （High Performance Computing） cluster is built on top of physical machines. It is usually not practical to reassign these machines to other tasks due to the fact that software installation is time consuming. As a result, these machines are usually dedicated for the cluster usage. Virtualization technology provides an abstract layer which allows several different operating systems （with different software packages） running on top of one physical machine. Cloud computing provides an easy way for the user to manage and interact with the computing resources （the virtual machines in this case）. In this work, we demonstrate the feasibility of building a cloud-based cluster for HPC on top of a set of desktop computers that are interconnected by means of Fast Ethernet. Our cluster has several advantages. For instance, the deployment time of the cluster is quite fast： We need only 5 min to deploy a cluster of 30 machines, Besides, several performance benchmarks have been carried out. As expected, the embarrassingly parallel problem has the linear relationship between the performance and the cluster size.
基金Suported by the National Natural Science Foundation of China(No.61472089,61572143)NSFC-Guangdong Joint Found(No.U1501254)China Scholarship Council(No.201608440336)。
文摘This work proposes ARS(FaaS) serverless framework scheduling and provisioning resources for streaming applications autonomously, which ensures real-time response on unpredictable and fluctuating streaming data. A HPC cloud platform is used as a de facto platform, on which serverless computing for stream analytic is explored. This work enables application developers to build and run steaming applications without worrying about servers, which means that the developers are able to focus on application features instead of scheduling and provisioning resources of the infrastructure. The serverless computing framework, ARS(FaaS), provides function-as-a-service to make the developers write code in discrete event-driven functions. ARS(FaaS) is capable of running and scaling the developer's code automatically, according to the throughput of streaming events. The major contribution of this serverless framework is effective and efficient autonomous resource scheduling for real-time streaming analytic, which enables the developers to build applications faster with autonomous resource scheduling. ARS(FaaS) framework is appropriate for real-time and stream analytic on event-driven data with spiky and variable compute requirements.
文摘Complex multi-tier applications deployed in cloud computing environments can experience rapid changes in their workloads. To ensure market readiness of such applications, adequate resources need to be provisioned so that the applications can meet the demands of specified workload levels and at the same time ensure that service level agreements are met. Multi-tier cloud applications can have complex deployment configurations with load balancers, web servers, application servers and database servers. Complex dependencies may exist between servers in various tiers. To support provisioning and capacity planning decisions, performance testing approaches with synthetic workloads are used. Accuracy of a performance testing approach is determined by how closely the generated synthetic workloads mimic the realistic workloads. Since multi-tier applications can have varied deployment configurations and characteristic workloads, there is a need for a generic performance testing methodology that allows accurately modeling the performance of applications. We propose a methodology for performance testing of complex multi-tier applications. The workloads of multi-tier cloud applications are captured in two different models-benchmark application and workload models. An architecture model captures the deployment configurations of multi-tier applications. We propose a rapid deployment prototyping methodology that can help in choosing the best and most cost effective deployments for multi-tier applications that meet the specified performance requirements. We also describe a system bottleneck detection approach based on experimental evaluation of multi-tier applications.
基金supported by National Key Research and Development Program 2016YFC0105406National Natural Science Foundation of China(11575095,61571262)。
文摘Objective:As a high computation cost discipline,nuclear science and engineering still relies heavily on traditional high performance computing(HPC)clusters.However,the usage of traditional HPC for nuclear science and engineering has been limited due to the poor flexibility,the software compatibility and the poor user interfaces.Virtualized/virtual HPC(vHPC)can mimic an HPC by using a cloud computing platform.In this work,we designed and developed a vHPC system for employment in nuclear engineering.Methods:The system is tested using the computation of the numberπby Monte Carlo and an X-ray digital imaging system simulation.The performance of the vHPC system is compared with that of the traditional HPCs.Results:As the number of the simulated particles increases,the virtual cluster computing time grows propor-tionally.The time used for the simulation of the X-ray imaging was about 21.1 h over a 12 kernels virtual server.Experimental results show that the performance of virtual cluster computing and the actual physical machine is almost the same.Conclusions:From these tests,it is concluded that vHPC is a good alternative for employing in nuclear engineering.The proposed vHPC in this paper will make HPC flexible and easy to deploy.
基金supported by the cooperation project of Research on Green Cloud IDC Resource Scheduling with ZTE Corporation
文摘MapReduce is a programming model for processing large data sets, and Hadoop is the most popular open-source implementation of MapReduce. To achieve high performance, up to 190 Hadoop configuration parameters must be manually tunned. This is not only time-consuming but also error-pron. In this paper, we propose a new performance model based on random forest, a recently devel- oped machine-learning algorithm. The model, called RFMS, is used to predict the performance of a Hadoop system according to the system＇ s configuration parameters. RFMS is created from 2000 distinct fine-grained performance observations with different Hadoop configurations. We test RFMS against the measured performance of representative workloads from the Hadoop Micro-benchmark suite. The results show that the prediction accuracy of RFMS achieves 95% on average and up to 99%. This new, highly accurate prediction model can be used to automatically optimize the performance of Hadoop systems.
文摘随着云计算技术的发展,高性能计算云(HPC in the Cloud)已得到学术界和产业界的关注。由于虚拟化技术带来的性能开销,高性能计算云面临着一些挑战。针对"高性能计算+云"的计算模式,分析了高性能计算云的优势,深入介绍了国内外基于基准测试的高性能计算云的性能评测、性能优化、能耗和成本效益等关键问题,得出了针对基准测试的高性能计算云研究的基本思路,并对当前面临的问题和今后的发展趋势进行了总结和展望。
