Low-power task scheduling algorithm for large-scale cloud data centers

How to effectively reduce the energy consumption of large-scale data centers is a key issue in cloud computing. This paper presents a novel low-power task scheduling algorithm （L3SA） for large-scale cloud data centers. The winner tree is introduced to make the data nodes as the leaf nodes of the tree and the final winner on the purpose of reducing energy consumption is selected. The complexity of large-scale cloud data centers is fully consider, and the task comparson coefficient is defined to make task scheduling strategy more reasonable. Experiments and performance analysis show that the proposed algorithm can effectively improve the node utilization, and reduce the overall power consumption of the cloud data center.

Efficient Packet Scheduling Technique for Data Merging in Wireless Sensor Networks

Wireless Sensor Networks(WSNs) has become a popular research topic due to its resource constraints. Energy consumption and transmission delay is crucial requirement to be handled to enhance the popularity of WSNs. In order to overcome these issues, we have proposed an Efficient Packet Scheduling Technique for Data Merging in WSNs. Packet scheduling is done by using three levels of priority queue and to reduce the transmission delay. Real-time data packets are placed in high priority queue and Non real-time data packets based on local or remote data are placed on other queues. In this paper, we have used Time Division Multiple Access(TDMA) scheme to efficiently determine the priority of the packet at each level and transmit the data packets from lower level to higher level through intermediate nodes. To reduce the number of transmission, efficient data merge technique is used to merge the data packet in intermediate nodes which has same destination node. Data merge utilize the maximum packet size by appending the merged packets with received packets till the maximum packet size or maximum waiting time is reached. Real-time data packets are directly forwarded to the next node without applying data merge. The performance is evaluated under various metrics like packet delivery ratio, packet drop, energy consumption and delay based on changing the number of nodes and transmission rate. Our results show significant reduction in various performance metrics.

Performance Prediction Based Workload Scheduling in Co-Located Cluster

Cloud service providers generally co-locate online services and batch jobs onto the same computer cluster,where the resources can be pooled in order to maximize data center resource utilization.Due to resource competition between batch jobs and online services,co-location frequently impairs the performance of online services.This study presents a quality of service(QoS)prediction-based schedulingmodel(QPSM)for co-locatedworkloads.The performance prediction of QPSM consists of two parts:the prediction of an online service’s QoS anomaly based on XGBoost and the prediction of the completion time of an offline batch job based on randomforest.On-line service QoS anomaly prediction is used to evaluate the influence of batch jobmix on on-line service performance,and batch job completion time prediction is utilized to reduce the total waiting time of batch jobs.When the same number of batch jobs are scheduled in experiments using typical test sets such as CloudSuite,the scheduling time required by QPSM is reduced by about 6 h on average compared with the first-come,first-served strategy and by about 11 h compared with the random scheduling strategy.Compared with the non-co-located situation,QPSM can improve CPU resource utilization by 12.15% and memory resource utilization by 5.7% on average.Experiments show that the QPSM scheduling strategy proposed in this study can effectively guarantee the quality of online services and further improve cluster resource utilization.

Energy-Aware Scheduling Scheme Using Workload-Aware Consolidation Technique in Cloud Data Centres

To reduce energy consumption in cloud data centres,in this paper,we propose two algorithms called the Energy-aware Scheduling algorithm using Workload-aware Consolidation Technique(ESWCT) and the Energyaware Live Migration algorithm using Workload-aware Consolidation Technique(ELMWCT).As opposed to traditional energy-aware scheduling algorithms,which often focus on only one-dimensional resource,the two algorithms are based on the fact that multiple resources(such as CPU,memory and network bandwidth)are shared by users concurrently in cloud data centres and heterogeneous workloads have different resource consumption characteristics.Both algorithms investigate the problem of consolidating heterogeneous workloads.They try to execute all Virtual Machines(VMs) with the minimum amount of Physical Machines(PMs),and then power off unused physical servers to reduce power consumption.Simulation results show that both algorithms efficiently utilise the resources in cloud data centres,and the multidimensional resources have good balanced utilizations,which demonstrate their promising energy saving capability.

Multi-Dimensional Aware Scheduling for Co-optimizing Utilization in Data Center

Resource Scheduling is crucial to data centers. However, most previous works focus only on one-dimensional resource models which ignoring the fact that multiple resources simultaneously utilized, including CPU, memory and network bandwidth. As cloud computing allows uncoordinated and heterogeneous users to share a data center, competition for multiple resources has become increasingly severe. Motivated by the differences on integrated utilization obtained from different packing schemes, in this paper we take the scheduling problem as a multi-dimensional combinatorial optimization problem with constraint satisfaction. With NP hardness, we present Multiple attribute decision based Integrated Resource Scheduling (MIRS), and a novel heuristic algorithm to gain the approximate optimal solution. Refers to simulation results, in face of various workload sets, our algorithm has significant superiorities in terms of efficiency and performance compared with previous methods.

Scheduling for Uncertain Data Broadcast in Mobile Networks

With the increasing popularity of wireless sensor network and GPS （ global positioning system）, uncertain data as a new type of data brings a new challenge for the traditional data processing methods. Data broadcast is an effective means for data dissemination in mobile networks. In this paper, the def＇mition of the mean uncertainty ratio of data is presented and a broadcasting scheme is proposed for uncertain data dissemination. Simulation results show that the scheme can reduce the uncertainty of the broadcasted uncertain data effectively at the cost of a minor increase in data access time, in the case of no transmission error and presence of transmission errors. As a result, lower uncertainty of data benefits the qualifies of the query results based on the data.

Smart Society and Artificial Intelligence:Big Data Scheduling and the Global Standard Method Applied to Smart Maintenance

The implementation of artificial intelligence(AI)in a smart society,in which the analysis of human habits is mandatory,requires automated data scheduling and analysis using smart applications,a smart infrastructure,smart systems,and a smart network.In this context,which is characterized by a large gap between training and operative processes,a dedicated method is required to manage and extract the massive amount of data and the related information mining.The method presented in this work aims to reduce this gap with near-zero-failure advanced diagnostics(AD)for smart management,which is exploitable in any context of Society 5.0,thus reducing the risk factors at all management levels and ensuring quality and sustainability.We have also developed innovative applications for a humancentered management system to support scheduling in the maintenance of operative processes,for reducing training costs,for improving production yield,and for creating a human–machine cyberspace for smart infrastructure design.The results obtained in 12 international companies demonstrate a possible global standardization of operative processes,leading to the design of a near-zero-failure intelligent system that is able to learn and upgrade itself.Our new method provides guidance for selecting the new generation of intelligent manufacturing and smart systems in order to optimize human–machine interactions,with the related smart maintenance and education.

Scheduling for constantly-evolving data broadcasting in asymmetric communication networks

Recent advances in wireless sensor networks and GPS have made constantly-evolving data a new type of data which bring a new challenge to traditional data processing methods. Data broadcasting is an effective means for data dissemination in asymmetric communication networks, such as wireless networks. In this paper, definition of the mean uncertainty ratio of data is presented and a broadcasting scheme is proposed for constantly-evolving data dissemination. Simulation results show that the scheme can reduce the uncertainty of the broadcasted constantly-evolving data effectively at the cost of minor increase in data access time, in the case of no transmission error, transmission errors present, and multiple broadcast channels. As a result it benefits the qualities of the query results based on the data.

Mathematical Optimization Modelling for Fast-Switched and Delay Minimized Scheduling for Intra-Cell Communication in an AWGR-Based PON Data Center

As the internet traffic along with the processing power in data centers are exponentially growing, the need for the design of energy efficient with highly elastic networking infrastructure to support the different applications and cloud services that can be hosted in data centers have become a hot research area. A key departure from the norm is that conventional routers and switches in conventional data centers are replaced with high performance Passive Optical Networks (PONs) to take over the role of routing and traffic forwarding through efficient resource provisioning algorithms. In this paper, the different aspects of PONs in the design of energy efficient, high capacity, and highly elastic networking infrastructures to support the applications and services hosted by modern data centers are considered. In this work, a mathematical optimization model for energy-efficient and delay-minimized scheduling in AWGR based PON data center for PON cell fabric configuration will be presented. The performance of the proposed architecture in terms of efficient scheduling against average delay and power consumption for different traffic loads and patterns will be evaluated. Different scenarios of traffic;random and unbalanced with hotspots are examined to evaluate the average delay and power consumption with and without sleep mode. Results have shown that with sleep mode enabled, power savings for two evaluated objective functions have shown similar results when examining different traffic patterns. The power savings range between 8% and 55% during low and high load activities, respectively. However, minimization of delay model has shown improvement in reducing total average delay reaching up to 42% if compared with the model with objective of minimization of power consumption.

Autonomous UAV 3D trajectory optimization and transmission scheduling for sensor data collection on uneven terrains

This paper considers a time-constrained data collection problem from a network of ground sensors located on uneven terrain by an Unmanned Aerial Vehicle(UAV),a typical Unmanned Aerial System(UAS).The ground sensors harvest renewable energy and are equipped with batteries and data buffers.The ground sensor model takes into account sensor data buffer and battery limitations.An asymptotically globally optimal method of joint UAV 3D trajectory optimization and data transmission schedule is developed.The developed method maximizes the amount of data transmitted to the UAV without losses and too long delays and minimizes the propulsion energy of the UAV.The developed algorithm of optimal trajectory optimization and transmission scheduling is based on dynamic programming.Computer simulations demonstrate the effectiveness of the proposed algorithm.

Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput.This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems.First,integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing,which reduces the search scope of the database and dramatically speeds up data searching.Next,exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first,which reduces the average waiting time of job execution.As a result,the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing,significantly improving the speed of data retrieval up to 53%and increasing system throughput by 53%.On the other hand,the proposed job scheduling algorithmdefeats both first-in-first-out andmemory-sensitive heterogeneous early finish time scheduling algorithms,effectively shortening the average waiting time up to 5%and average weighted turnaround time by 19%,respectively.

Traffic Scheduling Mechanism Based on Interference Avoidance for Meter Data Collection in Wireless Smart Grid Communication Networks

Meter Data Collection Building Area Network(MDCBAN) deployed in high rises is playing an increasingly important role in wireless multi-hop smart grid meter data collection. Recently, increasingly numerous application layer data traffic makes MDCBAN be facing serious communication pressure. In addition, large density of meter data collection devices scattered in the limited geographical space of high rises results in obvious communication interference. To solve these problems, a traffic scheduling mechanism based on interference avoidance for meter data collection in MDCBAN is proposed. Firstly, the characteristics of network topology are analyzed and the corresponding traffic distribution model is proposed. Next, a wireless multi-channel selection scheme for different Floor Gateways and a single-channel time unit assignment scheme for data collection devices in the same Floor Network are proposed to avoid interference. At last, a data balanced traffic scheduling algorithm is proposed. Simulation results show that balanced traffic distribution and highly efficient and reliable data transmission can be achieved on the basis of effective interference avoidance between data collection devices.

Peer priority based data scheduling algorithm in P2P streaming system

In order to solve the problem that the existing data scheduling algorithm cannot make full use of neighbors' bandwidth resources when allocating data request among several senders in the multisender based P2P streaming system,a peer priority based scheduling algorithm is proposed.The algorithm calculates neighbors' priority based on peers' historical service evaluation as well as how many wanted data that the neighbor has.The data request allocated to each neighbor is adjusted dynamically according to the priority when scheduling.Peers with high priority are preferred to allocate more data request.Experiment shows the algorithm can make full use of neighbors' bandwidth resources to transmit data to reduce server pressure effectively and improve system scalability.

Metrics and Algorithms for Scheduling of Data Dissemination in Mesh Units Assisted Vehicular Networks

Data dissemination is an important application in vehicular networks. We observe that messages in vehicular networks are usually subject to both time and space constraints, and therefore should be disseminated during a specified duration and within a specific coverage. Since vehicles are moving in and out of a region, dis-semination of a message should be repeated to achieve reliability. However, the reliable dissemination for some messages might be at the cost of unreliable or even no chance of dissemination for other messages, which raises tradeoffs between reliability and fairness. In this paper, we study the scheduling of data dis-semination in vehicular networks with mesh infrastructure. Firstly, we propose performance metrics for both reliability and fairness. Factors on both the time and space dimensions are incorporated in the reliability met-ric and the fairness in both network-wide and Mesh Roadside Unit-wise (MRU-wise) senses are considered in the fairness metric. Secondly, we propose several scheduling algorithms: one reliability-oriented algorithm, one fairness-oriented algorithm and three hybrid schemes. Finally, we perform extensive evaluation work to quantitatively analyze different scheduling algorithms. Our evaluation results show that 1) hybrid schemes outperform reliability-oriented and fairness-oriented algorithms in the sense of overall efficiency and 2) dif-ferent algorithms have quite different characteristics on reliability and fairness.

High Performance Priority Packets Scheduling Mechanism for Big Data in Smart Cities

Today,Internet of Things(IoT)is a technology paradigm which convinces many researchers for the purpose of achieving high performance of packets delivery in IoT applications such as smart cities.Interconnecting various physical devices such as sensors or actuators with the Internet may causes different constraints on the network resources such as packets delivery ratio,energy efficiency,end-to-end delays etc.However,traditional scheduling methodologies in large-scale environments such as big data smart cities cannot meet the requirements for high performance network metrics.In big data smart cities applications which need fast packets transmission ratio such as sending priority packets to hospitals for an emergency case,an efficient schedulingmechanism ismandatory which is the main concern of this paper.In this paper,we overcome the shortcoming issues of the traditional scheduling algorithms that are utilized in big data smart cities emergency applications.Transmission information about the priority packets between the source nodes(i.e.,people with emergency cases)and the destination nodes(i.e.,hospitals)is performed before sending the packets in order to reserve transmission channels and prepare the sequence of transmission of theses priority packets between the two parties.In our proposed mechanism,Software Defined Networking(SDN)with centralized communication controller will be responsible for determining the scheduling and processing sequences for priority packets in big data smart cities environments.In this paper,we compare between our proposed Priority Packets Deadline First scheduling scheme(PPDF)with existing and traditional scheduling algorithms that can be used in urgent smart cities applications in order to illustrate the outstanding network performance parameters of our scheme such as the average waiting time,packets loss rates,priority packets end-to-end delay,and efficient energy consumption.

MapReduce in the Cloud: Data-Location-Aware VM Scheduling

We have witnessed the fast-growing deployment of Hadoop,an open-source implementation of the MapReduce programming model,for purpose of data-intensive computing in the cloud.However,Hadoop was not originally designed to run transient jobs in which us ers need to move data back and forth between storage and computing facilities.As a result,Hadoop is inefficient and wastes resources when operating in the cloud.This paper discusses the inefficiency of MapReduce in the cloud.We study the causes of this inefficiency and propose a solution.Inefficiency mainly occurs during data movement.Transferring large data to computing nodes is very time-con suming and also violates the rationale of Hadoop,which is to move computation to the data.To address this issue,we developed a dis tributed cache system and virtual machine scheduler.We show that our prototype can improve performance significantly when run ning different applications.

A Multi-objective Optimization Data Scheduling Algorithm for P2P Video Streaming

In P2P video streaming, each peer requests its wanted streaming data from others and responses others' requests by its data scheduling algorithm. Recent years, some data scheduling algorithms are proposed either to optimize the perceived video quality, or to optimize the network throughput. However, optimizing the perceived video quality may lead to low utilization of the senders'upload capacity. On the other hand, optimizing the network throughput may lead to the degrading perceived quality, for some emergent data may not be transmitted in time. In this paper, to improve the two objectives simultaneously, we formulate the data scheduling problem as a multi-objective model. In the formulation, we not only consider the segment quality and emergency which affect the perceived video quality, but also consider the rarity of the segments, which influences the network throughput. Then, we propose a distributed data scheduling algorithm to solve the multi-objective problem in polynomial time. Through simulations, we show the proposed algorithm outperforms other conventional algorithms in perceived video quality and utilization of peers' upload capacity.

Hypergraph-Based Data Reduced Scheduling Policy for Data-Intensive Workflow in Clouds

Data-intensive computing is expected to be the next-generation IT computing paradigm. Data-intensive workflows in clouds are becoming more and more popular. How to schedule data-intensive workflow efficiently has become the key issue. In this paper, first, we build a directed hypergraph model for data-intensive workflow, since Hypergraphs can more accurately model communication volume and better represent asymmetric problems, and the cut metric of hypergraphs is well suited for minimizing the total volume of communication.Second, we propose a concept data supportive ability to help the presentation of data-intensive workflow application and provide the merge operation details considering the data supportive ability. Third, we present an optimized hypergraph multi-level partitioning algorithm. Finally we bring a data reduced scheduling policy HEFT-P for data-intensive workflow. Through simulation,we compare HEFT-P with three typical workflow scheduling policies.The results indicate that HEFT-P could obtain reduced data scheduling and reduce the makespan of executing data-intensive

Fine-Grained Resource Provisioning and Task Scheduling for Heterogeneous Applications in Distributed Green Clouds

An increasing number of enterprises have adopted cloud computing to manage their important business applications in distributed green cloud(DGC)systems for low response time and high cost-effectiveness in recent years.Task scheduling and resource allocation in DGCs have gained more attention in both academia and industry as they are costly to manage because of high energy consumption.Many factors in DGCs,e.g.,prices of power grid,and the amount of green energy express strong spatial variations.The dramatic increase of arriving tasks brings a big challenge to minimize the energy cost of a DGC provider in a market where above factors all possess spatial variations.This work adopts a G/G/1 queuing system to analyze the performance of servers in DGCs.Based on it,a single-objective constrained optimization problem is formulated and solved by a proposed simulated-annealing-based bees algorithm(SBA)to find SBA can minimize the energy cost of a DGC provider by optimally allocating tasks of heterogeneous applications among multiple DGCs,and specifying the running speed of each server and the number of powered-on servers in each GC while strictly meeting response time limits of tasks of all applications.Realistic databased experimental results prove that SBA achieves lower energy cost than several benchmark scheduling methods do.

Job schedulers for Big data processing in Hadoop environment： testing real-life schedulers using benchmark programs

At present, big data is very popular, because it has proved to be much successful in many fields such as social media, E-commerce transactions, etc. Big data describes the tools and technologies needed to capture, manage, store, distribute, and analyze petabyte or larger-sized datasets having different structures with high speed. Big data can be structured, unstructured, or semi structured. Hadoop is an open source framework that is used to process large amounts of data in an inexpensive and efficient way, and job scheduling is a key factor for achieving high performance in big data processing. This paper gives an overview of big data and highlights the problems and challenges in big data. It then highlights Hadoop Distributed File System （HDFS）, Hadoop MapReduce, and various parameters that affect the performance of job scheduling algorithms in big data such as Job Tracker, Task Tracker, Name Node, Data Node, etc. The primary purpose of this paper is to present a comparative study of job scheduling algorithms along with their experimental results in Hadoop environment. In addition, this paper describes the advantages, disadvantages, features, and drawbacks of various Hadoop job schedulers such as FIFO, Fair, capacity, Deadline Constraints, Delay, LATE, Resource Aware, etc, and provides a comparative study among these schedulers.