Q-Learning-Assisted Meta-Heuristics for Scheduling Distributed Hybrid Flow Shop Problems

The flow shop scheduling problem is important for the manufacturing industry.Effective flow shop scheduling can bring great benefits to the industry.However,there are few types of research on Distributed Hybrid Flow Shop Problems(DHFSP)by learning assisted meta-heuristics.This work addresses a DHFSP with minimizing the maximum completion time(Makespan).First,a mathematical model is developed for the concerned DHFSP.Second,four Q-learning-assisted meta-heuristics,e.g.,genetic algorithm(GA),artificial bee colony algorithm(ABC),particle swarm optimization(PSO),and differential evolution(DE),are proposed.According to the nature of DHFSP,six local search operations are designed for finding high-quality solutions in local space.Instead of randomselection,Q-learning assists meta-heuristics in choosing the appropriate local search operations during iterations.Finally,based on 60 cases,comprehensive numerical experiments are conducted to assess the effectiveness of the proposed algorithms.The experimental results and discussions prove that using Q-learning to select appropriate local search operations is more effective than the random strategy.To verify the competitiveness of the Q-learning assistedmeta-heuristics,they are compared with the improved iterated greedy algorithm(IIG),which is also for solving DHFSP.The Friedman test is executed on the results by five algorithms.It is concluded that the performance of four Q-learning-assisted meta-heuristics are better than IIG,and the Q-learning-assisted PSO shows the best competitiveness.

Energy-Saving Distributed Flexible Job Shop Scheduling Optimization with Dual Resource Constraints Based on Integrated Q-Learning Multi-Objective Grey Wolf Optimizer

The distributed flexible job shop scheduling problem(DFJSP)has attracted great attention with the growth of the global manufacturing industry.General DFJSP research only considers machine constraints and ignores worker constraints.As one critical factor of production,effective utilization of worker resources can increase productivity.Meanwhile,energy consumption is a growing concern due to the increasingly serious environmental issues.Therefore,the distributed flexible job shop scheduling problem with dual resource constraints(DFJSP-DRC)for minimizing makespan and total energy consumption is studied in this paper.To solve the problem,we present a multi-objective mathematical model for DFJSP-DRC and propose a Q-learning-based multi-objective grey wolf optimizer(Q-MOGWO).In Q-MOGWO,high-quality initial solutions are generated by a hybrid initialization strategy,and an improved active decoding strategy is designed to obtain the scheduling schemes.To further enhance the local search capability and expand the solution space,two wolf predation strategies and three critical factory neighborhood structures based on Q-learning are proposed.These strategies and structures enable Q-MOGWO to explore the solution space more efficiently and thus find better Pareto solutions.The effectiveness of Q-MOGWO in addressing DFJSP-DRC is verified through comparison with four algorithms using 45 instances.The results reveal that Q-MOGWO outperforms comparison algorithms in terms of solution quality.

GRU-integrated constrained soft actor-critic learning enabled fully distributed scheduling strategy for residential virtual power plant

In this study,a novel residential virtual power plant(RVPP)scheduling method that leverages a gate recurrent unit(GRU)-integrated deep reinforcement learning(DRL)algorithm is proposed.In the proposed scheme,the GRU-integrated DRL algorithm guides the RVPP to participate effectively in both the day-ahead and real-time markets,lowering the electricity purchase costs and consumption risks for end-users.The Lagrangian relaxation technique is introduced to transform the constrained Markov decision process(CMDP)into an unconstrained optimization problem,which guarantees that the constraints are strictly satisfied without determining the penalty coefficients.Furthermore,to enhance the scalability of the constrained soft actor-critic(CSAC)-based RVPP scheduling approach,a fully distributed scheduling architecture was designed to enable plug-and-play in the residential distributed energy resources(RDER).Case studies performed on the constructed RVPP scenario validated the performance of the proposed methodology in enhancing the responsiveness of the RDER to power tariffs,balancing the supply and demand of the power grid,and ensuring customer comfort.

Dynamic access task scheduling of LEO constellation based on space-based distributed computing

A dynamic multi-beam resource allocation algorithm for large low Earth orbit(LEO)constellation based on on-board distributed computing is proposed in this paper.The allocation is a combinatorial optimization process under a series of complex constraints,which is important for enhancing the matching between resources and requirements.A complex algorithm is not available because that the LEO on-board resources is limi-ted.The proposed genetic algorithm(GA)based on two-dimen-sional individual model and uncorrelated single paternal inheri-tance method is designed to support distributed computation to enhance the feasibility of on-board application.A distributed system composed of eight embedded devices is built to verify the algorithm.A typical scenario is built in the system to evalu-ate the resource allocation process,algorithm mathematical model,trigger strategy,and distributed computation architec-ture.According to the simulation and measurement results,the proposed algorithm can provide an allocation result for more than 1500 tasks in 14 s and the success rate is more than 91%in a typical scene.The response time is decreased by 40%com-pared with the conditional GA.

Research on Scheduling Strategy of Flexible Interconnection Distribution Network Considering Distributed Photovoltaic and Hydrogen Energy Storage

Distributed photovoltaic(PV)is one of the important power sources for building a new power system with new energy as the main body.The rapid development of distributed PV has brought new challenges to the operation of distribution networks.In order to improve the absorption ability of large-scale distributed PV access to the distribution network,the AC/DC hybrid distribution network is constructed based on flexible interconnection technology,and a coordinated scheduling strategy model of hydrogen energy storage(HS)and distributed PV is established.Firstly,the mathematical model of distributed PV and HS system is established,and a comprehensive energy storage system combining seasonal hydrogen energy storage(SHS)and battery(BT)is proposed.Then,a flexible interconnected distribution network scheduling optimization model is established to minimize the total active power loss,voltage deviation and system operating cost.Finally,simulation analysis is carried out on the improved IEEE33 node,the NSGA-II algorithm is used to solve specific examples,and the optimal scheduling results of the comprehensive economy and power quality of the distribution network are obtained.Compared with the method that does not consider HS and flexible interconnection technology,the network loss and voltage deviation of this method are lower,and the total system cost can be reduced by 3.55%,which verifies the effectiveness of the proposed method.

Data-driven Wasserstein distributionally robust chance-constrained optimization for crude oil scheduling under uncertainty

Crude oil scheduling optimization is an effective method to enhance the economic benefits of oil refining.But uncertainties,including uncertain demands of crude distillation units(CDUs),might make the production plans made by the traditional deterministic optimization models infeasible.A data-driven Wasserstein distributionally robust chance-constrained(WDRCC)optimization approach is proposed in this paper to deal with demand uncertainty in crude oil scheduling.First,a new deterministic crude oil scheduling optimization model is developed as the basis of this approach.The Wasserstein distance is then used to build ambiguity sets from historical data to describe the possible realizations of probability distributions of uncertain demands.A cross-validation method is advanced to choose suitable radii for these ambiguity sets.The deterministic model is reformulated as a WDRCC optimization model for crude oil scheduling to guarantee the demand constraints hold with a desired high probability even in the worst situation in ambiguity sets.The proposed WDRCC model is transferred into an equivalent conditional value-at-risk representation and further derived as a mixed-integer nonlinear programming counterpart.Industrial case studies from a real-world refinery are conducted to show the effectiveness of the proposed method.Out-of-sample tests demonstrate that the solution of the WDRCC model is more robust than those of the deterministic model and the chance-constrained model.

Competitive and Cooperative-Based Strength Pareto Evolutionary Algorithm for Green Distributed Heterogeneous Flow Shop Scheduling

This work aims to resolve the distributed heterogeneous permutation flow shop scheduling problem(DHPFSP)with minimizing makespan and total energy consumption(TEC).To solve this NP-hard problem,this work proposed a competitive and cooperative-based strength Pareto evolutionary algorithm(CCSPEA)which contains the following features:1)An initialization based on three heuristic rules is developed to generate a population with great diversity and convergence.2)A comprehensive metric combining convergence and diversity metrics is used to better represent the heuristic information of a solution.3)A competitive selection is designed which divides the population into a winner and a loser swarms based on the comprehensive metric.4)A cooperative evolutionary schema is proposed for winner and loser swarms to accelerate the convergence of global search.5)Five local search strategies based on problem knowledge are designed to improve convergence.6)Aproblem-based energy-saving strategy is presented to reduce TEC.Finally,to evaluate the performance of CCSPEA,it is compared to four state-of-art and run on 22 instances based on the Taillard benchmark.The numerical experiment results demonstrate that 1)the proposed comprehensive metric can efficiently represent the heuristic information of each solution to help the later step divide the population.2)The global search based on the competitive and cooperative schema can accelerate loser solutions convergence and further improve the winner’s exploration.3)The problembased initialization,local search,and energy-saving strategies can efficiently reduce the makespan and TEC.4)The proposed CCSPEA is superior to the state-of-art for solving DHPFSP.

Trusted Data Acquisition Mechanism for Cloud Resource Scheduling Based on Distributed Agents

Goud computing is a new paradigm in which dynamic and virtualized computing resources are provided as services over the Internet. However, because cloud resource is open and dynamically configured, resource allocation and scheduling are extremely important challenges in cloud infrastructure. Based on distributed agents, this paper presents trusted data acquisition mechanism for efficient scheduling cloud resources to satisfy various user requests. Our mechanism defines, collects and analyzes multiple key trust targets of cloud service resources based on historical information of servers in a cloud data center. As a result, using our trust computing mechanism, cloud providers can utilize their resources efficiently and also provide highly trusted resources and services to many users.

Fine-Grained Resource Provisioning and Task Scheduling for Heterogeneous Applications in Distributed Green Clouds

An increasing number of enterprises have adopted cloud computing to manage their important business applications in distributed green cloud(DGC)systems for low response time and high cost-effectiveness in recent years.Task scheduling and resource allocation in DGCs have gained more attention in both academia and industry as they are costly to manage because of high energy consumption.Many factors in DGCs,e.g.,prices of power grid,and the amount of green energy express strong spatial variations.The dramatic increase of arriving tasks brings a big challenge to minimize the energy cost of a DGC provider in a market where above factors all possess spatial variations.This work adopts a G/G/1 queuing system to analyze the performance of servers in DGCs.Based on it,a single-objective constrained optimization problem is formulated and solved by a proposed simulated-annealing-based bees algorithm(SBA)to find SBA can minimize the energy cost of a DGC provider by optimally allocating tasks of heterogeneous applications among multiple DGCs,and specifying the running speed of each server and the number of powered-on servers in each GC while strictly meeting response time limits of tasks of all applications.Realistic databased experimental results prove that SBA achieves lower energy cost than several benchmark scheduling methods do.

A Hybrid Genetic Scheduling Algorithm to Heterogeneous Distributed System

In parallel and distributed computing, development of an efficient static task scheduling algorithm for directed acyclic graph (DAG) applications is an important problem. The static task scheduling problem is NP-complete in its general form. The complexity of the problem increases when task scheduling is to be done in a heterogeneous environment, consisting of processors with varying processing capabilities and network links with varying bandwidths. List scheduling algorithms are generally preferred since they generate good quality schedules with less complexity. But these list algorithms leave a lot of room for improvement, especially when these algorithms are used in specialized heterogeneous environments This paper presents an hybrid genetic task scheduling algorithm for the tasks run on the network of heterogeneous systems and represented by Directed Acyclic Graphs (DAGs). First, the algorithm assigns a coupling factor to each task to present the tasks should be scheduled onto the same processor by avoiding the large communication time. Second, the algorithm generate some high quality initial solution by scheduling the tasks which are strongly coupled with each other onto the same processor, and improve the quality of the solution by using coupling initial solutions, random solution, near optimal solutions obtained by the list scheduling algorithm in the crossover and mutation operator. The performance of the algorithm is illustrated by comparing with the existing effectively scheduling algorithms.

Metaheuristic Based Resource Scheduling Technique for Distributed Robotic Control Systems

The design of controllers for robots is a complex system that is to be dealt with several tasks in real time for enabling the robots to function independently.The distributed robotic control system can be used in real time for resolving various challenges such as localization,motion controlling,mapping,route planning,etc.The distributed robotic control system can manage different kinds of heterogenous devices.Designing a distributed robotic control system is a challenging process as it needs to operate effectually under different hardware configurations and varying computational requirements.For instance,scheduling of resources(such as communication channel,computation unit,robot chassis,or sensor input)to the various system components turns out to be an essential requirement for completing the tasks on time.Therefore,resource scheduling is necessary for ensuring effective execution.In this regard,this paper introduces a novel chaotic shell game optimization algorithm(CSGOA)for resource scheduling,known as the CSGOA-RS technique for the distributed robotic control system environment.The CSGOA technique is based on the integration of the chaotic maps concept to the SGO algorithm for enhancing the overall performance.The CSGOA-RS technique is designed for allocating the resources in such a way that the transfer time is minimized and the resource utilization is increased.The CSGOA-RS technique is applicable even for the unpredicted environment where the resources are to be allotted dynamically based on the early estimations.For validating the enhanced performance of the CSGOA-RS technique,a series of simulations have been carried out and the obtained results have been examined with respect to a selected set of measures.The resultant outcomes highlighted the promising performance of the CSGOA-RS technique over the other resource scheduling techniques.

Distributed fair queuing algorithm based on compensation coordination scheduling in WMN

This paper proposes a distributed fair queuing algorithm which is based on compensation coordi- nation scheduling in wireless mesh networks, considering such problems as the location-dependent competition and unfair channel bandwidth allocation among nodes. The data communication process requiring the establishment of compensation coordination scheduling model is divided into three periods： the sending period, the compensation period and the dormancy period. According to model parameters, time constraint functions are designed to limit the execution length of each period. The algorithms guarantee that the nodes complete fair transmission of network packets together in accordance with the fixed coordination scheduling rule of the model. Simulations and analysis demonstrate the effectiveness of the proposed algorithm in network throughput and fairness.

Distributed link scheduling method with physical interference model in wireless multi-hop networks

To further increase the throughput of wireless multi-hop networks,a distributed scheduling method is proposed,which takes physical interference model into account.It is assumed that nodes in the network can perform physical carrier sensing,and the carrier sensing range can be set to different values.In the traditional carrier sensing mechanism,the carrier sensing range is computed under the protocol interference model,which is not accurate.Here the optimal carrier sensing range with physical interference model is achieved.Each sending node implements the distributed approach in three phases at each time slot,and all the concurrent transmissions are interference free.Good performance can be achieved under this scheduling approach.The approximation ratio of the distributed method to the optimal one is also proved.

Proposal of Distributed Scheduling Heuristics Using Mediation Agent

This paper proposes an agent-based distributed scheduling system against the background of the deregulation of electric utility and the smart grid for the renewable energy, and then focuses on a maintenance scheduling in the context of real problems. A synchronous backtrack algorithm, a welD-known method for distributed scheduling problems, has difficulties handling （A） rapid schedule adjustments and （B） impartial assignment. Thus, this paper proposes two kinds of heuristics： （1） parallel assignment and （2） multiple priority strategies, and developed the distributed scheduling system which makes use of the heuristics. It consists of schedulers for each power station and mediation agents which have cloning and merging functions to support the implementation of the heuristics. Finally, the result of experiment shows an improvement when handling the rapid adjustment and the impartiality issues with reasonable computational overhead.

Novel operating theatre scheduling method based on estimation of distribution algorithm

In order to improve the efficiency of operating rooms,reduce the costs for hospitals and improve the level of service qualities, a scheduling method was developed based on an estimation of distribution algorithm（ EDA）. First, a scheduling problem domain is described. Based on assignment constraints and resource capacity constraints, the mathematical programming models are set up with an objective function to minimize the system makespan. On the basis of the descriptions mentioned above, a solution policy of generating feasible scheduling solutions is established. Combined with the specific constraints of operating theatres, the EDA-based algorithm is put forward to solve scheduling problems. Finally, simulation experiments are designed to evaluate the scheduling method. The orthogonal table is chosen to determine the parameters in the proposed method. Then the genetic algorithm and the particle swarm optimization algorithm are chosen for comparison with the EDA-based algorithm, and the results indicate that the proposed method can decrease the makespan of the surgical system regardless of the size of operations. Moreover, the computation time of the EDA-based algorithm is only approximately 5 s when solving the large scale problems, which means that the proposed algorithm is suitable for carrying out an on-line scheduling optimization of the patients.

Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput.This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems.First,integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing,which reduces the search scope of the database and dramatically speeds up data searching.Next,exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first,which reduces the average waiting time of job execution.As a result,the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing,significantly improving the speed of data retrieval up to 53%and increasing system throughput by 53%.On the other hand,the proposed job scheduling algorithmdefeats both first-in-first-out andmemory-sensitive heterogeneous early finish time scheduling algorithms,effectively shortening the average waiting time up to 5%and average weighted turnaround time by 19%,respectively.

A novel hybrid estimation of distribution algorithm for solving hybrid flowshop scheduling problem with unrelated parallel machine

The hybrid flow shop scheduling problem with unrelated parallel machine is a typical NP-hard combinatorial optimization problem, and it exists widely in chemical, manufacturing and pharmaceutical industry. In this work, a novel mathematic model for the hybrid flow shop scheduling problem with unrelated parallel machine(HFSPUPM) was proposed. Additionally, an effective hybrid estimation of distribution algorithm was proposed to solve the HFSPUPM, taking advantage of the features in the mathematic model. In the optimization algorithm, a new individual representation method was adopted. The(EDA) structure was used for global search while the teaching learning based optimization(TLBO) strategy was used for local search. Based on the structure of the HFSPUPM, this work presents a series of discrete operations. Simulation results show the effectiveness of the proposed hybrid algorithm compared with other algorithms.

Production Scheduling and Distribution in Downstream Sector Using Block-Structured Linear Programming Solution Technique ： A Comparative Analysis

The aim of this paper is to compare block-structured linear programming （LP） models against other practical optimization methods for solving downstream product refinery problems using a solution method different from the existing ones （like mixed integer linear programming （MILP） method）. The work X-rays the Nigerian petroleum refining industries and their channel of distribution in the local setting and identifies the critical features of scheduling and allocation of refined crude products; either for distribution within the country or for exportation to the international market. Applying our model to the distribution model, the computational results reveal a better route with lowest transportation cost for the scheduling problem and the best optimal blend with higher revenue for the production problem.

An effective estimation of distribution algorithm for parallel litho machine scheduling with reticle constraints

In order to improve the scheduling efficiency of photolithography,bottleneck process of wafer fabrications in the semiconductor industry,an effective estimation of distribution algorithm is proposed for scheduling problems of parallel litho machines with reticle constraints,where multiple reticles are available for each reticle type.First,the scheduling problem domain of parallel litho machines is described with reticle constraints and mathematical programming formulations are put forward with the objective of minimizing total weighted completion time.Second,estimation of distribution algorithm is developed with a decoding scheme specially designed to deal with the reticle constraints.Third,an insert-based local search with the first move strategy is introduced to enhance the local exploitation ability of the algorithm.Finally,simulation experiments and analysis demonstrate the effectiveness of the proposed algorithm.

Improved Estimation of Distribution Algorithm for Solving Unrelated Parallel Machine Scheduling Problem

Scheduling problem is a well-known combinatorial optimization problem.An effective improved estimation of distribution algorithm(IEDA) was proposed for minimizing the makespan of the unrelated parallel machine scheduling problem(UPMSP).Mathematical description was given for the UPMSP.The IEDA which was combined with variable neighborhood search(IEDA_VNS) was proposed to solve the UPMSP in order to improve local search ability.A new encoding method was designed for representing the feasible solutions of the UPMSP.More knowledge of the UPMSP were taken consideration in IEDA_ VNS for probability matrix which was based the processing time matrix.The simulation results show that the proposed IEDA_VNS can solve the problem effectively.