Q-Learning-Assisted Meta-Heuristics for Scheduling Distributed Hybrid Flow Shop Problems

The flow shop scheduling problem is important for the manufacturing industry.Effective flow shop scheduling can bring great benefits to the industry.However,there are few types of research on Distributed Hybrid Flow Shop Problems(DHFSP)by learning assisted meta-heuristics.This work addresses a DHFSP with minimizing the maximum completion time(Makespan).First,a mathematical model is developed for the concerned DHFSP.Second,four Q-learning-assisted meta-heuristics,e.g.,genetic algorithm(GA),artificial bee colony algorithm(ABC),particle swarm optimization(PSO),and differential evolution(DE),are proposed.According to the nature of DHFSP,six local search operations are designed for finding high-quality solutions in local space.Instead of randomselection,Q-learning assists meta-heuristics in choosing the appropriate local search operations during iterations.Finally,based on 60 cases,comprehensive numerical experiments are conducted to assess the effectiveness of the proposed algorithms.The experimental results and discussions prove that using Q-learning to select appropriate local search operations is more effective than the random strategy.To verify the competitiveness of the Q-learning assistedmeta-heuristics,they are compared with the improved iterated greedy algorithm(IIG),which is also for solving DHFSP.The Friedman test is executed on the results by five algorithms.It is concluded that the performance of four Q-learning-assisted meta-heuristics are better than IIG,and the Q-learning-assisted PSO shows the best competitiveness.

GRU-integrated constrained soft actor-critic learning enabled fully distributed scheduling strategy for residential virtual power plant

In this study,a novel residential virtual power plant(RVPP)scheduling method that leverages a gate recurrent unit(GRU)-integrated deep reinforcement learning(DRL)algorithm is proposed.In the proposed scheme,the GRU-integrated DRL algorithm guides the RVPP to participate effectively in both the day-ahead and real-time markets,lowering the electricity purchase costs and consumption risks for end-users.The Lagrangian relaxation technique is introduced to transform the constrained Markov decision process(CMDP)into an unconstrained optimization problem,which guarantees that the constraints are strictly satisfied without determining the penalty coefficients.Furthermore,to enhance the scalability of the constrained soft actor-critic(CSAC)-based RVPP scheduling approach,a fully distributed scheduling architecture was designed to enable plug-and-play in the residential distributed energy resources(RDER).Case studies performed on the constructed RVPP scenario validated the performance of the proposed methodology in enhancing the responsiveness of the RDER to power tariffs,balancing the supply and demand of the power grid,and ensuring customer comfort.

Energy-Saving Distributed Flexible Job Shop Scheduling Optimization with Dual Resource Constraints Based on Integrated Q-Learning Multi-Objective Grey Wolf Optimizer

The distributed flexible job shop scheduling problem(DFJSP)has attracted great attention with the growth of the global manufacturing industry.General DFJSP research only considers machine constraints and ignores worker constraints.As one critical factor of production,effective utilization of worker resources can increase productivity.Meanwhile,energy consumption is a growing concern due to the increasingly serious environmental issues.Therefore,the distributed flexible job shop scheduling problem with dual resource constraints(DFJSP-DRC)for minimizing makespan and total energy consumption is studied in this paper.To solve the problem,we present a multi-objective mathematical model for DFJSP-DRC and propose a Q-learning-based multi-objective grey wolf optimizer(Q-MOGWO).In Q-MOGWO,high-quality initial solutions are generated by a hybrid initialization strategy,and an improved active decoding strategy is designed to obtain the scheduling schemes.To further enhance the local search capability and expand the solution space,two wolf predation strategies and three critical factory neighborhood structures based on Q-learning are proposed.These strategies and structures enable Q-MOGWO to explore the solution space more efficiently and thus find better Pareto solutions.The effectiveness of Q-MOGWO in addressing DFJSP-DRC is verified through comparison with four algorithms using 45 instances.The results reveal that Q-MOGWO outperforms comparison algorithms in terms of solution quality.

Competitive and Cooperative-Based Strength Pareto Evolutionary Algorithm for Green Distributed Heterogeneous Flow Shop Scheduling

This work aims to resolve the distributed heterogeneous permutation flow shop scheduling problem(DHPFSP)with minimizing makespan and total energy consumption(TEC).To solve this NP-hard problem,this work proposed a competitive and cooperative-based strength Pareto evolutionary algorithm(CCSPEA)which contains the following features:1)An initialization based on three heuristic rules is developed to generate a population with great diversity and convergence.2)A comprehensive metric combining convergence and diversity metrics is used to better represent the heuristic information of a solution.3)A competitive selection is designed which divides the population into a winner and a loser swarms based on the comprehensive metric.4)A cooperative evolutionary schema is proposed for winner and loser swarms to accelerate the convergence of global search.5)Five local search strategies based on problem knowledge are designed to improve convergence.6)Aproblem-based energy-saving strategy is presented to reduce TEC.Finally,to evaluate the performance of CCSPEA,it is compared to four state-of-art and run on 22 instances based on the Taillard benchmark.The numerical experiment results demonstrate that 1)the proposed comprehensive metric can efficiently represent the heuristic information of each solution to help the later step divide the population.2)The global search based on the competitive and cooperative schema can accelerate loser solutions convergence and further improve the winner’s exploration.3)The problembased initialization,local search,and energy-saving strategies can efficiently reduce the makespan and TEC.4)The proposed CCSPEA is superior to the state-of-art for solving DHPFSP.

Research on task scheduling and concurrent processing technology for energy internet operation platform

The energy Internet operation platform provides market entities such as energy users,energy enterprises,suppliers,and governments with the ability to interact,transact,and manage various operations.Owing to the large number of platform users,complex businesses,and large amounts of data-mining tasks,it is necessary to solve the problems afflicting platform task scheduling and the provision of simultaneous access to a large number of users.This study examines the two core technologies of platform task scheduling and multiuser concurrent processing,proposing a distributed task-scheduling method and a technical implementation scheme based on the particle swarm optimization algorithm,and presents a systematic solution in concurrent processing for massive user numbers.Based on the results of this study,the energy internet operation platform can effectively deal with the concurrent access of tens of millions of users and complex task-scheduling problems.

Distributionally Robust Scheduling for Benefit Allocation in Regional Integrated Energy System with Multiple Stakeholders

A distributionally robust scheduling strategy is proposed to address the complex benefit allocation problem in regional integrated energy systems(RIESs)with multiple stakeholders.A two-level Stackelberg game model is established,with the RIES operator as the leader and the users as the followers.It considers the interests of the RIES operator and demand response users in energy trading.The leader optimizes time-of-use(TOU)energy prices to minimize costs while users formulate response plans based on prices.A two-stage distributionally robust game model with comprehensive norm constraints,which encompasses the two-level Stackelberg game model in the day-ahead scheduling stage,is constructed to manage wind power uncertainty.Karush-Kuhn-Tucker(KKT)conditions transform the two-level Stackelberg game model into a single-level robust optimization model,which is then solved using column and constraint generation(C&CG).Numerical results demonstrate the effectiveness of the proposed strategy in balancing stakeholders'interests and mitigating wind power risks.

Root length density distribution and associated soil water dynamics for tomato plants under furrow irrigation in a solar greenhouse

Furrow irrigation is a traditional widely-used irrigation method in the world. Understanding the dynamics of soil water distribution is essential to developing effective furrow irrigation strategies, especially in water-limited regions. The objectives of this study are to analyze root length density distribution and to explore soil water dynamics by simulating soil water content using a HYDRUS-2D model with consideration of root water uptake for furrow irrigated tomato plants in a solar greenhouse in Northwest China. Soil water contents were also in-situ observed by the ECH_2O sensors from 4 June to 19 June and from 21 June to 4 July, 2012. Results showed that the root length density of tomato plants was concentrated in the 0–50 cm soil layers, and radiated 0–18 cm toward the furrow and 0–30 cm along the bed axis. Soil water content values simulated by the HYDRUS-2D model agreed well with those observed by the ECH_2O sensors, with regression coefficient of 0.988, coefficient of determination of 0.89, and index of agreement of 0.97. The HYDRUS-2D model with the calibrated parameters was then applied to explore the optimal irrigation scheduling. Infrequent irrigation with a large amount of water for each irrigation event could result in 10%–18% of the irrigation water losses. Thus we recommend high irrigation frequency with a low amount of water for each irrigation event in greenhouses for arid region. The maximum high irrigation amount and the suitable irrigation interval required to avoid plant water stress and drainage water were 34 mm and 6 days, respectively, for given daily average transpiration rate of 4.0 mm/d. To sum up, the HYDRUS-2D model with consideration of root water uptake can be used to improve irrigation scheduling for furrow irrigated tomato plants in greenhouses in arid regions.

A CENTRAL-STAGE BUFFERED THREE-STAGE CLOS SWITCHING FABRIC AND THE SCHEDULING ALGORITHM

Current MSM switching fabric has poor performance under unbalanced traffic. This paper presents an alternative, novel Central-stage Buffered Three-stage Clos switching (CB-3Clos) fabric and proves that this fabric can emulate output queuing switch without any speedup. By analyzing the condition to satisfy the central-stage load-balance, this paper also proposes a Central-stage Load-balanced-based Distributed Scheduling algorithm (CLDS) for CB-3Clos. The results show that, compared with Concurrent Round-Robin based Dispatching (CRRD) algorithm based on MSM, CLDS algorithm has high throughput irrespective with the traffic model and better performance in mean packet delay.

Distributed fair queuing algorithm based on compensation coordination scheduling in WMN

This paper proposes a distributed fair queuing algorithm which is based on compensation coordi- nation scheduling in wireless mesh networks, considering such problems as the location-dependent competition and unfair channel bandwidth allocation among nodes. The data communication process requiring the establishment of compensation coordination scheduling model is divided into three periods： the sending period, the compensation period and the dormancy period. According to model parameters, time constraint functions are designed to limit the execution length of each period. The algorithms guarantee that the nodes complete fair transmission of network packets together in accordance with the fixed coordination scheduling rule of the model. Simulations and analysis demonstrate the effectiveness of the proposed algorithm in network throughput and fairness.

Distributed link scheduling method with physical interference model in wireless multi-hop networks

To further increase the throughput of wireless multi-hop networks,a distributed scheduling method is proposed,which takes physical interference model into account.It is assumed that nodes in the network can perform physical carrier sensing,and the carrier sensing range can be set to different values.In the traditional carrier sensing mechanism,the carrier sensing range is computed under the protocol interference model,which is not accurate.Here the optimal carrier sensing range with physical interference model is achieved.Each sending node implements the distributed approach in three phases at each time slot,and all the concurrent transmissions are interference free.Good performance can be achieved under this scheduling approach.The approximation ratio of the distributed method to the optimal one is also proved.

Proposal of Distributed Scheduling Heuristics Using Mediation Agent

This paper proposes an agent-based distributed scheduling system against the background of the deregulation of electric utility and the smart grid for the renewable energy, and then focuses on a maintenance scheduling in the context of real problems. A synchronous backtrack algorithm, a welD-known method for distributed scheduling problems, has difficulties handling （A） rapid schedule adjustments and （B） impartial assignment. Thus, this paper proposes two kinds of heuristics： （1） parallel assignment and （2） multiple priority strategies, and developed the distributed scheduling system which makes use of the heuristics. It consists of schedulers for each power station and mediation agents which have cloning and merging functions to support the implementation of the heuristics. Finally, the result of experiment shows an improvement when handling the rapid adjustment and the impartiality issues with reasonable computational overhead.

Resource Load Prediction of Internet of Vehicles Mobile Cloud Computing

Load-time series data in mobile cloud computing of Internet of Vehicles(IoV)usually have linear and nonlinear composite characteristics.In order to accurately describe the dynamic change trend of such loads,this study designs a load prediction method by using the resource scheduling model for mobile cloud computing of IoV.Firstly,a chaotic analysis algorithm is implemented to process the load-time series,while some learning samples of load prediction are constructed.Secondly,a support vector machine(SVM)is used to establish a load prediction model,and an improved artificial bee colony(IABC)function is designed to enhance the learning ability of the SVM.Finally,a CloudSim simulation platform is created to select the perminute CPU load history data in the mobile cloud computing system,which is composed of 50 vehicles as the data set;and a comparison experiment is conducted by using a grey model,a back propagation neural network,a radial basis function(RBF)neural network and a RBF kernel function of SVM.As shown in the experimental results,the prediction accuracy of the method proposed in this study is significantly higher than other models,with a significantly reduced real-time prediction error for resource loading in mobile cloud environments.Compared with single-prediction models,the prediction method proposed can build up multidimensional time series in capturing complex load time series,fit and describe the load change trends,approximate the load time variability more precisely,and deliver strong generalization ability to load prediction models for mobile cloud computing resources.

Distributed Scheduling Problems in Intelligent Manufacturing Systems

Currently,manufacturing enterprises face increasingly fierce market competition due to the various demands of customers and the rapid development of economic globalization.Hence,they have to extend their production mode into distributed environments and establish multiple factories in various geographical locations.Nowadays,distributed manufacturing systems have been widely adopted in industrial production processes.In recent years,many studies have been done on the modeling and optimization of distributed scheduling problems.This work provides a literature review on distributed scheduling problems in intelligent manufacturing systems.By summarizing and evaluating existing studies on distributed scheduling problems,we analyze the achievements and current research status in this field and discuss ongoing studies.Insights regarding prior works are discussed to uncover future research directions,particularly swarm intelligence and evolutionary algorithms,which are used for managing distributed scheduling problems in manufacturing systems.This work focuses on journal papers discovered using Google Scholar.After reviewing the papers,in this work,we discuss the research trends of distributed scheduling problems and point out some directions for future studies.

Multi-objective Optimization of the Distributed Permutation Flow Shop Scheduling Problem with Transportation and Eligibility Constraints

In this paper,we consider the distributed permutation flow shop scheduling problem(DPFSSP)with transportation and eligibility constrains.Three objectives are taken into account,i.e.,makespan,maximum lateness and total costs(transportation costs and setup costs).To the best of our knowledge,there is no published work on multi-objective optimization of the DPFSSP with transportation and eligibility constraints.First,we present the mathematics model and constructive heuristics for single objective;then,we propose an improved The Nondominated Sorting Genetic Algorithm II(NSGA-II)for the multi-objective DPFSSP to find Pareto optimal solutions,in which a novel solution representation,a new population re-/initialization,effective crossover and mutation operators,as well as local search methods are developed.Based on extensive computational and statistical experiments,the proposed algorithm performs better than the well-known NSGA-II and the Strength Pareto Evolutionary Algorithm 2(SPEA2).

Distributed scheduling based on polling policy with maximal spatial reuse in multi-hop WMNs

The scheduling algorithm based on the three-way handshaking scheme in IEEE 802.16d-2004 standard has some serious problems because of the complexity of the algorithm and low scheduling efficiency. To enhance the scheduling efficiency and improve the performance of multi-hop wireless mesh networks （WMNs）, one distributed scheduling algorithm that can maximize the spatial and time reuse with an interference-based network model is proposed. Compared to the graph-based network model, the proposed network model can achieve a better throughput performance with maximal spatial reuse. Furthermore, this proposed scheduling algorithm also keeps fairly scheduling to all links, with a priority-based polling policy. Both the theoretical analysis and simulation results show that this proposed distributed scheduling algorithm is simple and efficient.

Improved gray wolf optimizer for distributed flexible job shop scheduling problem

The distributed flexible job shop scheduling problem(DFJSP),which is an extension of the flexible job shop scheduling problem,is a famous NP-complete combinatorial optimization problem.This problem is widespread in the manufacturing industries and comprises the following three subproblems:the assignment of jobs to factories,the scheduling of operations to machines,and the sequence of operations on machines.However,studies on DFJSP are seldom because of its difficulty.This paper proposes an effective improved gray wolf optimizer(IGWO)to solve the aforementioned problem.In this algorithm,new encoding and decoding schemes are designed to represent the three subproblems and transform the encoding into a feasible schedule,respectively.Four crossover operators are developed to expand the search space.A local search strategy with the concept of a critical factory is also proposed to improve the exploitability of IGWO.Effective schedules can be obtained by changing factory assignments and operation sequences in the critical factory.The proposed IGWO algorithm is evaluated on 69 famous benchmark instances and compared with six state-of-the-art algorithms to demonstrate its efficacy considering solution quality and computational efficiency.Experimental results show that the proposed algorithm has achieved good improvement.Particularly,the proposed IGWO updates the new upper bounds of 13 difficult benchmark instances.

QoS-TEOS:QoS Guaranteed Throughput-Efficiency Optimal Distributed Scheduling in WiMAX Mesh Networks

WiMAX distributed scheduling can be modeled as two procedures：three-way handshaking procedure and data subframe scheduling procedure.Due to manipulating data transmission directly,data subframe scheduling has a closer relationship with user Quality of Service（QoS） satisfaction,and has more severe impact on network performance,compared with handshaking procedure.A QoS guaranteed Throughput-Efficiency Optimal distributed data subframe Scheduling scheme,named as QoS-TEOS,is proposed.QoS-TEOS achieves QoS guarantee through modeling services into different ranks and assigning them with corresponding priorities.A service with higher priority is scheduled ahead of that with lower priority and offered with high QoS quality.Same kinds of services that request similar QoS quality are classified into one service set.Different service sets are scheduled with different strategies.QoS-TEOS promotes network performance through improving network throughput and efficiency.Theoretical analysis shows that the scheduled data transmission should balance data generation rate from upper layer and transmission rate of physical layer,to avoid network throughput and efficiency declining.Simulation results show that QoS-TEOS works excellently to achieve throughput-efficiency optimization and guarantee a high QoS.

Cooperated Bayesian algorithm for distributed scheduling problem

This paper presents a new distributed Bayesian optimization algorithm(BOA)to overcome the efficiency problem when solving NP scheduling problems.The pro-posed approach integrates BOA into the co-evolutionary schema,which builds up a concurrent computing environ-ment.A new search strategy is also introduced for local op-timization process.It integrates the reinforcement learning(RL)mechanism into the BOA search processes,and then uses the mixed probability information from BOA(post-probability)and RL(pre-probability)to enhance the cooperation between different local controllers,which im-proves the optimization ability of the algorithm.The ex-periment shows that the new algorithm does better in both optimization(2.2%)and convergence(11.7%),compared with classic BOA.

A Dynamic Load Balancing Mechanism for Distributed Systems

It is desirable in a distributed system to have the system load balanced evenly among the nodes so that the mean job response time is minimized. In this paper, we present.a dynamic load balancing mechanism (DLB). It adopts a centralized approach and is network topology independent. The DLB mechanism employs a set of thresholds which are automatically adjusted as the system load changes. lt also provides a simple mechanism for the system to switch between periodic and instantaneous load balancing policies with ease. The performance of the proposed algorithm is evaluated by intensive simulations for various parameters. The simulAtion results show that the mean job response time in a system implementing DLB algorithm is significantly lower than the same system without load balancings. Furthermore, compared with a previously proposed algorithm, DLB algorithm demonstrates improved performance, especially when the system is heavily loaded and the load is unevenly distributed.