Deep Reinforcement Learning Solves Job-shop Scheduling Problems

To solve the sparse reward problem of job-shop scheduling by deep reinforcement learning,a deep reinforcement learning framework considering sparse reward problem is proposed.The job shop scheduling problem is transformed into Markov decision process,and six state features are designed to improve the state feature representation by using two-way scheduling method,including four state features that distinguish the optimal action and two state features that are related to the learning goal.An extended variant of graph isomorphic network GIN++is used to encode disjunction graphs to improve the performance and generalization ability of the model.Through iterative greedy algorithm,random strategy is generated as the initial strategy,and the action with the maximum information gain is selected to expand it to optimize the exploration ability of Actor-Critic algorithm.Through validation of the trained policy model on multiple public test data sets and comparison with other advanced DRL methods and scheduling rules,the proposed method reduces the minimum average gap by 3.49%,5.31%and 4.16%,respectively,compared with the priority rule-based method,and 5.34%compared with the learning-based method.11.97%and 5.02%,effectively improving the accuracy of DRL to solve the approximate solution of JSSP minimum completion time.

Q-Learning-Assisted Meta-Heuristics for Scheduling Distributed Hybrid Flow Shop Problems

The flow shop scheduling problem is important for the manufacturing industry.Effective flow shop scheduling can bring great benefits to the industry.However,there are few types of research on Distributed Hybrid Flow Shop Problems(DHFSP)by learning assisted meta-heuristics.This work addresses a DHFSP with minimizing the maximum completion time(Makespan).First,a mathematical model is developed for the concerned DHFSP.Second,four Q-learning-assisted meta-heuristics,e.g.,genetic algorithm(GA),artificial bee colony algorithm(ABC),particle swarm optimization(PSO),and differential evolution(DE),are proposed.According to the nature of DHFSP,six local search operations are designed for finding high-quality solutions in local space.Instead of randomselection,Q-learning assists meta-heuristics in choosing the appropriate local search operations during iterations.Finally,based on 60 cases,comprehensive numerical experiments are conducted to assess the effectiveness of the proposed algorithms.The experimental results and discussions prove that using Q-learning to select appropriate local search operations is more effective than the random strategy.To verify the competitiveness of the Q-learning assistedmeta-heuristics,they are compared with the improved iterated greedy algorithm(IIG),which is also for solving DHFSP.The Friedman test is executed on the results by five algorithms.It is concluded that the performance of four Q-learning-assisted meta-heuristics are better than IIG,and the Q-learning-assisted PSO shows the best competitiveness.

Energy-Saving Distributed Flexible Job Shop Scheduling Optimization with Dual Resource Constraints Based on Integrated Q-Learning Multi-Objective Grey Wolf Optimizer

The distributed flexible job shop scheduling problem(DFJSP)has attracted great attention with the growth of the global manufacturing industry.General DFJSP research only considers machine constraints and ignores worker constraints.As one critical factor of production,effective utilization of worker resources can increase productivity.Meanwhile,energy consumption is a growing concern due to the increasingly serious environmental issues.Therefore,the distributed flexible job shop scheduling problem with dual resource constraints(DFJSP-DRC)for minimizing makespan and total energy consumption is studied in this paper.To solve the problem,we present a multi-objective mathematical model for DFJSP-DRC and propose a Q-learning-based multi-objective grey wolf optimizer(Q-MOGWO).In Q-MOGWO,high-quality initial solutions are generated by a hybrid initialization strategy,and an improved active decoding strategy is designed to obtain the scheduling schemes.To further enhance the local search capability and expand the solution space,two wolf predation strategies and three critical factory neighborhood structures based on Q-learning are proposed.These strategies and structures enable Q-MOGWO to explore the solution space more efficiently and thus find better Pareto solutions.The effectiveness of Q-MOGWO in addressing DFJSP-DRC is verified through comparison with four algorithms using 45 instances.The results reveal that Q-MOGWO outperforms comparison algorithms in terms of solution quality.

An Elite-Class Teaching-Learning-Based Optimization for Reentrant Hybrid Flow Shop Scheduling with Bottleneck Stage

Bottleneck stage and reentrance often exist in real-life manufacturing processes;however,the previous research rarely addresses these two processing conditions in a scheduling problem.In this study,a reentrant hybrid flow shop scheduling problem(RHFSP)with a bottleneck stage is considered,and an elite-class teaching-learning-based optimization(ETLBO)algorithm is proposed to minimize maximum completion time.To produce high-quality solutions,teachers are divided into formal ones and substitute ones,and multiple classes are formed.The teacher phase is composed of teacher competition and teacher teaching.The learner phase is replaced with a reinforcement search of the elite class.Adaptive adjustment on teachers and classes is established based on class quality,which is determined by the number of elite solutions in class.Numerous experimental results demonstrate the effectiveness of new strategies,and ETLBO has a significant advantage in solving the considered RHFSP.

Strengthened Dominance Relation NSGA-Ⅲ Algorithm Based on Differential Evolution to Solve Job Shop Scheduling Problem

The job shop scheduling problem is a classical combinatorial optimization challenge frequently encountered in manufacturing systems.It involves determining the optimal execution sequences for a set of jobs on various machines to maximize production efficiency and meet multiple objectives.The Non-dominated Sorting Genetic Algorithm Ⅲ(NSGA-Ⅲ)is an effective approach for solving the multi-objective job shop scheduling problem.Nevertheless,it has some limitations in solving scheduling problems,including inadequate global search capability,susceptibility to premature convergence,and challenges in balancing convergence and diversity.To enhance its performance,this paper introduces a strengthened dominance relation NSGA-Ⅲ algorithm based on differential evolution(NSGA-Ⅲ-SD).By incorporating constrained differential evolution and simulated binary crossover genetic operators,this algorithm effectively improves NSGA-Ⅲ’s global search capability while mitigating pre-mature convergence issues.Furthermore,it introduces a reinforced dominance relation to address the trade-off between convergence and diversity in NSGA-Ⅲ.Additionally,effective encoding and decoding methods for discrete job shop scheduling are proposed,which can improve the overall performance of the algorithm without complex computation.To validate the algorithm’s effectiveness,NSGA-Ⅲ-SD is extensively compared with other advanced multi-objective optimization algorithms using 20 job shop scheduling test instances.The experimental results demonstrate that NSGA-Ⅲ-SD achieves better solution quality and diversity,proving its effectiveness in solving the multi-objective job shop scheduling problem.

An Improved Harris Hawk Optimization Algorithm for Flexible Job Shop Scheduling Problem

Flexible job shop scheduling problem(FJSP)is the core decision-making problem of intelligent manufacturing production management.The Harris hawk optimization(HHO)algorithm,as a typical metaheuristic algorithm,has been widely employed to solve scheduling problems.However,HHO suffers from premature convergence when solving NP-hard problems.Therefore,this paper proposes an improved HHO algorithm(GNHHO)to solve the FJSP.GNHHO introduces an elitism strategy,a chaotic mechanism,a nonlinear escaping energy update strategy,and a Gaussian random walk strategy to prevent premature convergence.A flexible job shop scheduling model is constructed,and the static and dynamic FJSP is investigated to minimize the makespan.This paper chooses a two-segment encoding mode based on the job and the machine of the FJSP.To verify the effectiveness of GNHHO,this study tests it in 23 benchmark functions,10 standard job shop scheduling problems(JSPs),and 5 standard FJSPs.Besides,this study collects data from an agricultural company and uses the GNHHO algorithm to optimize the company’s FJSP.The optimized scheduling scheme demonstrates significant improvements in makespan,with an advancement of 28.16%for static scheduling and 35.63%for dynamic scheduling.Moreover,it achieves an average increase of 21.50%in the on-time order delivery rate.The results demonstrate that the performance of the GNHHO algorithm in solving FJSP is superior to some existing algorithms.

Research on Flexible Job Shop Scheduling Optimization Based on Segmented AGV

As a typical transportation tool in the intelligent manufacturing system,Automatic Guided Vehicle(AGV)plays an indispensable role in the automatic production process of the workshop.Therefore,integrating AGV resources into production scheduling has become a research hotspot.For the scheduling problem of the flexible job shop adopting segmented AGV,a dual-resource scheduling optimization mathematical model of machine tools and AGVs is established by minimizing the maximum completion time as the objective function,and an improved genetic algorithmis designed to solve the problem in this study.The algorithmdesigns a two-layer codingmethod based on process coding and machine tool coding and embeds the task allocation of AGV into the decoding process to realize the real dual resource integrated scheduling.When initializing the population,three strategies are designed to ensure the diversity of the population.In order to improve the local search ability and the quality of the solution of the genetic algorithm,three neighborhood structures are designed for variable neighborhood search.The superiority of the improved genetic algorithmand the influence of the location and number of transfer stations on scheduling results are verified in two cases.

A Novel Collaborative Evolutionary Algorithm with Two-Population for Multi-Objective Flexible Job Shop Scheduling

Job shop scheduling(JS)is an important technology for modern manufacturing.Flexible job shop scheduling(FJS)is critical in JS,and it has been widely employed in many industries,including aerospace and energy.FJS enables any machine from a certain set to handle an operation,and this is an NP-hard problem.Furthermore,due to the requirements in real-world cases,multi-objective FJS is increasingly widespread,thus increasing the challenge of solving the FJS problems.As a result,it is necessary to develop a novel method to address this challenge.To achieve this goal,a novel collaborative evolutionary algorithmwith two-population based on Pareto optimality is proposed for FJS,which improves the solutions of FJS by interacting in each generation.In addition,several experimental results have demonstrated that the proposed method is promising and effective for multi-objective FJS,which has discovered some new Pareto solutions in the well-known benchmark problems,and some solutions can dominate the solutions of some other methods.

具有模糊加工时间的Flexible Job-Shop Scheduling问题的研究

采用梯形模糊数来表征柔性生产系统中的时间参数,并在此基础上对具有模糊加工时间的柔性作业车间最小化制造跨度调度问题进行了描述。然后给出了基于粒子群优化的柔性作业车间调度模型。最后通过实例验证了模型的有效性。

An Effective Neighborhood Solution Clipping Method for Large-Scale Job Shop Scheduling Problem

The job shop scheduling problem(JSSP)is a classical combinatorial optimization problem that exists widely in diverse scenarios of manufacturing systems.It is a well-known NP-hard problem,when the number of jobs increases,the difficulty of solving the problem exponentially increases.Therefore,a major challenge is to increase the solving efficiency of current algorithms.Modifying the neighborhood structure of the solutions can effectively improve the local search ability and efficiency.In this paper,a genetic Tabu search algorithm with neighborhood clipping(GTS_NC)is proposed for solving JSSP.A neighborhood solution clipping method is developed and embedded into Tabu search to improve the efficiency of the local search by clipping the search actions of unimproved neighborhood solutions.Moreover,a feasible neighborhood solution determination method is put forward,which can accurately distinguish feasible neighborhood solutions from infeasible ones.Both of the methods are based on the domain knowledge of JSSP.The proposed algorithmis compared with several competitive algorithms on benchmark instances.The experimental results show that the proposed algorithm can achieve superior results compared to other competitive algorithms.According to the numerical results of the experiments,it is verified that the neighborhood solution clippingmethod can accurately identify the unimproved solutions and reduces the computational time by at least 28%.

Competitive and Cooperative-Based Strength Pareto Evolutionary Algorithm for Green Distributed Heterogeneous Flow Shop Scheduling

This work aims to resolve the distributed heterogeneous permutation flow shop scheduling problem(DHPFSP)with minimizing makespan and total energy consumption(TEC).To solve this NP-hard problem,this work proposed a competitive and cooperative-based strength Pareto evolutionary algorithm(CCSPEA)which contains the following features:1)An initialization based on three heuristic rules is developed to generate a population with great diversity and convergence.2)A comprehensive metric combining convergence and diversity metrics is used to better represent the heuristic information of a solution.3)A competitive selection is designed which divides the population into a winner and a loser swarms based on the comprehensive metric.4)A cooperative evolutionary schema is proposed for winner and loser swarms to accelerate the convergence of global search.5)Five local search strategies based on problem knowledge are designed to improve convergence.6)Aproblem-based energy-saving strategy is presented to reduce TEC.Finally,to evaluate the performance of CCSPEA,it is compared to four state-of-art and run on 22 instances based on the Taillard benchmark.The numerical experiment results demonstrate that 1)the proposed comprehensive metric can efficiently represent the heuristic information of each solution to help the later step divide the population.2)The global search based on the competitive and cooperative schema can accelerate loser solutions convergence and further improve the winner’s exploration.3)The problembased initialization,local search,and energy-saving strategies can efficiently reduce the makespan and TEC.4)The proposed CCSPEA is superior to the state-of-art for solving DHPFSP.

Applying Job Shop Scheduling to SMEs Manufacturing Platform to Revitalize B2B Relationship

A small and medium enterprises(SMEs)manufacturing platform aims to perform as a significant revenue to SMEs and vendors by providing scheduling and monitoring capabilities.The optimal job shop scheduling is generated by utilizing the scheduling system of the platform,and a minimum production time,i.e.,makespan decides whether the scheduling is optimal or not.This scheduling result allows manufacturers to achieve high productivity,energy savings,and customer satisfaction.Manufacturing in Industry 4.0 requires dynamic,uncertain,complex production environments,and customer-centered services.This paper proposes a novel method for solving the difficulties of the SMEs manufacturing by applying and implementing the job shop scheduling system on a SMEs manufacturing platform.The primary purpose of the SMEs manufacturing platform is to improve the B2B relationship between manufacturing companies and vendors.The platform also serves qualified and satisfactory production opportunities for buyers and producers by meeting two key factors:early delivery date and fulfillment of processing as many orders as possible.The genetic algorithm(GA)-based scheduling method results indicated that the proposed platform enables SME manufacturers to obtain optimized schedules by solving the job shop scheduling problem(JSSP)by comparing with the real-world data from a textile weaving factory in South Korea.The proposed platform will provide producers with an optimal production schedule,introduce new producers to buyers,and eventually foster relationships and mutual economic interests.

Modified bottleneck-based heuristic for large-scale job-shop scheduling problems with a single bottleneck

A modified bottleneck-based （MB） heuristic for large-scale job-shop scheduling problems with a welldefined bottleneck is suggested, which is simpler but more tailored than the shifting bottleneck （SB） procedure. In this algorithm, the bottleneck is first scheduled optimally while the non-bottleneck machines are subordinated around the solutions of the bottleneck schedule by some effective dispatching rules. Computational results indicate that the MB heuristic can achieve a better tradeoff between solution quality and computational time compared to SB procedure for medium-size problems. Furthermore, it can obtain a good solution in a short time for large-scale jobshop scheduling problems.

Project Scheduling问题和Job-Shop问题的神经网络解

Project Scheduling问题和Job-Shop问题是著名的NP难题。本文用神经网络方法去解这两个问题,软件模拟结果是令人满意的。这种方法也为解一大类组合优化问题提供了一个新的途径。

A Cross Entropy-Genetic Algorithm for m-Machines No-Wait Job-ShopScheduling Problem

No-wait job-shop scheduling (NWJSS) problem is one of the classical scheduling problems that exist on many kinds of industry with no-wait constraint, such as metal working, plastic, chemical, and food industries. Several methods have been proposed to solve this problem, both exact (i.e. integer programming) and metaheuristic methods. Cross entropy (CE), as a new metaheuristic, can be an alternative method to solve NWJSS problem. This method has been used in combinatorial optimization, as well as multi-external optimization and rare-event simulation. On these problems, CE implementation results an optimal value with less computational time in average. However, using original CE to solve large scale NWJSS requires high computational time. Considering this shortcoming, this paper proposed a hybrid of cross entropy with genetic algorithm (GA), called CEGA, on m-machines NWJSS. The results are compared with other metaheuritics: Genetic Algorithm-Simulated Annealing (GASA) and hybrid tabu search. The results showed that CEGA providing better or at least equal makespans in comparison with the other two methods.

Energy-efficient Approach to Minimizing the Energy Consumption in An Extended Job-shop Scheduling Problem

The traditional production planning and scheduling problems consider performance indicators like time, cost and quality as optimization objectives in manufacturing processes. However, environmentally-friendly factors like energy consumption of production have not been completely taken into consideration. Against this background, this paper addresses an approach to modify a given schedule generated by a production plarming and scheduling system in a job shop floor, where machine tools can work at different cutting speeds. It can adjust the cutting speeds of the operations while keeping the original assignment and processing sequence of operations of each job fixed in order to obtain energy savings. First, the proposed approach, based on a mixed integer programming mathematical model, changes the total idle time of the given schedule to minimize energy consumption in the job shop floor while accepting the optimal solution of the scheduling objective, makespan. Then, a genetic-simulated annealing algorithm is used to explore the optimal solution due to the fact that the problem is strongly NP-hard. Finally, the effectiveness of the approach is performed small- and large-size instances, respectively. The experimental results show that the approach can save 5%-10% of the average energy consumption while accepting the optimal solution of the makespan in small-size instances. In addition, the average maximum energy saving ratio can reach to 13%. And it can save approximately 1%-4% of the average energy consumption and approximately 2.4% of the average maximum energy while accepting the near-optimal solution of the makespan in large-size instances. The proposed research provides an interesting point to explore an energy-aware schedule optimization for a traditional production planning and scheduling problem.

A bi-objective model for job-shop scheduling problem to minimize both energy consumption and makespan

The issue of reducing energy consumption for the job-shop scheduling problem in machining systems is addressed, whose dual objectives are to minimize both the energy consumption and the makespan. First, the bi- objective model for the job-shop scheduling problem is proposed. The objective function value of the model represents synthesized optimization of energy consumption and makespan. Then, a heuristic algorithm is developed to locate the optimal or near optimal solutions of the model based on the Tabu search mechanism. Finally, the experimental case is presented to demonstrate the effectiveness of the proposed model and the algorithm.

APPLYING PARTICLE SWARM OPTIMIZATION TO JOB-SHOPSCHEDULING PROBLEM

A new heuristic algorithm is proposed for the problem of finding the minimummakespan in the job-shop scheduling problem. The new algorithm is based on the principles ofparticle swarm optimization (PSO). PSO employs a collaborative population-based search, which isinspired by the social behavior of bird flocking. It combines local search (by self experience) andglobal search (by neighboring experience), possessing high search efficiency. Simulated annealing(SA) employs certain probability to avoid becoming trapped in a local optimum and the search processcan be controlled by the cooling schedule. By reasonably combining these two different searchalgorithms, a general, fast and easily implemented hybrid optimization algorithm, named HPSO, isdeveloped. The effectiveness and efficiency of the proposed PSO-based algorithm are demonstrated byapplying it to some benchmark job-shop scheduling problems and comparing results with otheralgorithms in literature. Comparing results indicate that PSO-based algorithm is a viable andeffective approach for the job-shop scheduling problem.

Solving Job-Shop Scheduling Problem Based on Improved Adaptive Particle Swarm Optimization Algorithm

An improved adaptive particle swarm optimization(IAPSO)algorithm is presented for solving the minimum makespan problem of job shop scheduling problem(JSP).Inspired by hormone modulation mechanism,an adaptive hormonal factor(HF),composed of an adaptive local hormonal factor(H l)and an adaptive global hormonal factor(H g),is devised to strengthen the information connection between particles.Using HF,each particle of the swarm can adjust its position self-adaptively to avoid premature phenomena and reach better solution.The computational results validate the effectiveness and stability of the proposed IAPSO,which can not only find optimal or close-to-optimal solutions but also obtain both better and more stability results than the existing particle swarm optimization(PSO)algorithms.

Job-shop Scheduling with Multi-objectives Based on Genetic Algorithms

The technology of production planning and scheduling is one of the critical technologies that decide whether the automated manufacturing systems can get the expected economy. Job shop scheduling belongs to the special class of NP-hard problems. Most of the algorithms used to optimize this class of problems have an exponential time; that is, the computation time increases exponentially with problem size. In scheduling study, makespan is often considered as the main objective. In this paper, makespan, the due date request of the key jobs, the availability of the key machine, the average wait-time of the jobs, and the similarities between the jobs and so on are taken into account based on the application of mechanical engineering. The job shop scheduling problem with multi-objectives is analyzed and studied by using genetic algorithms based on the mechanics of genetics and natural selection. In this research, the tactics of the coding and decoding and the design of the genetic operators, along with the description of the mathematic model of the multi-objective functions, are presented. Finally an illu-strative example is given to testify the validity of this algorithm.