Q-Learning-Assisted Meta-Heuristics for Scheduling Distributed Hybrid Flow Shop Problems

The flow shop scheduling problem is important for the manufacturing industry.Effective flow shop scheduling can bring great benefits to the industry.However,there are few types of research on Distributed Hybrid Flow Shop Problems(DHFSP)by learning assisted meta-heuristics.This work addresses a DHFSP with minimizing the maximum completion time(Makespan).First,a mathematical model is developed for the concerned DHFSP.Second,four Q-learning-assisted meta-heuristics,e.g.,genetic algorithm(GA),artificial bee colony algorithm(ABC),particle swarm optimization(PSO),and differential evolution(DE),are proposed.According to the nature of DHFSP,six local search operations are designed for finding high-quality solutions in local space.Instead of randomselection,Q-learning assists meta-heuristics in choosing the appropriate local search operations during iterations.Finally,based on 60 cases,comprehensive numerical experiments are conducted to assess the effectiveness of the proposed algorithms.The experimental results and discussions prove that using Q-learning to select appropriate local search operations is more effective than the random strategy.To verify the competitiveness of the Q-learning assistedmeta-heuristics,they are compared with the improved iterated greedy algorithm(IIG),which is also for solving DHFSP.The Friedman test is executed on the results by five algorithms.It is concluded that the performance of four Q-learning-assisted meta-heuristics are better than IIG,and the Q-learning-assisted PSO shows the best competitiveness.

A Review on Swarm Intelligence and Evolutionary Algorithms for Solving Flexible Job Shop Scheduling Problems

Flexible job shop scheduling problems(FJSP)have received much attention from academia and industry for many years.Due to their exponential complexity,swarm intelligence(SI)and evolutionary algorithms(EA)are developed,employed and improved for solving them.More than 60%of the publications are related to SI and EA.This paper intents to give a comprehensive literature review of SI and EA for solving FJSP.First,the mathematical model of FJSP is presented and the constraints in applications are summarized.Then,the encoding and decoding strategies for connecting the problem and algorithms are reviewed.The strategies for initializing algorithms?population and local search operators for improving convergence performance are summarized.Next,one classical hybrid genetic algorithm(GA)and one newest imperialist competitive algorithm(ICA)with variables neighborhood search(VNS)for solving FJSP are presented.Finally,we summarize,discus and analyze the status of SI and EA for solving FJSP and give insight into future research directions.

Solving Multi-Area Environmental/Economic Dispatch by Pareto-Based Chemical-Reaction Optimization Algorithm

In this study, we present a Pareto-based chemicalreaction optimization(PCRO) algorithm for solving the multiarea environmental/economic dispatch optimization problems.Two objectives are minimized simultaneously, i.e., total fuel cost and emission. In the proposed algorithm, each solution is represented by a chemical molecule. A novel encoding mechanism for solving the multi-area environmental/economic dispatch optimization problems is designed to dynamically enhance the performance of the proposed algorithm. Then, an ensemble of effective neighborhood approaches is developed, and a selfadaptive neighborhood structure selection mechanism is also embedded in PCRO to increase the search ability while maintaining population diversity. In addition, a grid-based crowding distance strategy is introduced, which can obviously enable the algorithm to easily converge near the Pareto front. Furthermore,a kinetic-energy-based search procedure is developed to enhance the global search ability. Finally, the proposed algorithm is tested on sets of the instances that are generated based on realistic production. Through the analysis of experimental results, the highly effective performance of the proposed PCRO algorithm is favorably compared with several algorithms, with regards to both solution quality and diversity.

Decomposition Methods for Manufacturing System Scheduling:A Survey

Manufacturing is the application of labor, tools,machines, chemical and biological processing, to an original raw material by changing its physical and geometrical characteristics, in order to make finished products. Since the first industrial revolution, to accommodate the large-scale production,tremendous changes have happened to manufacturing through the innovations of technology, organization, management, transportation and communication. This work first reviews the highvolume low-mix process by focusing on the quantity production,transfer line and single model assembly line. Then, it reviews the high-volume high-mix process. For such a process type,mixed/multi model assembly line is usually adopted. Hence,two main decisions on them, i.e., balancing and, sequencing are reviewed. Thereafter, it discusses the low-volume high-mix process in detail. Then, technology gap and future work is discussed, and at last, conclusions are given.

Position Encoding Based Convolutional Neural Networks for Machine Remaining Useful Life Prediction

Accurate remaining useful life(RUL)prediction is important in industrial systems.It prevents machines from working under failure conditions,and ensures that the industrial system works reliably and efficiently.Recently,many deep learning based methods have been proposed to predict RUL.Among these methods,recurrent neural network(RNN)based approaches show a strong capability of capturing sequential information.This allows RNN based methods to perform better than convolutional neural network(CNN)based approaches on the RUL prediction task.In this paper,we question this common paradigm and argue that existing CNN based approaches are not designed according to the classic principles of CNN,which reduces their performances.Additionally,the capacity of capturing sequential information is highly affected by the receptive field of CNN,which is neglected by existing CNN based methods.To solve these problems,we propose a series of new CNNs,which show competitive results to RNN based methods.Compared with RNN,CNN processes the input signals in parallel so that the temporal sequence is not easily determined.To alleviate this issue,a position encoding scheme is developed to enhance the sequential information encoded by a CNN.Hence,our proposed position encoding based CNN called PE-Net is further improved and even performs better than RNN based methods.Extensive experiments are conducted on the C-MAPSS dataset,where our PE-Net shows state-of-the-art performance.

An embedded vertical‐federated feature selection algorithm based on particle swarm optimisation

In real life,a large amount of data describing the same learning task may be stored in different institutions(called participants),and these data cannot be shared among par-ticipants due to privacy protection.The case that different attributes/features of the same instance are stored in different institutions is called vertically distributed data.The pur-pose of vertical‐federated feature selection(FS)is to reduce the feature dimension of vertical distributed data jointly without sharing local original data so that the feature subset obtained has the same or better performance as the original feature set.To solve this problem,in the paper,an embedded vertical‐federated FS algorithm based on particle swarm optimisation(PSO‐EVFFS)is proposed by incorporating evolutionary FS into the SecureBoost framework for the first time.By optimising both hyper‐parameters of the XGBoost model and feature subsets,PSO‐EVFFS can obtain a feature subset,which makes the XGBoost model more accurate.At the same time,since different participants only share insensitive parameters such as model loss function,PSO‐EVFFS can effec-tively ensure the privacy of participants'data.Moreover,an ensemble ranking strategy of feature importance based on the XGBoost tree model is developed to effectively remove irrelevant features on each participant.Finally,the proposed algorithm is applied to 10 test datasets and compared with three typical vertical‐federated learning frameworks and two variants of the proposed algorithm with different initialisation strategies.Experi-mental results show that the proposed algorithm can significantly improve the classifi-cation performance of selected feature subsets while fully protecting the data privacy of all participants.

A Q-Learning Based Hybrid Meta-Heuristic for Integrated Scheduling of Disassembly and Reprocessing Processes Considering Product Structures and Stochasticity

Remanufacturing is regarded as a sustainable manufacturing paradigm of energy conservation and environment protection.To improve the efficiency of the remanufacturing process,this work investigates an integrated scheduling problem for disassembly and reprocessing in a remanufacturing process,where product structures and uncertainty are taken into account.First,a stochastic programming model is developed to minimize the maximum completion time(makespan).Second,a Q-learning based hybrid meta-heuristic(Q-HMH)is specially devised.In each iteration,a Q-learning method is employed to adaptively choose a premium algorithm from four candidate ones,including genetic algorithm(GA),artificial bee colony(ABC),shuffled frog-leaping algorithm(SFLA),and simulated annealing(SA)methods.At last,simulation experiments are carried out by using sixteen instances with different scales,and three state-of-the-art algorithms in literature and an exact solver CPLEX are chosen for comparisons.By analyzing the results with the average relative percentage deviation(RPD)metric,we find that Q-HMH outperforms its rivals by 9.79%-26.76%.The results and comparisons verify the excellent competitiveness of Q-HMH for solving the concerned problems.

A Penalty Groups-Assisted Iterated Greedy Integrating Idle Time Insertion:Solving the Hybrid Flow Shop Group Scheduling with Delivery Time Windows

The hybrid flow shop group scheduling problem(HFGSP)with the delivery time windows has been widely studied owing to its better flexibility and suitability for the current just-in-time production mode.However,there are several unresolved challenges in problem modeling and algorithmic design tailored for HFGSP.In our study,we place emphasis on the constraint of timeliness.Therefore,this paper first constructs a mixed integer linear programming model of HFGSP with sequence-dependent setup time and delivery time windows to minimize the total weighted earliness and tardiness(TWET).Then a penalty groups-assisted iterated greedy integrating idle time insertion(PG IG ITI)is proposed to solve the above problem.In the PG IG ITI,a double decoding strategy is proposed based on the earliest available machine rule and the idle time insertion rule to calculate the TWET value.Subsequently,to reduce the amount of computation,a skip-based destruction and reconstruction strategy is designed,and a penalty groups-assisted local search is proposed to further improve the quality of the solution by disturbing the penalized groups,i.e.,early and tardy groups.Finally,through comprehensive statistical experiments on 270 test instances,the results prove that the proposed algorithm is effective compared to four state-of-the-art algorithms.

Distributed Scheduling Problems in Intelligent Manufacturing Systems

Currently,manufacturing enterprises face increasingly fierce market competition due to the various demands of customers and the rapid development of economic globalization.Hence,they have to extend their production mode into distributed environments and establish multiple factories in various geographical locations.Nowadays,distributed manufacturing systems have been widely adopted in industrial production processes.In recent years,many studies have been done on the modeling and optimization of distributed scheduling problems.This work provides a literature review on distributed scheduling problems in intelligent manufacturing systems.By summarizing and evaluating existing studies on distributed scheduling problems,we analyze the achievements and current research status in this field and discuss ongoing studies.Insights regarding prior works are discussed to uncover future research directions,particularly swarm intelligence and evolutionary algorithms,which are used for managing distributed scheduling problems in manufacturing systems.This work focuses on journal papers discovered using Google Scholar.After reviewing the papers,in this work,we discuss the research trends of distributed scheduling problems and point out some directions for future studies.

Intelligent Optimization Under Multiple Factories: Hybrid Flow Shop Scheduling Problem with Blocking Constraints Using an Advanced Iterated Greedy Algorithm

The distributed hybrid flow shop scheduling problem(DHFSP),which integrates distributed manufacturing models with parallel machines,has gained significant attention.However,in actual scheduling,some adjacent machines do not have buffers between them,resulting in blocking.This paper focuses on addressing the DHFSP with blocking constraints(DBHFSP)based on the actual production conditions.To solve DBHFSP,we construct a mixed integer linear programming(MILP)model for DBHFSP and validate its correctness using the Gurobi solver.Then,an advanced iterated greedy(AIG)algorithm is designed to minimize the makespan,in which we modify the Nawaz,Enscore,and Ham(NEH)heuristic to solve blocking constraints.To balance the global and local search capabilities of AIG,two effective inter-factory neighborhood search strategies and a swap-based local search strategy are designed.Additionally,each factory is mutually independent,and the movement within one factory does not affect the others.In view of this,we specifically designed a memory-based decoding method for insertion operations to reduce the computation time of the objective.Finally,two shaking strategies are incorporated into the algorithm to mitigate premature convergence.Five advanced algorithms are used to conduct comparative experiments with AIG on 80 test instances,and experimental results illustrate that the makespan and the relative percentage increase(RPI)obtained by AIG are 1.0%and 86.1%,respectively,better than the comparative algorithms.

A Multi-Objective Scheduling and Routing Problem for Home Health Care Services via Brain Storm Optimization

At present,home health care(HHC)has been accepted as an effective method for handling the healthcare problems of the elderly.The HHC scheduling and routing problem(HHCSRP)attracts wide concentration from academia and industrial communities.This work proposes an HHCSRP considering several care centers,where a group of customers(i.e.,patients and the elderly)require being assigned to care centers.Then,various kinds of services are provided by caregivers for customers in different regions.By considering the skill matching,customers’appointment time,and caregivers’workload balancing,this article formulates an optimization model with multiple objectives to achieve minimal service cost and minimal delay cost.To handle it,we then introduce a brain storm optimization method with particular multi-objective search mechanisms(MOBSO)via combining with the features of the investigated HHCSRP.Moreover,we perform experiments to test the effectiveness of the designed method.Via comparing the MOBSO with two excellent optimizers,the results confirm that the developed method has significant superiority in addressing the considered HHCSRP.