Q-Learning-Assisted Meta-Heuristics for Scheduling Distributed Hybrid Flow Shop Problems

The flow shop scheduling problem is important for the manufacturing industry.Effective flow shop scheduling can bring great benefits to the industry.However,there are few types of research on Distributed Hybrid Flow Shop Problems(DHFSP)by learning assisted meta-heuristics.This work addresses a DHFSP with minimizing the maximum completion time(Makespan).First,a mathematical model is developed for the concerned DHFSP.Second,four Q-learning-assisted meta-heuristics,e.g.,genetic algorithm(GA),artificial bee colony algorithm(ABC),particle swarm optimization(PSO),and differential evolution(DE),are proposed.According to the nature of DHFSP,six local search operations are designed for finding high-quality solutions in local space.Instead of randomselection,Q-learning assists meta-heuristics in choosing the appropriate local search operations during iterations.Finally,based on 60 cases,comprehensive numerical experiments are conducted to assess the effectiveness of the proposed algorithms.The experimental results and discussions prove that using Q-learning to select appropriate local search operations is more effective than the random strategy.To verify the competitiveness of the Q-learning assistedmeta-heuristics,they are compared with the improved iterated greedy algorithm(IIG),which is also for solving DHFSP.The Friedman test is executed on the results by five algorithms.It is concluded that the performance of four Q-learning-assisted meta-heuristics are better than IIG,and the Q-learning-assisted PSO shows the best competitiveness.

Meta-heuristic算法研究进展

对模拟退火、遗传算法和禁忌搜索法等代表性 meta-heuristic算法在理论与应用方面的研究进行综述 ,探讨算法结构和研究体系上的统一性 ,并归纳指出其发展方向。

Predicting rock size distribution in mine blasting using various novel soft computing models based on meta-heuristics and machine learning algorithms

Blasting is well-known as an effective method for fragmenting or moving rock in open-pit mines.To evaluate the quality of blasting,the size of rock distribution is used as a critical criterion in blasting operations.A high percentage of oversized rocks generated by blasting operations can lead to economic and environmental damage.Therefore,this study proposed four novel intelligent models to predict the size of rock distribution in mine blasting in order to optimize blasting parameters,as well as the efficiency of blasting operation in open mines.Accordingly,a nature-inspired algorithm(i.e.,firefly algorithm-FFA)and different machine learning algorithms(i.e.,gradient boosting machine(GBM),support vector machine(SVM),Gaussian process(GP),and artificial neural network(ANN))were combined for this aim,abbreviated as FFA-GBM,FFA-SVM,FFA-GP,and FFA-ANN,respectively.Subsequently,predicted results from the abovementioned models were compared with each other using three statistical indicators(e.g.,mean absolute error,root-mean-squared error,and correlation coefficient)and color intensity method.For developing and simulating the size of rock in blasting operations,136 blasting events with their images were collected and analyzed by the Split-Desktop software.In which,111 events were randomly selected for the development and optimization of the models.Subsequently,the remaining 25 blasting events were applied to confirm the accuracy of the proposed models.Herein,blast design parameters were regarded as input variables to predict the size of rock in blasting operations.Finally,the obtained results revealed that the FFA is a robust optimization algorithm for estimating rock fragmentation in bench blasting.Among the models developed in this study,FFA-GBM provided the highest accuracy in predicting the size of fragmented rocks.The other techniques(i.e.,FFA-SVM,FFA-GP,and FFA-ANN)yielded lower computational stability and efficiency.Hence,the FFA-GBM model can be used as a powerful and precise soft computing tool that can be applied to practical engineering cases aiming to improve the quality of blasting and rock fragmentation.

Damage Identification of A TLP Floating Wind Turbine by Meta-Heuristic Algorithms

Damage identification of the offshore floating wind turbine by vibration/dynamic signals is one of the important and new research fields in the Structural Health Monitoring（SHM）. In this paper a new damage identification method is proposed based on meta-heuristic algorithms using the dynamic response of the TLP（Tension-Leg Platform） floating wind turbine structure. The Genetic Algorithms（GA）, Artificial Immune System（AIS）, Particle Swarm Optimization（PSO）, and Artificial Bee Colony（ABC） are chosen for minimizing the object function, defined properly for damage identification purpose. In addition to studying the capability of mentioned algorithms in correctly identifying the damage, the effect of the response type on the results of identification is studied. Also, the results of proposed damage identification are investigated with considering possible uncertainties of the structure. Finally, for evaluating the proposed method in real condition, a 1/100 scaled experimental setup of TLP Floating Wind Turbine（TLPFWT） is provided in a laboratory scale and the proposed damage identification method is applied to the scaled turbine.

QoS-Aware Energy-Efficient Task Scheduling on HPC Cloud Infrastructures Using Swarm-Intelligence Meta-Heuristics

Cloud computing infrastructure has been evolving as a cost-effective platform for providing computational resources in the form of high-performance computing as a service(HPCaaS)to users for executing HPC applications.However,the broader use of the Cloud services,the rapid increase in the size,and the capacity of Cloud data centers bring a remarkable rise in energy consumption leading to a significant rise in the system provider expenses and carbon emissions in the environment.Besides this,users have become more demanding in terms of Quality-of-service(QoS)expectations in terms of execution time,budget cost,utilization,and makespan.This situation calls for the design of task scheduling policy,which ensures efficient task sequencing and allocation of computing resources to tasks to meet the trade-off between QoS promises and service provider requirements.Moreover,the task scheduling in the Cloud is a prevalent NP-Hard problem.Motivated by these concerns,this paper introduces and implements a QoS-aware Energy-Efficient Scheduling policy called as CSPSO,for scheduling tasks in Cloud systems to reduce the energy consumption of cloud resources and minimize the makespan of workload.The proposed multi-objective CSPSO policy hybridizes the search qualities of two robust metaheuristics viz.cuckoo search(CS)and particle swarm optimization(PSO)to overcome the slow convergence and lack of diversity of standard CS algorithm.A fitness-aware resource allocation(FARA)heuristic was developed and used by the proposed policy to allocate resources to tasks efficiently.A velocity update mechanism for cuckoo individuals is designed and incorporated in the proposed CSPSO policy.Further,the proposed scheduling policy has been implemented in the CloudSim simulator and tested with real supercomputing workload traces.The comparative analysis validated that the proposed scheduling policy can produce efficient schedules with better performance over other well-known heuristics and meta-heuristics scheduling policies.

Meta-Heuristic算法在二维图形优化排样中的应用

二维排样优化问题是指如何在一个任意形状的母板上排满任意形状的样件使母板的利用率达到最大,二维排样优化方法分为精确方法和近似方法两大类。在求解二维排样这类NP难问题时,Meta-Heuristic算法是较好的解决方案。基于此,笔者针对Meta-Heuristic算法在二维排样问题的应用,综述了多种Meta-Heuristic算法在二维排样优化领域的研究现状,分析了不同算法的性能和适应场景,并总结了发展趋势。

Training Neuro-Fuzzy by Using Meta-Heuristic Algorithms for MPPT

It is one of the topics that have been studied extensively on maximum power point tracking(MPPT)recently.Traditional or soft computing methods are used for MPPT.Since soft computing approaches are more effective than traditional approaches,studies on MPPT have shifted in this direction.This study aims comparison of performance of seven meta-heuristic training algorithms in the neuro-fuzzy training for MPPT.The meta-heuristic training algorithms used are particle swarm optimization(PSO),harmony search(HS),cuckoo search(CS),artificial bee colony(ABC)algorithm,bee algorithm(BA),differential evolution(DE)and flower pollination algorithm(FPA).The antecedent and conclusion parameters of neuro-fuzzy are determined by these algorithms.The data of a 250 W photovoltaic(PV)is used in the applications.For effective MPPT,different neuro-fuzzy structures,different membership functions and different control parameter values are evaluated in detail.Related training algorithms are compared in terms of solution quality and convergence speed.The strengths and weaknesses of these algorithms are revealed.It is seen that the type and number of membership function,colony size,number of generations affect the solution quality and convergence speed of the training algorithms.As a result,it has been observed that CS and ABC algorithm are more effective than other algorithms in terms of solution quality and convergence in solving the related problem.

Meta-heuristics for Feature Selection and Classification in Diagnostic Breast Cancer

One of the most common kinds of cancer is breast cancer.The early detection of it may help lower its overall rates of mortality.In this paper,we robustly propose a novel approach for detecting and classifying breast cancer regions in thermal images.The proposed approach starts with data preprocessing the input images and segmenting the significant regions of interest.In addition,to properly train the machine learning models,data augmentation is applied to increase the number of segmented regions using various scaling ratios.On the other hand,to extract the relevant features from the breast cancer cases,a set of deep neural networks(VGGNet,ResNet-50,AlexNet,and GoogLeNet)are employed.The resulting set of features is processed using the binary dipper throated algorithm to select the most effective features that can realize high classification accuracy.The selected features are used to train a neural network to finally classify the thermal images of breast cancer.To achieve accurate classification,the parameters of the employed neural network are optimized using the continuous dipper throated optimization algorithm.Experimental results show the effectiveness of the proposed approach in classifying the breast cancer cases when compared to other recent approaches in the literature.Moreover,several experiments were conducted to compare the performance of the proposed approach with the other approaches.The results of these experiments emphasized the superiority of the proposed approach.

Meta-Heuristic Optimized Hybrid Wavelet Features for Arrhythmia Classification

The non-invasive evaluation of the heart through EectroCardioG-raphy(ECG)has played a key role in detecting heart disease.The analysis of ECG signals requires years of learning and experience to interpret and extract useful information from them.Thus,a computerized system is needed to classify ECG signals with more accurate results effectively.Abnormal heart rhythms are called arrhythmias and cause sudden cardiac deaths.In this work,a Computerized Abnormal Heart Rhythms Detection(CAHRD)system is developed using ECG signals.It consists of four stages;preprocessing,feature extraction,feature optimization and classifier.At first,Pan and Tompkins algorithm is employed to detect the envelope of Q,R and S waves in the preprocessing stage.It uses a recursive filter to eliminate muscle noise,T-wave interference and baseline wander.As the analysis of ECG signal in the spatial domain does not provide a complete description of the signal,the feature extraction involves using frequency contents obtained from multiple wavelet filters;bi-orthogonal,Symlet and Daubechies at different resolution levels in the feature extraction stage.Then,Black Widow Optimization(BWO)is applied to optimize the hybrid wavelet features in the feature optimization stage.Finally,a kernel based Support Vector Machine(SVM)is employed to classify heartbeats into five classes.In SVM,Radial Basis Function(RBF),polynomial and linear kernels are used.A total of∼15000 ECG signals are obtained from the Massachusetts Institute of Technology-Beth Israel Hospital(MIT-BIH)arrhythmia database for performance evaluation of the proposed CAHRD system.Results show that the proposed CAHRD system proved to be a powerful tool for ECG analysis.It correctly classifies five classes of heartbeats with 99.91%accuracy using an RBF kernel with 2nd level wavelet coefficients.The CAHRD system achieves an improvement of∼6%over random projections with the ensemble SVM approach and∼2%over morphological and ECG segment based features with the RBF classifier.

Modeling the Scheduling Problem in Cellular Manufacturing Systems Using Genetic Algorithm as an Efficient Meta-Heuristic Approach

This paper presents a new,bi-criteria mixed_integer programming model for scheduling cells and pieces within each cell in a manufacturing cellular system.The objective of this model is to minimize the makespan and intercell movements simultaneously,while considering sequence-dependent cell setup times.In the cellular manufacturing systems design and planning,three main steps must be considered,namely cell formation(i.e,piece families and machine grouping),inter and intra-cell layouts,and scheduling issue.Due to the fact that the cellular manufacturing systems problem is NP-Hard,a genetic algorithm as an efficient meta-heuristic method is proposed to solve such a hard problem.Finally,a number of test problems are solved to show the efficiency of the proposed genetic algorithm and the related computational results are compared with the results obtained by the use of an optimization tool.

Trust Based Meta-Heuristics Workflow Scheduling in Cloud Service Environment

Cloud computing has emerged as a new style of computing in distributed environment. An efficient and dependable Workflow Scheduling is crucial for achieving high performance and incorporating with enterprise systems. As an effective security services aggregation methodology, Trust Work-flow Technology (TWT) has been used to construct composite services. However, in cloud environment, the existing closed network services are maintained and functioned by third-party organizations or enterprises. Therefore service-oriented trust strategies must be considered in workflow scheduling. TWFS related algorithms consist of trust policies and strategies to overcome the threats of the application with heuristic workflow scheduling. As a significance of this work, trust based Meta heuristic workflow scheduling (TMWS) is proposed. The TMWS algorithm will improve the efficiency and reliability of the operation in the cloud system and the results show that the TMWS approach is effective and feasible.

A Q-Learning Based Hybrid Meta-Heuristic for Integrated Scheduling of Disassembly and Reprocessing Processes Considering Product Structures and Stochasticity

Remanufacturing is regarded as a sustainable manufacturing paradigm of energy conservation and environment protection.To improve the efficiency of the remanufacturing process,this work investigates an integrated scheduling problem for disassembly and reprocessing in a remanufacturing process,where product structures and uncertainty are taken into account.First,a stochastic programming model is developed to minimize the maximum completion time(makespan).Second,a Q-learning based hybrid meta-heuristic(Q-HMH)is specially devised.In each iteration,a Q-learning method is employed to adaptively choose a premium algorithm from four candidate ones,including genetic algorithm(GA),artificial bee colony(ABC),shuffled frog-leaping algorithm(SFLA),and simulated annealing(SA)methods.At last,simulation experiments are carried out by using sixteen instances with different scales,and three state-of-the-art algorithms in literature and an exact solver CPLEX are chosen for comparisons.By analyzing the results with the average relative percentage deviation(RPD)metric,we find that Q-HMH outperforms its rivals by 9.79%-26.76%.The results and comparisons verify the excellent competitiveness of Q-HMH for solving the concerned problems.

Domain Knowledge Used in Meta-Heuristic Algorithms for the Job-Shop Scheduling Problem:Review and Analysis

Meta-heuristic algorithms search the problem solution space to obtain a satisfactory solution within a reasonable timeframe.By combining domain knowledge of the specific optimization problem,the search efficiency and quality of meta-heuristic algorithms can be significantly improved,making it crucial to identify and summarize domain knowledge within the problem.In this paper,we summarize and analyze domain knowledge that can be applied to meta-heuristic algorithms in the job-shop scheduling problem(JSP).Firstly,this paper delves into the importance of domain knowledge in optimization algorithm design.After that,the development of different methods for the JSP are reviewed,and the domain knowledge in it for meta-heuristic algorithms is summarized and classified.Applications of this domain knowledge are analyzed,showing it is indispensable in ensuring the optimization performance of meta-heuristic algorithms.Finally,this paper analyzes the relationship among domain knowledge,optimization problems,and optimization algorithms,and points out the shortcomings of the existing research and puts forward research prospects.This paper comprehensively summarizes the domain knowledge in the JSP,and discusses the relationship between the optimization problems,optimization algorithms and domain knowledge,which provides a research direction for the metaheuristic algorithm design for solving the JSP in the future.

The Bedbug Meta-heuristic Algorithm to Solve Optimization Problems

Small parasitic Hemipteran insects known as bedbugs(Cimicidae)feed on warm-blooded mammal’s blood.The most famous member of this family is the Cimex lectularius or common bedbug.The current paper proposes a novel swarm intelligence optimization algorithm called the Bedbug Meta-Heuristic Algorithm(BMHA).The primary inspiration for the bedbug algorithm comes from the static and dynamic swarming behaviors of bedbugs in nature.The two main stages of optimization algorithms,exploration,and exploitation,are designed by modeling bedbug social interaction to search for food.The proposed algorithm is benchmarked qualitatively and quantitatively using many test functions including CEC2019.The results of evaluating BMHA prove that this algorithm can improve the initial random population for a given optimization problem to converge towards global optimization and provide highly competitive results compared to other well-known optimization algorithms.The results also prove the new algorithm's performance in solving real optimization problems in unknown search spaces.To achieve this,the proposed algorithm has been used to select the features of fake news in a semi-supervised manner,the results of which show the good performance of the proposed algorithm in solving problems.

Consensus-based dispatch optimization of a microgrid considering meta-heuristic-based demand response scheduling and network packet loss characterization

The uncertainty inherent in power load forecasts represents a major factor in the mismatches between supply and demand in renewables-rich electricity networks, which consequently increases the energy bills and curtailed generation. As the transition to a power grid founded on the so-called grid-of-grids becomes more evident, the need for distributed control algorithms capable of handling computationally challenging problems in the energy sector does so as well. In this light, the consensus-based distributed algorithm has recently been shown to provide an effective platform for solving the complex energy management problem in microgrids. More specifically, in a microgrid context, the consensus-based distributed algorithm requires reliable information exchange with customers to achieve convergence. However, packet losses remain an important issue, which can potentially result in the failure of the overall system. In this setting, this paper introduces a novel method to effectively characterize such packet losses during information exchange between the customers and the microgrid operator, whilst solving the microgrid scheduling optimization problem for a multi-agent-based microgrid. More specifically, the proposed framework leverages the virulence optimization algorithm and the earth-worm optimization algorithm to optimally shift the energy consumption during peak periods to lower-priced off-peak hours. The effectiveness of the proposed method in minimizing the overall active power mismatches in the presence of packet losses has also been demonstrated based on benchmarking the results against the business-as-usual iterative scheduling algorithm. Also, the robustness of the overall meta-heuristic- and multi-agent-based method in producing optimal results is confirmed based on comparing the results obtained by several well-established meta-heuristic optimization algorithms, including the binary particle swarm optimization, the genetic algorithm, and the cuckoo search optimization.

Meta-heuristic optimization inspired by proton-electron swarm

While solving unimodal function problems,conventional meta-heuristic algorithms often suffer from low accuracy and slow convergence.Therefore,in this paper,a novel meta-heuristic optimization algorithm,named proton-electron swarm(PES),is proposed based on physical rules.This algorithm simulates the physical phenomena of like-charges repelling each other while opposite charges attracting in protons and electrons,and establishes a mathematical model to realize the optimization process.By balancing the global exploration and local exploitation ability,this algorithm achieves high accuracy and avoids falling into local optimum when solving target problem.In order to evaluate the effectiveness of this algorithm,23 classical benchmark functions were selected for comparative experiments.Experimental results show that,compared with the contrast algorithms,the proposed algorithm cannot only obtain higher accuracy and convergence speed in solving unimodal function problems,but also maintain strong optimization ability in solving multimodal function problems.

A comparative study on using meta-heuristic algorithms for road maintenance planning:Insights from field study in a developing country

Optimized road maintenance planning seeks for solutions that can minimize the life-cycle cost of a road network and concurrently maximize pavement condition. Aiming at pro- posing an optimal set of road maintenance solutions, robust meta-heuristic algorithms are used in research. Two main optimization techniques are applied including single-objective and multi-objective optimization. Genetic algorithms （GA）, particle swarm optimization （PSO）, and combination of genetic algorithm and particle swarm optimization （GAPSO） as single-objective techniques are used, while the non-domination sorting genetic algorithm II （NSGAII） and multi-objective particle swarm optimization （MOPSO） which are sufficient for solving computationally complex large-size optimization problems as multi-objective techniques are applied and compared. A real case study from the rural transportation network of Iran is employed to illustrate the sufficiency of the optimum algorithm. The formulation of the optimization model is carried out in such a way that a cost-effective maintenance strategy is reached by preserving the performance level of the road network at a desirable level. So, the objective functions are pavement performance maximization and maintenance cost minimization. It is concluded that multi-objective algorithms including non-domination sorting genetic algorithm II （NSGAII） and multi-objective particle swarm optimization performed better than the single objective algorithms due to the capability to balance between both objectives. And between multi-objective algorithms the NSGAII provides the optimum solution for the road maintenance planning.

An Improved Lung Cancer Segmentation Based on Nature-Inspired Optimization Approaches

The distinction and precise identification of tumor nodules are crucial for timely lung cancer diagnosis andplanning intervention. This research work addresses the major issues pertaining to the field of medical imageprocessing while focusing on lung cancer Computed Tomography (CT) images. In this context, the paper proposesan improved lung cancer segmentation technique based on the strengths of nature-inspired approaches. Thebetter resolution of CT is exploited to distinguish healthy subjects from those who have lung cancer. In thisprocess, the visual challenges of the K-means are addressed with the integration of four nature-inspired swarmintelligent techniques. The techniques experimented in this paper are K-means with Artificial Bee Colony (ABC),K-means with Cuckoo Search Algorithm (CSA), K-means with Particle Swarm Optimization (PSO), and Kmeanswith Firefly Algorithm (FFA). The testing and evaluation are performed on Early Lung Cancer ActionProgram (ELCAP) database. The simulation analysis is performed using lung cancer images set against metrics:precision, sensitivity, specificity, f-measure, accuracy,Matthews Correlation Coefficient (MCC), Jaccard, and Dice.The detailed evaluation shows that the K-means with Cuckoo Search Algorithm (CSA) significantly improved thequality of lung cancer segmentation in comparison to the other optimization approaches utilized for lung cancerimages. The results exhibit that the proposed approach (K-means with CSA) achieves precision, sensitivity, and Fmeasureof 0.942, 0.964, and 0.953, respectively, and an average accuracy of 93%. The experimental results prove thatK-meanswithABC,K-meanswith PSO,K-meanswith FFA, andK-meanswithCSAhave achieved an improvementof 10.8%, 13.38%, 13.93%, and 15.7%, respectively, for accuracy measure in comparison to K-means segmentationfor lung cancer images. Further, it is highlighted that the proposed K-means with CSA have achieved a significantimprovement in accuracy, hence can be utilized by researchers for improved segmentation processes of medicalimage datasets for identifying the targeted region of interest.

Heuristic-Based Optimal Load Frequency Control with Offsite Backup Controllers in Interconnected Microgrids

The primary factor contributing to frequency instability in microgrids is the inherent intermittency of renewable energy sources.This paper introduces novel dual-backup controllers utilizing advanced fractional order proportional integral derivative(FOPID)controllers to enhance frequency and tie-line power stability in microgrids amid increasing renewable energy integration.To improve load frequency control,the proposed controllers are applied to a two-area interconnectedmicrogrid system incorporating diverse energy sources,such as wind turbines,photovoltaic cells,diesel generators,and various storage technologies.A novelmeta-heuristic algorithm is adopted to select the optimal parameters of the proposed controllers.The efficacy of the advanced FOPID controllers is demonstrated through comparative analyses against traditional proportional integral derivative(PID)and FOPID controllers,showcasing superior performance inmanaging systemfluctuations.The optimization algorithm is also evaluated against other artificial intelligent methods for parameter optimization,affirming the proposed solution’s efficiency.The robustness of the intelligent controllers against system uncertainties is further validated under extensive power disturbances,proving their capability to maintain grid stability.The dual-controller configuration ensures redundancy,allowing them to operate as mutual backups,enhancing system reliability.This research underlines the importance of sophisticated control strategies for future-proofing microgrid operations against the backdrop of evolving energy landscapes.

A Critical Review of Active Distribution Network Reconfiguration:Concepts,Development,and Perspectives

In recent years,the large-scale grid connection of various distributed power sources has made the planning and operation of distribution grids increasingly complex.Consequently,a large number of active distribution network reconfiguration techniques have emerged to reduce system losses,improve system safety,and enhance power quality via switching switches to change the system topology while ensuring the radial structure of the network.While scholars have previously reviewed these methods,they all have obvious shortcomings,such as a lack of systematic integration of methods,vague classification,lack of constructive suggestions for future study,etc.Therefore,this paper attempts to provide a comprehensive and profound review of 52 methods and applications of active distribution network reconfiguration through systematic method classification and enumeration.Specifically,these methods are classified into five categories,i.e.,traditional methods,mathematical methods,meta-heuristic algorithms,machine learning methods,and hybrid methods.A thorough comparison of the various methods is also scored in terms of their practicality,complexity,number of switching actions,performance improvement,advantages,and disadvantages.Finally,four summaries and four future research prospects are presented.In summary,this paper aims to provide an up-to-date and well-rounded manual for subsequent researchers and scholars engaged in related fields.