Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput.This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems.First,integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing,which reduces the search scope of the database and dramatically speeds up data searching.Next,exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first,which reduces the average waiting time of job execution.As a result,the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing,significantly improving the speed of data retrieval up to 53%and increasing system throughput by 53%.On the other hand,the proposed job scheduling algorithmdefeats both first-in-first-out andmemory-sensitive heterogeneous early finish time scheduling algorithms,effectively shortening the average waiting time up to 5%and average weighted turnaround time by 19%,respectively.

A Heuristic for the Job Scheduling Problem with a Common Due Window on Parallel and Non-Identical Machines

In this paper, we give a mathematical model for earliness-tardiness job scheduling problem with a common due window on parallel and non-identical machines. Because the job scheduling problem discussed in the paper contains a problem of minimizing make-span, which is NP-complete on parallel and uniform machines, a heuristic algorithm is presented to find an approximate solution for the scheduling problem after proving an important theorem. Two numerical examples illustrate that the heuristic algorithm is very useful and effective in obtaining the near-optimal solution.

Energy-Saving Distributed Flexible Job Shop Scheduling Optimization with Dual Resource Constraints Based on Integrated Q-Learning Multi-Objective Grey Wolf Optimizer

The distributed flexible job shop scheduling problem(DFJSP)has attracted great attention with the growth of the global manufacturing industry.General DFJSP research only considers machine constraints and ignores worker constraints.As one critical factor of production,effective utilization of worker resources can increase productivity.Meanwhile,energy consumption is a growing concern due to the increasingly serious environmental issues.Therefore,the distributed flexible job shop scheduling problem with dual resource constraints(DFJSP-DRC)for minimizing makespan and total energy consumption is studied in this paper.To solve the problem,we present a multi-objective mathematical model for DFJSP-DRC and propose a Q-learning-based multi-objective grey wolf optimizer(Q-MOGWO).In Q-MOGWO,high-quality initial solutions are generated by a hybrid initialization strategy,and an improved active decoding strategy is designed to obtain the scheduling schemes.To further enhance the local search capability and expand the solution space,two wolf predation strategies and three critical factory neighborhood structures based on Q-learning are proposed.These strategies and structures enable Q-MOGWO to explore the solution space more efficiently and thus find better Pareto solutions.The effectiveness of Q-MOGWO in addressing DFJSP-DRC is verified through comparison with four algorithms using 45 instances.The results reveal that Q-MOGWO outperforms comparison algorithms in terms of solution quality.

Strengthened Dominance Relation NSGA-Ⅲ Algorithm Based on Differential Evolution to Solve Job Shop Scheduling Problem

The job shop scheduling problem is a classical combinatorial optimization challenge frequently encountered in manufacturing systems.It involves determining the optimal execution sequences for a set of jobs on various machines to maximize production efficiency and meet multiple objectives.The Non-dominated Sorting Genetic Algorithm Ⅲ(NSGA-Ⅲ)is an effective approach for solving the multi-objective job shop scheduling problem.Nevertheless,it has some limitations in solving scheduling problems,including inadequate global search capability,susceptibility to premature convergence,and challenges in balancing convergence and diversity.To enhance its performance,this paper introduces a strengthened dominance relation NSGA-Ⅲ algorithm based on differential evolution(NSGA-Ⅲ-SD).By incorporating constrained differential evolution and simulated binary crossover genetic operators,this algorithm effectively improves NSGA-Ⅲ’s global search capability while mitigating pre-mature convergence issues.Furthermore,it introduces a reinforced dominance relation to address the trade-off between convergence and diversity in NSGA-Ⅲ.Additionally,effective encoding and decoding methods for discrete job shop scheduling are proposed,which can improve the overall performance of the algorithm without complex computation.To validate the algorithm’s effectiveness,NSGA-Ⅲ-SD is extensively compared with other advanced multi-objective optimization algorithms using 20 job shop scheduling test instances.The experimental results demonstrate that NSGA-Ⅲ-SD achieves better solution quality and diversity,proving its effectiveness in solving the multi-objective job shop scheduling problem.

An Improved Harris Hawk Optimization Algorithm for Flexible Job Shop Scheduling Problem

Flexible job shop scheduling problem(FJSP)is the core decision-making problem of intelligent manufacturing production management.The Harris hawk optimization(HHO)algorithm,as a typical metaheuristic algorithm,has been widely employed to solve scheduling problems.However,HHO suffers from premature convergence when solving NP-hard problems.Therefore,this paper proposes an improved HHO algorithm(GNHHO)to solve the FJSP.GNHHO introduces an elitism strategy,a chaotic mechanism,a nonlinear escaping energy update strategy,and a Gaussian random walk strategy to prevent premature convergence.A flexible job shop scheduling model is constructed,and the static and dynamic FJSP is investigated to minimize the makespan.This paper chooses a two-segment encoding mode based on the job and the machine of the FJSP.To verify the effectiveness of GNHHO,this study tests it in 23 benchmark functions,10 standard job shop scheduling problems(JSPs),and 5 standard FJSPs.Besides,this study collects data from an agricultural company and uses the GNHHO algorithm to optimize the company’s FJSP.The optimized scheduling scheme demonstrates significant improvements in makespan,with an advancement of 28.16%for static scheduling and 35.63%for dynamic scheduling.Moreover,it achieves an average increase of 21.50%in the on-time order delivery rate.The results demonstrate that the performance of the GNHHO algorithm in solving FJSP is superior to some existing algorithms.

A Novel Collaborative Evolutionary Algorithm with Two-Population for Multi-Objective Flexible Job Shop Scheduling

Job shop scheduling(JS)is an important technology for modern manufacturing.Flexible job shop scheduling(FJS)is critical in JS,and it has been widely employed in many industries,including aerospace and energy.FJS enables any machine from a certain set to handle an operation,and this is an NP-hard problem.Furthermore,due to the requirements in real-world cases,multi-objective FJS is increasingly widespread,thus increasing the challenge of solving the FJS problems.As a result,it is necessary to develop a novel method to address this challenge.To achieve this goal,a novel collaborative evolutionary algorithmwith two-population based on Pareto optimality is proposed for FJS,which improves the solutions of FJS by interacting in each generation.In addition,several experimental results have demonstrated that the proposed method is promising and effective for multi-objective FJS,which has discovered some new Pareto solutions in the well-known benchmark problems,and some solutions can dominate the solutions of some other methods.

An Effective Neighborhood Solution Clipping Method for Large-Scale Job Shop Scheduling Problem

The job shop scheduling problem(JSSP)is a classical combinatorial optimization problem that exists widely in diverse scenarios of manufacturing systems.It is a well-known NP-hard problem,when the number of jobs increases,the difficulty of solving the problem exponentially increases.Therefore,a major challenge is to increase the solving efficiency of current algorithms.Modifying the neighborhood structure of the solutions can effectively improve the local search ability and efficiency.In this paper,a genetic Tabu search algorithm with neighborhood clipping(GTS_NC)is proposed for solving JSSP.A neighborhood solution clipping method is developed and embedded into Tabu search to improve the efficiency of the local search by clipping the search actions of unimproved neighborhood solutions.Moreover,a feasible neighborhood solution determination method is put forward,which can accurately distinguish feasible neighborhood solutions from infeasible ones.Both of the methods are based on the domain knowledge of JSSP.The proposed algorithmis compared with several competitive algorithms on benchmark instances.The experimental results show that the proposed algorithm can achieve superior results compared to other competitive algorithms.According to the numerical results of the experiments,it is verified that the neighborhood solution clippingmethod can accurately identify the unimproved solutions and reduces the computational time by at least 28%.

Applying Job Shop Scheduling to SMEs Manufacturing Platform to Revitalize B2B Relationship

A small and medium enterprises(SMEs)manufacturing platform aims to perform as a significant revenue to SMEs and vendors by providing scheduling and monitoring capabilities.The optimal job shop scheduling is generated by utilizing the scheduling system of the platform,and a minimum production time,i.e.,makespan decides whether the scheduling is optimal or not.This scheduling result allows manufacturers to achieve high productivity,energy savings,and customer satisfaction.Manufacturing in Industry 4.0 requires dynamic,uncertain,complex production environments,and customer-centered services.This paper proposes a novel method for solving the difficulties of the SMEs manufacturing by applying and implementing the job shop scheduling system on a SMEs manufacturing platform.The primary purpose of the SMEs manufacturing platform is to improve the B2B relationship between manufacturing companies and vendors.The platform also serves qualified and satisfactory production opportunities for buyers and producers by meeting two key factors:early delivery date and fulfillment of processing as many orders as possible.The genetic algorithm(GA)-based scheduling method results indicated that the proposed platform enables SME manufacturers to obtain optimized schedules by solving the job shop scheduling problem(JSSP)by comparing with the real-world data from a textile weaving factory in South Korea.The proposed platform will provide producers with an optimal production schedule,introduce new producers to buyers,and eventually foster relationships and mutual economic interests.

On/Off-Line Prediction Applied to Job Scheduling on Non-Dedicated NOWs

This paper proposes a prediction engine designed for non-dedicated clusters, which is able to estimate the turnaround time for parallel applications, even in the presence of serial workload of the workstation owner. The prediction engine can be configured to work with three different estimation kernels： a Historical kernel, a Simulation kernel based on analytical models and an integration of both, named Hybrid kernel. These estimation proposals were integrated into a scheduling system, named CISNE, which can be executed in an on-line or off-line mode. The accuracy of the proposed estimation methods was evaluated in relation to different job scheduling policies in a real and a simulated cluster environment. In both environments, we observed that the Hybrid system gives the best results because it combines the ability of a simulation engine to capture the dynamism of a non-dedicated environment together with the accuracy of the historical methods to estimate the application runtime considering the state of the resources.

Improved Heuristic Job Scheduling Method to Enhance Throughput for Big Data Analytics

Data-parallel computing platforms,such as Hadoop and Spark,are deployed in computing clusters for big data analytics.There is a general tendency that multiple users share the same computing cluster.The schedule of multiple jobs becomes a serious challenge.Over a long period in the past,the Shortest-Job-First(SJF)method has been considered as the optimal solution to minimize the average job completion time.However,the SJF method leads to a low system throughput in the case where a small number of short jobs consume a large amount of resources.This factor prolongs the average job completion time.We propose an improved heuristic job scheduling method,called the Densest-Job-Set-First(DJSF)method.The DJSF method schedules jobs by maximizing the number of completed jobs per unit time,aiming to decrease the average Job Completion Time(JCT)and improve the system throughput.We perform extensive simulations based on Google cluster data.Compared with the SJF method,the DJSF method decreases the average JCT by 23.19% and enhances the system throughput by 42.19%.Compared with Tetris,the job packing method improves the job completion efficiency by 55.4%,so that the computing platforms complete more jobs in a short time span.

Fuzzy Logic-Based Secure and Fault Tolerant Job Scheduling in Grid

The uncertainties of grid sites security are main hurdle to make the job scheduling secure, reliable and fault-tolerant. Most existing scheduling algorithms use fixed-number job replications to provide fault tolerant ability and high scheduling success rate, which consume excessive resources or can not provide sufficient fault tolerant functions when grid security conditions change. In this paper a fuzzy-logic-based self-adaptive replication scheduling (FSARS) algorithm is proposed to handle the fuzziness or uncertainties of job replication number which is highly related to trust factors behind grid sites and user jobs. Remote sensing-based soil moisture extraction (RSBSME) workload experiments in real grid environment are performed to evaluate the proposed approach and the results show that high scheduling success rate of up to 95% and less grid resource utilization can be achieved through FSARS. Extensive experiments show that FSARS scales well when user jobs and grid sites increase.

QoS-aware simulation job scheduling algorithm in virtualized cloud environment

Cloud computing is attracting an increasing number of simulation applications running in the virtualized cloud data center.These applications are submitted to the cloud in the form of simulation jobs.Meanwhile,the management and scheduling of simulation jobs are playing an essential role to offer efficient and high productivity computational service.In this paper,we design a management and scheduling service framework for simulation jobs in two-tier virtualization-based private cloud data center,named simulation execution as a service(SimEaaS).It aims at releasing users from complex simulation running settings,while guaranteeing the QoS requirements adaptively.Furthermore,a novel job scheduling algorithm named adaptive deadline-aware job size adjustment(ADaSA)algorithm is designed to realize high job responsiveness under QoS requirement for SimEaaS.ADaSA tries to make full use of the idle fragmentation resources by tuning the number of requested processes of submitted jobs in the queue adaptively,while guaranteeing that jobs’deadline requirements are not violated.Extensive experiments with trace-driven simulation are conducted to evaluate the performance of our ADaSA.The results show that ADaSA outperforms both cloud-based job scheduling algorithm KCEASY and traditional EASY in terms of response time(up to 90%)and bounded slow down(up to 95%),while obtains approximately equivalent deadline-missed rate.ADaSA also outperforms two representative moldable scheduling algorithms in terms of deadline-missed rate(up to 60%).

Modified bottleneck-based heuristic for large-scale job-shop scheduling problems with a single bottleneck

A modified bottleneck-based （MB） heuristic for large-scale job-shop scheduling problems with a welldefined bottleneck is suggested, which is simpler but more tailored than the shifting bottleneck （SB） procedure. In this algorithm, the bottleneck is first scheduled optimally while the non-bottleneck machines are subordinated around the solutions of the bottleneck schedule by some effective dispatching rules. Computational results indicate that the MB heuristic can achieve a better tradeoff between solution quality and computational time compared to SB procedure for medium-size problems. Furthermore, it can obtain a good solution in a short time for large-scale jobshop scheduling problems.

Clonal Selection Based Memetic Algorithm for Job Shop Scheduling Problems

A clonal selection based memetic algorithm is proposed for solving job shop scheduling problems in this paper. In the proposed algorithm, the clonal selection and the local search mechanism are designed to enhance exploration and exploitation. In the clonal selection mechanism, clonal selection, hypermutation and receptor edit theories are presented to construct an evolutionary searching mechanism which is used for exploration. In the local search mechanism, a simulated annealing local search algorithm based on Nowicki and Smutnicki＇s neighborhood is presented to exploit local optima. The proposed algorithm is examined using some well-known benchmark problems. Numerical results validate the effectiveness of the proposed algorithm.

Solving flexible job shop scheduling problem by a multi-swarm collaborative genetic algorithm

The flexible job shop scheduling problem(FJSP),which is NP-hard,widely exists in many manufacturing industries.It is very hard to be solved.A multi-swarm collaborative genetic algorithm(MSCGA)based on the collaborative optimization algorithm is proposed for the FJSP.Multi-population structure is used to independently evolve two sub-problems of the FJSP in the MSCGA.Good operators are adopted and designed to ensure this algorithm to achieve a good performance.Some famous FJSP benchmarks are chosen to evaluate the effectiveness of the MSCGA.The adaptability and superiority of the proposed method are demonstrated by comparing with other reported algorithms.

Emergency Local Searching Approach for Job Shop Scheduling

Existing methods of local search mostly focus on how to reach optimal solution.However,in some emergency situations,search time is the hard constraint for job shop scheduling problem while optimal solution is not necessary.In this situation,the existing method of local search is not fast enough.This paper presents an emergency local search(ELS) approach which can reach feasible and nearly optimal solution in limited search time.The ELS approach is desirable for the aforementioned emergency situations where search time is limited and a nearly optimal solution is sufficient,which consists of three phases.Firstly,in order to reach a feasible and nearly optimal solution,infeasible solutions are repaired and a repair technique named group repair is proposed.Secondly,in order to save time,the amount of local search moves need to be reduced and this is achieved by a quickly search method named critical path search(CPS).Finally,CPS sometimes stops at a solution far from the optimal one.In order to jump out the search dilemma of CPS,a jump technique based on critical part is used to improve CPS.Furthermore,the schedule system based on ELS has been developed and experiments based on this system completed on the computer of Intel Pentium(R) 2.93 GHz.The experimental result shows that the optimal solutions of small scale instances are reached in 2 s,and the nearly optimal solutions of large scale instances are reached in 4 s.The proposed ELS approach can stably reach nearly optimal solutions with manageable search time,and can be applied on some emergency situations.

NEW NONSTANDARD JOB SHOP SCHEDULING ALGORITHM

Considering the complex constraint between operations in nonstandard job shop scheduling problem （NJSSP）, critical path of job manufacturing tree is determined according to priority scheduling function constructed. Operations are divided into dependent operations and independent operations with the idea of subsection, and corresponding scheduling strategy is put forward according to operation characteristic in the segment and the complementarities of identical function machines. Forward greedy rule is adopted mainly for dependent operations to make operations arranged in the right position of machine selected, then each operation can be processed as early as possible, and the total processing time of job can be shortened as much as possible. For independent operations optimum scheduling rule is adopted mainly, the inserting position of operations will be determined according to the gap that the processing time of operations is subtracted from idle time of machine, and the operation will be inserted in the position with minimal gap. Experiments show, under the same conditions, the result that operations are scheduled according to the object function constructed, and the scheduling strategy adopted is better than the result that operations are scheduled according to efficiency scheduling algorithm.

Deep Reinforcement Learning-Based Job Shop Scheduling of Smart Manufacturing

Industry 4.0 production environments and smart manufacturing systems integrate both the physical and decision-making aspects of manufacturing operations into autonomous and decentralized systems.One of the key aspects of these systems is a production planning,specifically,Scheduling operations on the machines.To cope with this problem,this paper proposed a Deep Reinforcement Learning with an Actor-Critic algorithm(DRLAC).We model the Job-Shop Scheduling Problem(JSSP)as a Markov Decision Process(MDP),represent the state of a JSSP as simple Graph Isomorphism Networks(GIN)to extract nodes features during scheduling,and derive the policy of optimal scheduling which guides the included node features to the best next action of schedule.In addition,we adopt the Actor-Critic(AC)network’s training algorithm-based reinforcement learning for achieving the optimal policy of the scheduling.To prove the proposed model’s effectiveness,first,we will present a case study that illustrated a conflict between two job scheduling,secondly,we will apply the proposed model to a known benchmark dataset and compare the results with the traditional scheduling methods and trending approaches.The numerical results indicate that the proposed model can be adaptive with real-time production scheduling,where the average percentage deviation(APD)of our model achieved values between 0.009 and 0.21 comparedwith heuristic methods and values between 0.014 and 0.18 compared with other trending approaches.

Combined Timetabling Procedure and Complete Local Search for No-Wait Job Shop Scheduling with Total Tardiness

The strong non-deterministic polynomial-hard( NP-hard)character of job shop scheduling problem( JSSP) has been acknowledged widely and it becomes stronger when attaches the nowait constraint,which widely exists in many production processes,such as chemistry process, metallurgical process. However,compared with the massive research on traditional job shop problem,little attention has been paid on the no-wait constraint.Therefore,in this paper, we have dealt with this problem by decomposing it into two sub-problems, the timetabling and sequencing problems,in traditional frame work. A new efficient combined non-order timetabling method,coordinated with objective of total tardiness,is proposed for the timetabling problems. As for the sequencing one,we have presented a modified complete local search with memory combined by crossover operator and distance counting. The entire algorithm was tested on well-known benchmark problems and compared with several existing algorithms.Computational experiments showed that our proposed algorithm performed both effectively and efficiently.

TWO SCHEDULING PROBLEMS IN GROUP TECHNOLOGY WITH DETERIORATING JOBS

This paper considers single-machine scheduling problems in group technology with the jobs' processing times being simple linear functions of their start times.The objective functions are the ~minimizing of makespan and total weighted completion time.Some optimal conditions and algorithms are given and the fact that the problem of total weighted completion times is NP-hard is proved.