To solve the sparse reward problem of job-shop scheduling by deep reinforcement learning,a deep reinforcement learning framework considering sparse reward problem is proposed.The job shop scheduling problem is transfo...To solve the sparse reward problem of job-shop scheduling by deep reinforcement learning,a deep reinforcement learning framework considering sparse reward problem is proposed.The job shop scheduling problem is transformed into Markov decision process,and six state features are designed to improve the state feature representation by using two-way scheduling method,including four state features that distinguish the optimal action and two state features that are related to the learning goal.An extended variant of graph isomorphic network GIN++is used to encode disjunction graphs to improve the performance and generalization ability of the model.Through iterative greedy algorithm,random strategy is generated as the initial strategy,and the action with the maximum information gain is selected to expand it to optimize the exploration ability of Actor-Critic algorithm.Through validation of the trained policy model on multiple public test data sets and comparison with other advanced DRL methods and scheduling rules,the proposed method reduces the minimum average gap by 3.49%,5.31%and 4.16%,respectively,compared with the priority rule-based method,and 5.34%compared with the learning-based method.11.97%and 5.02%,effectively improving the accuracy of DRL to solve the approximate solution of JSSP minimum completion time.展开更多
Maintenance scheduling is essential and crucial for wind turbines (WTs) to avoid breakdowns andreduce maintenance costs. Many maintenance models have been developed for WTs’ maintenance planning, suchas corrective, p...Maintenance scheduling is essential and crucial for wind turbines (WTs) to avoid breakdowns andreduce maintenance costs. Many maintenance models have been developed for WTs’ maintenance planning, suchas corrective, preventive, and predictive maintenance. Due to communities’ dependence on WTs for electricityneeds, preventive maintenance is the most widely used method for maintenance scheduling. The downside tousing this approach is that preventive maintenance (PM) is often done in fixed intervals, which is inefficient. In thispaper, a more detailed maintenance plan for a 2 MW WT has been developed. The paper’s focus is to minimize aWT’s maintenance cost based on a WT’s reliability model. This study uses a two-layer optimization framework:Fibonacci and genetic algorithm. The first layer in the optimization method (Fibonacci) finds the optimal numberof PM required for the system. In the second layer, the optimal times for preventative maintenance and optimalcomponents to maintain have been determined to minimize maintenance costs. The Monte Carlo simulationestimates WT component failure times using their lifetime distributions from the reliability model. The estimatedfailure times are then used to determine the overall corrective and PM costs during the system’s lifetime. Finally,an optimal PM schedule is proposed for a 2 MW WT using the presented method. The method used in this papercan be expanded to a wind farm or similar engineering systems.展开更多
可再生能源和负荷的波动性、不确定性等给综合能源系统(integrated energy system,IES)的安全灵活运行带来了极大挑战。为提高IES灵活调节能力与可再生能源消纳水平,提出一种计及灵活性资源的IES源荷协调优化调度方法。针对系统内运行...可再生能源和负荷的波动性、不确定性等给综合能源系统(integrated energy system,IES)的安全灵活运行带来了极大挑战。为提高IES灵活调节能力与可再生能源消纳水平,提出一种计及灵活性资源的IES源荷协调优化调度方法。针对系统内运行灵活性需求,精细刻画各类资源灵活性能力,源侧根据电氢耦合单元运行特性构建热电联产机组(combined heating and power,CHP)和氢燃料电池(hydrogen fuel cell,HFC)联合运行模型,荷侧考虑综合需求响应的灵活性供给能力,建立系统综合灵活性供给模型。根据不同时刻运行灵活性不足问题分成2种调度模式,构建基于IES灵活性约束的优化调度模型,并进行仿真分析。仿真结果表明,所提出的优化调度方法能够有效提高IES灵活调节能力和可再生能源消纳水平。展开更多
To solve the problem of small amount of machining centers in small and medium flexible manufacture systems(FMS), a scheduling mode of single automated guided vehicle(AGV) is adopted to deal with multiple transport req...To solve the problem of small amount of machining centers in small and medium flexible manufacture systems(FMS), a scheduling mode of single automated guided vehicle(AGV) is adopted to deal with multiple transport requests in this paper. Firstly, a workshop scheduling mechanism of AGV is analyzed and a mathematical model is established using Genetic Algorithm. According to several sets of transport priority of AGV, processes of FMS are encoded, and fitness function, selection, crossover, and variation methods are designed. The transport priority which has the least impact on scheduling results is determined based on the simulation analysis of Genetic Algorithm, and the makespan, the longest waiting time, and optimal route of the car are calculated. According to the actual processing situation of the workshop, feasibility of this method is verified successfully to provide an effective solution to the scheduling problem of single AGV.展开更多
The architecture of edge-cloud cooperation is proposed as a compromising solution that combines the advantage of MEC and central cloud. In this paper we investigated the problem of how to reduce the average delay of M...The architecture of edge-cloud cooperation is proposed as a compromising solution that combines the advantage of MEC and central cloud. In this paper we investigated the problem of how to reduce the average delay of MEC application by collaborative task scheduling. The collaborative task scheduling is modeled as a constrained shortest path problem over an acyclic graph. By characterizing the optimal solution, the constrained optimization problem is simplified according to one-climb theory and enumeration algorithm. Generally, the edge-cloud collaborative task scheduling scheme performance better than independent scheme in reducing average delay. In heavy workload scenario, high blocking probability and retransmission delay at MEC is the key factor for average delay. Hence, more task executed on central cloud with abundant resource is the optimal scheme. Otherwise, transmission delay is inevitable compared with execution delay. MEC configured with higher priority and deployed close to terminals obtain more performance gain.展开更多
SATech-01 is an experimental satellite for space science exploration and on-orbit demonstration of advanced technologies.The satellite is equipped with 16 experimental payloads and supports multiple working modes to m...SATech-01 is an experimental satellite for space science exploration and on-orbit demonstration of advanced technologies.The satellite is equipped with 16 experimental payloads and supports multiple working modes to meet the observation requirements of various payloads.Due to the limitation of platform power supply and data storage systems,proposing reasonable mission planning schemes to improve scientific revenue of the payloads becomes a critical issue.In this article,we formulate the integrated task scheduling of SATech-01 as a multi-objective optimization problem and propose a novel Fair Integrated Scheduling with Proximal Policy Optimization(FIS-PPO)algorithm to solve it.We use multiple decision heads to generate decisions for each task and design the action mask to ensure the schedule meeting the platform constraints.Experimental results show that FIS-PPO could push the capability of the platform to the limit and improve the overall observation efficiency by 31.5%compared to rule-based plans currently used.Moreover,fairness is considered in the reward design and our method achieves much better performance in terms of equal task opportunities.Because of its low computational complexity,our task scheduling algorithm has the potential to be directly deployed on board for real-time task scheduling in future space projects.展开更多
基金Shaanxi Provincial Key Research and Development Project(2023YBGY095)and Shaanxi Provincial Qin Chuangyuan"Scientist+Engineer"project(2023KXJ247)Fund support.
文摘To solve the sparse reward problem of job-shop scheduling by deep reinforcement learning,a deep reinforcement learning framework considering sparse reward problem is proposed.The job shop scheduling problem is transformed into Markov decision process,and six state features are designed to improve the state feature representation by using two-way scheduling method,including four state features that distinguish the optimal action and two state features that are related to the learning goal.An extended variant of graph isomorphic network GIN++is used to encode disjunction graphs to improve the performance and generalization ability of the model.Through iterative greedy algorithm,random strategy is generated as the initial strategy,and the action with the maximum information gain is selected to expand it to optimize the exploration ability of Actor-Critic algorithm.Through validation of the trained policy model on multiple public test data sets and comparison with other advanced DRL methods and scheduling rules,the proposed method reduces the minimum average gap by 3.49%,5.31%and 4.16%,respectively,compared with the priority rule-based method,and 5.34%compared with the learning-based method.11.97%and 5.02%,effectively improving the accuracy of DRL to solve the approximate solution of JSSP minimum completion time.
基金the Natural Sciences and Engineering Research Council of Canada(Grant No.RGPIN-2019-05361)and the University Research Grants Program.
文摘Maintenance scheduling is essential and crucial for wind turbines (WTs) to avoid breakdowns andreduce maintenance costs. Many maintenance models have been developed for WTs’ maintenance planning, suchas corrective, preventive, and predictive maintenance. Due to communities’ dependence on WTs for electricityneeds, preventive maintenance is the most widely used method for maintenance scheduling. The downside tousing this approach is that preventive maintenance (PM) is often done in fixed intervals, which is inefficient. In thispaper, a more detailed maintenance plan for a 2 MW WT has been developed. The paper’s focus is to minimize aWT’s maintenance cost based on a WT’s reliability model. This study uses a two-layer optimization framework:Fibonacci and genetic algorithm. The first layer in the optimization method (Fibonacci) finds the optimal numberof PM required for the system. In the second layer, the optimal times for preventative maintenance and optimalcomponents to maintain have been determined to minimize maintenance costs. The Monte Carlo simulationestimates WT component failure times using their lifetime distributions from the reliability model. The estimatedfailure times are then used to determine the overall corrective and PM costs during the system’s lifetime. Finally,an optimal PM schedule is proposed for a 2 MW WT using the presented method. The method used in this papercan be expanded to a wind farm or similar engineering systems.
文摘可再生能源和负荷的波动性、不确定性等给综合能源系统(integrated energy system,IES)的安全灵活运行带来了极大挑战。为提高IES灵活调节能力与可再生能源消纳水平,提出一种计及灵活性资源的IES源荷协调优化调度方法。针对系统内运行灵活性需求,精细刻画各类资源灵活性能力,源侧根据电氢耦合单元运行特性构建热电联产机组(combined heating and power,CHP)和氢燃料电池(hydrogen fuel cell,HFC)联合运行模型,荷侧考虑综合需求响应的灵活性供给能力,建立系统综合灵活性供给模型。根据不同时刻运行灵活性不足问题分成2种调度模式,构建基于IES灵活性约束的优化调度模型,并进行仿真分析。仿真结果表明,所提出的优化调度方法能够有效提高IES灵活调节能力和可再生能源消纳水平。
基金Supported by the National Natural Science Foundation of China(No.51765043)
文摘To solve the problem of small amount of machining centers in small and medium flexible manufacture systems(FMS), a scheduling mode of single automated guided vehicle(AGV) is adopted to deal with multiple transport requests in this paper. Firstly, a workshop scheduling mechanism of AGV is analyzed and a mathematical model is established using Genetic Algorithm. According to several sets of transport priority of AGV, processes of FMS are encoded, and fitness function, selection, crossover, and variation methods are designed. The transport priority which has the least impact on scheduling results is determined based on the simulation analysis of Genetic Algorithm, and the makespan, the longest waiting time, and optimal route of the car are calculated. According to the actual processing situation of the workshop, feasibility of this method is verified successfully to provide an effective solution to the scheduling problem of single AGV.
文摘The architecture of edge-cloud cooperation is proposed as a compromising solution that combines the advantage of MEC and central cloud. In this paper we investigated the problem of how to reduce the average delay of MEC application by collaborative task scheduling. The collaborative task scheduling is modeled as a constrained shortest path problem over an acyclic graph. By characterizing the optimal solution, the constrained optimization problem is simplified according to one-climb theory and enumeration algorithm. Generally, the edge-cloud collaborative task scheduling scheme performance better than independent scheme in reducing average delay. In heavy workload scenario, high blocking probability and retransmission delay at MEC is the key factor for average delay. Hence, more task executed on central cloud with abundant resource is the optimal scheme. Otherwise, transmission delay is inevitable compared with execution delay. MEC configured with higher priority and deployed close to terminals obtain more performance gain.
基金supported by the Strategic Priority Program on Space Science,Chinese Academy of Sciences。
文摘SATech-01 is an experimental satellite for space science exploration and on-orbit demonstration of advanced technologies.The satellite is equipped with 16 experimental payloads and supports multiple working modes to meet the observation requirements of various payloads.Due to the limitation of platform power supply and data storage systems,proposing reasonable mission planning schemes to improve scientific revenue of the payloads becomes a critical issue.In this article,we formulate the integrated task scheduling of SATech-01 as a multi-objective optimization problem and propose a novel Fair Integrated Scheduling with Proximal Policy Optimization(FIS-PPO)algorithm to solve it.We use multiple decision heads to generate decisions for each task and design the action mask to ensure the schedule meeting the platform constraints.Experimental results show that FIS-PPO could push the capability of the platform to the limit and improve the overall observation efficiency by 31.5%compared to rule-based plans currently used.Moreover,fairness is considered in the reward design and our method achieves much better performance in terms of equal task opportunities.Because of its low computational complexity,our task scheduling algorithm has the potential to be directly deployed on board for real-time task scheduling in future space projects.
