基于深度强化学习的柔性作业车间节能调度研究

Energy-efficient Flexible Job-shop Scheduling Based on Deep Reinforcement Learning

针对当前柔性作业车间节能调度研究无法充分利用历史生产数据,且对复杂、动态、多变的车间生产环境适应性不足的问题,引入深度强化学习思想,利用具有代表性的深度Q网络(deep Q-network,DQN)求解柔性作业车间节能调度问题。将柔性作业车间节能调度问题转化为强化学习对应的马尔科夫决策过程。进而,提炼表征车间生产状态特征的状态值作为神经网络输入,通过神经网络拟合状态值函数,输出复合调度动作规则实现对工件以及加工机器的选择,并利用动作规则与奖励函数协同优化能耗。在3个不同规模的案例上与非支配排序遗传算法、超启发式遗传算法、改进狼群算法等典型智能优化方法进行求解效果对比。结果表明,DQN算法有较强的搜索能力,且最优解分布情况与提出的柔性作业车间节能调度模型聚焦能耗目标相一致,从而验证了所用DQN方法的有效性。

The current research on energy-efficient flexible job-shop scheduling problems(EFJSPs)cannot make full use of historical production data,and is insufficiently adaptable to the complex,dynamic and changeable job-shop production environment.In view of this,deep reinforcement learning is introduced to solve EFJSPs,where a representative method named deep Q-network(DQN)is utilized.First,EFJSP is transformed into a Markov decision process corresponding to reinforcement learning.Moreover,the state values characterizing the job-shop production states are extracted as inputs of a neural network.By fitting the state value function through the neural network,compound scheduling action rules are output to achieve the selection of workpieces and processing machines.Furthermore,scheduling action rules and reward functions are utilized to jointly optimize the total production energy consumption.Finally,solutions of the proposed method are compared with those using typical intelligent optimization algorithms,such as non-dominated sorting genetic algorithm,hyper-heuristic genetic algorithm and multi-objective wolf pack algorithm,in three cases with different scales.Results demonstrate the powerful search capability of DQN algorithm,and the distribution of optimal solutions is consistent with the optimization objective obtained by the proposed EJFSP model.These verify the effectiveness of the utilized DQN method.