基于多动作深度强化学习的纺机制造车间调度方法

Multi-action deep reinforcement learning based scheduling method for spinning machine manufacturing shop floor

纺机制造车间调度问题是一种具有复杂工艺约束和序列相关设置时间的柔性作业车间调度问题,为了保证调度方案的质量,提升企业的订单准时交付能力,提出了一种以最小化最大完工期为优化目标的多动作深度强化学习算法。首先,将调度问题建模为多马尔可夫决策过程。然后,针对纺机制造车间调度的工件选择和机器选择两个子问题,分别设计了用于定义工序选择策略和机器选择策略的两个编码器,以预测选择不同工序和机器的概率分布。其中,在工序选择编码器中,采用图神经网络对析取图进行编码,以降低问题规模对解的质量的影响。其次,提出了一种具有多动作空间的强化学习训练算法,用于学习两个子策略。最后,经某纺机制造企业的实际生产案例验证,该方法的性能受问题规模影响较小,与其他对比算法相比,能够获得较高质量的调度方案,训练的模型具有较好的泛化能力和稳定性。

The spinning machine manufacturing shop scheduling problem is a flexible Job-Shop scheduling problem with complex process constraints and sequence-dependent setup times.This paper proposed a multi-action deep reinforcement learning algorithm with the optimization objective of minimizing the maximum completion time to ensure the quality of the scheduling solution and improve the on-time order delivery capability of the enterprise.Firstly,this paper modeled the scheduling problem as a multi-Markov decision process.Then,in order to predict the probability distribution of selecting different processes and machines,this paper designed two encoders for the two sub-problems of workpiece selection and machine selection of spinning machine manufacturing plant scheduling,for defining the process selection policy and machine selection policy,respectively.In the process selection encoder,it used a graphical neural network to encode the disjunctive graph to reduce the impact of problem size on the quality of the solution.Based on this,the paper designed a reinforcement learning training algorithm with multiple action spaces for the two substrategies.Finally,it validated the proposed method on a real production case of a spinning machine manufacturing company.The results show that the method exhibits good performance on problems of different scales,is able to obtain higher quality scheduling solutions compared with other comparative algorithms,and the model has better generalization ability and stability.