基于DQN算法的考虑AGV小车搬运的离散制造车间调度方法

Study on the Discrete Manufacturing Workshop Scheduling Method Based on DQN Algorithm Considering AGV

针对离散制造车间生产调度不仅需要确定工件各工序的加工设备及设备上工序的加工顺序,同时要根据工件调度方案,需要在规定时间点前由AGV小车将各工件运送到工序相应的设备上加工,以提高调度方案执行率的需求,构建考虑车间设备布局、工件工艺路线、AGV小车搬运时间与小车位置等约束,工件完工时间最小化和AGV小车运载均衡为综合目标的离散制造车间调度模型。依据离散制造车间调度数学模型构建强化学习环境,包括工件、机器和小车的状态空间,调度决策动作空间和奖励函数;基于建立的强化学习环境,设计基于DQN算法的工件小车调度方法,设计工件智能体,读取车间局部环境,将局部环境映射到工件状态参数的权重,根据该权重得到工件调度列表实现从车间状态到工件调度的动作选择。设计小车智能体,通过读取工件智能体调度决策和车间信息得到小车搬运相关参数,实现小车智能体与工件智能体的交互,将搬运相关参数和车间局部环境中小车状态信息映射成小车调度相关权重,根据权重得到小车调度列表实现小车调度的动作选择。最后,通过离散制造车间实际案例对算法进行测试,测试结果表明,基于DQN算法的调度算法能够有效地求解考虑小车搬运的离散制造车间调度问题,可最小化工件的最大完工时间,均衡小车的搬运负载,具有良好的综合调度性能。

For the production scheduling of discrete manufacturing workshops,it is not only necessary to determine the processing machine of each process of the job and the processing sequence of the processes on the machine,but also according to the job scheduling plan,the AGV needs to transport each job to the corresponding machine for processing before the specified time point,In order to improve the execution rate of the scheduling scheme,a discrete manufacturing workshop scheduling model is constructed that considers constraints such as workshop machine layout,job process route,AGV handling time and AGV position,and minimizes job completion time and AGV load balance as comprehensive goals.Build a reinforcement learning environment based on the discrete manufacturing workshop scheduling mathematical model,including the state space of job,machine and car,scheduling decision action space and reward function;based on the established reinforcement learning environment,design a job car scheduling method based on DQN algorithm,and design job agent,read the local environment of the workshop,map the local environment to the weight of the relevant parameters of the job,and obtain the job scheduling list according to the weight to realize the action selection from the workshop state to the job scheduling.Design the AGV agent,and obtain the relevant parameters of the AGV handling by reading the scheduling decision and workshop information of the job agent,and realize the interaction between the AGV agent and the job agent.The handling related parameters and the car status information in the local environment of the workshop are mapped into the relevant weights of the car scheduling,and the car scheduling list is obtained according to the weights to realize the action selection of the car scheduling.Finally,the algorithm is tested through the actual case of discrete manufacturing workshop.The test results show that the scheduling algorithm based on the DQN algorithm can effectively solve the discrete manufacturing workshop scheduling problem considering the handling of AGVs,minimize the maximum completion time of job,and balance the handling of AGVs load.