基于深度强化学习的节能工艺路线发现方法

Energy-saving process route discovery method based on deep reinforcement learning

由于传统基于固定加工环境的工艺路线制定规则,无法快速响应加工环境的动态变化制定节能工艺路线。因此提出了基于深度Q网络(deep Q network,DQN)的节能工艺路线发现方法。基于马尔可夫决策过程,定义状态向量、动作空间、奖励函数,建立节能工艺路线模型,并将加工环境动态变化的节能工艺路线规划问题,转化为DQN智能体决策问题,利用决策经验的可复用性和可扩展性,进行求解,同时为了提高DQN的收敛速度和解的质量,提出了基于S函数探索机制和加权经验池,并使用了双Q网络。仿真结果表明,相比较改进前,改进后的算法在动态加工环境中能够更快更好地发现节能工艺路线;与遗传算法、模拟退火算法以及粒子群算法相比,改进后的算法不仅能够以最快地速度发现节能工艺路线,而且能得到相同甚至更高精度的解。

Due to the traditional process route formulation rules based on the fixed processing environment,it is unable to quickly respond to the dynamic changes of the processing environment to formulate energy-saving process routes.Therefore,an energy-saving process route discovery method based on deep Q network(DQN)is proposed in this paper.Based on the Markov decision process,we define the state vector,action space,and reward function,establish an energy-saving process route model,and transform the energy-saving process route planning problem with dynamic changes in the processing environment into a DQN agent decision-making problem,which uses the reusable and extensible decision-making experience to solve the problem.At the same time,an exploration mechanism based on the S function,a weighted experience pool,and a double-Q network are used to improve the convergence speed and solution quality of DQN.The simulation results show that compared with that before improvement,the improved algorithm can find energy-saving process routes faster and better in the dynamic processing environment;and compared with genetic algorithm,simulated annealing algorithm,as well as particle swarm algorithm,the improved algorithm can not only discover energy-saving process routes at the fastest speed,but also obtain the same or even higher precision solutions.