基于强化学习的浓密机底流浓度在线控制算法

Online Reinforcement Learning Control Algorithm for Concentration of Thickener Underflow

复杂过程工业控制一直是控制应用领域研究的前沿问题.浓密机作为一种复杂大型工业设备广泛用于冶金、采矿等领域.由于其在运行过程中具有多变量、非线性、高时滞等特点,浓密机的底流浓度控制技术一直是学界、工业界的研究难点与热点.本文提出了一种基于强化学习技术的浓密机在线控制算法.该算法在传统启发式动态规划(Heuristic dynamic programming,HDP)算法的基础上,设计融合了评价网络与模型网络的双网结构,并提出了基于短期经验回放的方法用于增强评价网络的训练准确性,实现了对浓密机底流浓度的稳定控制,并保持控制输入稳定在设定范围之内.最后,通过浓密机仿真实验的方式验证了算法的有效性,实验结果表明本文提出的方法在时间消耗、控制精度上优于其他算法.

Complex process industrial control is a widely concerned problem in the field of control application.As a kind of complex huge industrial equipment,thickener has been widely used in metallurgy,mining and other applications.Due to its characteristics of complicated variables,nonlinear and long delay in the operational process,the control strategy of underflow concentration for thickener has always been a hot and difficult issue in the academia and industry.This paper proposes a novel online control algorithm for thickener which is based on reinforcement learning.Inspired by the traditional heuristic dynamic programming(Heuristic dynamic programming,HDP)algorithm.The proposed method designs a double net framework which is composed of the critic network and the model network.To achieve the stabilization of underflow concentration,an optimal method which is based on reviewing the history data in a short term is proposed in the training phase of critic network.Simulation experiments verify efficiency of the proposed method.The results show that the proposed method can maintain the concentration of underflow in a stable horizon and performs better than other algorithms in accuracy and time consuming.