基于强化学习的柴油机调速算法研究

Research on Diesel Engine Speed Regulation Algorithm Based on Reinforcement Learning

为了更好地调节柴油机转速,提出一种强化学习–比例积分微分(proportional integral derivative, PID)控制器,并应用到了柴油机转速控制中。基于连续动作空间的柔性动作–评价(soft actor-critic, SAC)算法,结合连续型PID控制器,设计了一种强化学习–PID控制器,可代替传统PID控制的转速环。优化设计了基于演员–评论家(actor-critic)框架的输入输出和奖励函数以匹配柴油机特性,采用随机动作增加寻优效率,形成SAC-PID控制柴油机转速的网络交互结构,达到快速调整转速,减小稳定时间的效果。构建了以柴油机D6114为原型机的MATLAB/Simulink平均值模型,并利用试验数据验证了仿真模型的有效性。利用平均值模型,仿真验证了该控制算法效果。经过仿真验证本算法使柴油机转速响应曲线超调量更小,响应时间更快,鲁棒性更强,SAC-PID控制负载瞬态调速率和稳定时间均已达到1级精度指标。仿真对比验证了SAC算法的联合控制效果,结果表明其较其他算法更佳。

In order to better regulate the speed of diesel engines,a reinforcement learning proportional-integral-derivative(PID)controller was proposed and applied to diesel engine speed control.Based on the soft actor-critic(SAC)algorithm with continuous action space,combined with a continuous PID controller,a reinforcement learning PID controller that can replace the speed loop of traditional PID control was designed.The design of input-output and reward functions based on the actor-critic framework was optimized to match the characteristics of diesel engines.Random actions were used to increase optimization efficiency,forming a network interaction structure for SAC-PID control of diesel engine speed,achieving the effect of quickly adjusting speed and reducing stabilization time.A MATLAB/Simulink average model was constructed using the diesel engine D6114 as the prototype,and the effectiveness of the simulation model was verified using experimental data.The average value model was used to simulate the effectiveness of this control algorithm.The simulation verification results show that the SAC-PID algorithm reduces the overshoot of the diesel engine speed response curve,makes the response time faster,and has stronger robustness.The SAC-PID control load transient speed regulation rate and stability time have reached the first level accuracy index,and the joint control effect of SAC algorithm is better than other algorithms.