基于强化学习的混合动力挖掘机实时能量管理控制器设计（英文）

Real-time energy management controller design for a hybrid excavator using reinforcement learning

目的:混合动力挖掘机的能量管理策略直接影响着系统的燃油经济性。本文旨在通过研究混合动力挖掘机能量管理系统,得到最优能量管理策略,并开发实时能量管理控制器,降低系统的燃油消耗。创新点:1.通过强化学习算法,设计时间无关的实时能量管理控制器;2.通过极大值原理求得最优能量管理问题的解析解,并用来辅助实时能量管理控制器设计。方法:1.建立负载的马尔科夫模型,运用强化学习算法,得到实时能量管理控制器;2.运用极大值原理,求得最优能量管理问题的解析解,并将其作为初始能量管理策略;3.通过仿真模拟和实验研究,验证所设计的实时能量控制器的性能。结论:1.基于强化学习的能量管理控制器是一个可以在线应用的与时间无关的实时能量管理控制器;2.基于强化学习的能量管理控制器优于广泛使用的恒温控制器和等效消耗最小化策略控制器;3.基于强化学习的能量管理控制器由于其闭环特性可适用于不同类型的作业工况。

Real-time energy management of a hybrid excavator is addressed using reinforcement learning （RL）. Due to the computational complexity and need for a priori knowledge of the load cycles, a traditional optimal control method, like dynamic programming （DP）, is not feasible for real-time control. Real-time controllers derived from traditional optimal control methods compute the solutions either in a cycle-dependent manner or far away from the optimal. An RL-based energy management con- troller is proposed to solve this problem. The simulation and experimental results demonstrate that the RL controller has a better performance than the widely used thermostat and equivalent consumption minimization strategy （ECMS） controllers. It also shows that the RL controller is cycle-independent. Pontryagin＇s minimum principle （PMP） is used to obtain the analytical solution of the energy management problem, and this can help to reduce the iteration time in the design process.