燃料电池电动汽车改进深度强化学习能量管理

Energy Management of Fuel Cell Electric Vehicle based on Improved Deep Reinforcement Learning

针对配置有燃料电池、锂电池和超级电容3能量源的混合动力汽车,提出一种基于改进深度确定性策略梯度(DDPG)算法的分层能量管理策略,以降低氢耗、提高燃料电池工作效率及维持锂电池荷电状态(SoC)。首先,采用基于模糊规则的自适应低通滤波器对功率进行分层处理,由超级电容承担峰值功率。其次,设计基于DDPG的能量管理框架,利用等效消耗最小策略的计算思想构建优化函数,并加入与燃料电池效率和锂电池SoC偏差有关的惩罚因子,优化燃料电池和锂电池的功率分配。此外,为避免噪声探索导致极端动作值的频繁出现,利用动态规划最优解辅助策略训练,提升优化效果。最后,在不同工况下进行仿真,并搭建试验平台进行验证。结果表明:与基于传统DDPG策略相比,所提策略可以有效减少锂电池SoC消耗,更好确保燃料电池工作在高效率区间,并且显著降低氢消耗,在燃料经济性方面平均可提升19%。

For three energy sources hybrid electric vehicles equipped with fuel cell,lithium battery and ultracapacitor,a hierarchical energy management strategy based on the improved deep deterministic policy gradient(DDPG)algorithm was proposed to reduce hydrogen consumption,improve fuel cell efficiency and maintain the state of charge(SoC)of lithium battery.First,an adaptive low-pass filter based on fuzzy rules was adopted to layer the power,and the peak power was assumed by the ultracapacitor.Secondly,an energy management framework based on DDPG was designed,and the optimization function was constructed by using the calculation idea of the equivalent consumption minimization strategy.A penalty factor related to fuel cell efficiency and lithium battery SoC deviation was added to optimize the power distribution of fuel cells and lithium battery.In addition,to avoid the frequent occurrence of extreme action values caused by noise exploration,the optimal solution of dynamic programming was used to assist strategy training to improve the optimization effect.Finally,the simulation was carried out under different driving conditions,and a test platform was built for verification.The results show that,compared with the traditional DDPG-based strategy,the proposed strategy can effectively reduce the consumption of lithium battery SoC,better ensure that the fuel cell works in the high-efficiency range,and significantly reduce hydrogen consumption,with an average increase of 19%in fuel economy.