基于深度强化学习的燃料电池混合动力汽车能量管理策略研究

Research on Energy Management Strategy of Fuel Cell Hybrid Vehicles Based on Deep Reinforcement Learning

为提高燃料电池混合动力汽车的燃油经济性和燃料电池寿命,该文提出一种基于深度强化学习(Deep Reinforcement Learning,DRL)的能量管理策略。该策略首先在DRL奖励信号中加入寿命因子,通过降低燃料电池功率波动,起到延长燃料电池寿命的效果;其次,通过限制DRL的动作空间的方法,使燃料电池系统工作在高效率区间,从而提高整车效率。在UDDS、WLTC、Japan1015三个标准工况下进行了离线训练,并在NEDC工况下实时应用以验证所提出策略的工况适应性。仿真结果显示,在离线训练中,所提出的策略可以快速收敛,表明其具有较好的稳定性。在燃油经济性方面,与基于动态规划的策略相比,在3个训练工况下的差异仅为5.58%、3.03%和4.65%,接近最优燃油经济性;相比基于强化学习的策略,分别提升了4.46%、7.26%和5.35%。与无寿命因子的DRL策略相比,所提出的策略在3个训练工况下将燃料电池平均功率波动降低了10.27%、47.95%和10.85%,这有利于提升燃料电池寿命。在未知工况的实时应用中,所提出策略的燃油经济性比基于强化学习的策略提升了3.39%,这表明其工况适应性。

In order to improve the fuel economy and fuel cell lifetime of fuel cell hybrid vehicles,this research proposes an energy management strategy based on deep reinforcement learning(DRL).The strategy first adds a lifetime factor to reward signal of DRL,the lifetime of fuel cell is extended by limiting the power fluctuation.Then,the fuel cell system works in a high efficiency range by limiting the action space of DRL,improving the efficiency of the entire vehicle.After offline training under UDDS,WLTC,and Japan1015,it is applied in real time under NEDC to verify the adaptability of the proposed strategy.The results show that the proposed strategy can converge quickly in offline training,which proves its stability.Compared with dynamic programming-based strategy,the fuel economy difference in training cycles is only 5.58%,3.03%and 4.65%,which is close to the optimal,and the promotion is 4.46%,7.26%and 5.35%compared with reinforcement learning-based strategy.Compared with the DRL-based strategy without a lifetime factor,the proposed strategy reduces the average power fluctuation by 10.27%,47.95%,and 10.85%under training cycles,which is beneficial to improve the fuel cell lifetime.In the real-time application,the fuel economy of the proposed strategy is improved by 3.39%compared with the reinforcement learningbased strategy,which proves its adaptability to unknown cycles.