计及稳压率和经济性的城轨直流牵引供电光储系统深度Q网络优化控制方法

Deep Q network optimization control method for DC traction power supply photovoltaic-energy storage system of urban rail considering voltage stabilization rate and economy

光伏应用于直流牵引供电系统可提高新能源渗透率、降低系统能耗,但可再生能源出力的不确定性及列车负荷的强波动性增加了控制策略的寻优难度。针对该问题,提出一种基于深度强化学习的控制策略优化方法。该方法基于深度Q网络,将源-储-荷能量管理系统作为智能代理,通过光伏出力、储能荷电状态、牵引网压等外部状态训练代理,得到可实现系统经济可靠运行的优化策略。介绍源-储-荷综合系统的框架结构及传统控制策略,并对各设备进行外特性建模;对源-储-荷综合系统的能量管理问题开展马尔可夫决策过程建模,确立强化学习框架;根据某市域线路数据在MATLAB平台上进行仿真以验证所提方法的有效性。研究结果表明,所提方法通过动态调整储能电压阈值,可实现控制策略优化;通过与几种传统控制策略对比可知,所提方法在兼顾系统稳压水平与运行经济性方面占据优势;不同环境下的收敛效果对比体现了所提方法的可继承性,并在多组测试样本下验证了该方法的普适性。

The application of photovoltaic in DC traction power supply system can improve the penetration rate of new energy and reduce the energy consumption of the system,but the uncertainty of renewable energy output and the strong fluctuation of train load increase the difficulty of control strategy optimization.To solve this problem,a deep reinforcement learning-based control strategy optimization method is proposed.Based on the deep Q network(DQN),the source-energy storage-load energy management system is used as an intelli⁃gent agent,and the agent is trained by the external states such as the photovoltaic output,the state of charge of energy storage,the traction network voltage,and so on,so as to obtain an optimal strategy to realize the economic and reliable operation of the system.The framework structure and traditional control strategy of source-energy storage-load integrated system are introduced and the external characteristics of each device are modeled.Then,Markov decision process modeling is carried out for the energy management problem of source-energy storage-load integrated system and the reinforcement learning framework is established.The effectiveness of the proposed method is verified by simulation in MATLAB platform based on a municipal line data.The results show that the proposed method can optimize the control strategy by dynamically adjus-ting the voltage threshold of energy storage.Compared with several traditional control strategies,the proposed method has advantages in considering both the system voltage stability level and operation economy.The comparison of convergence effect in different environments shows the inheritability of the proposed method,and verifies the universality of the proposed method under multiple sets of test samples.