基于专家示范深度强化学习的光伏系统MPPT控制

MPPT control of photovoltaic system based on expert demonstration of deep reinforcement learning

提出一种基于专家示范的深度确定性策略梯度算法(ED-DDPG)的MPPT算法,该算法将最大功率点跟踪问题转化为马尔科夫决策过程(MDP),选择基于连续动作空间的深度确定性策略梯度算法(DDPG)训练MPPT控制算法,来提高输出电压的控制精度,并基于专家示范改进了DDPG算法,使其在强化学习算法的经验池中预先加入传统方法的经验,从而加快算法神经网络训练的收敛速度。该算法解决了传统MPPT算法在面对温度、光照不断变化的复杂环境下动态效率较低的问题,解决了深度确定性策略梯度算法(DDPG)在求解MPPT问题时训练时间过长、收敛难度较大的问题。算例表明该算法在EN50530标准下MPPT动态效率平均达到97.3%,具有较强的鲁棒性。

A MPPT algorithm based on expert demonstration of ED-DDPG was proposed.The algorithm transformed the maximum power point tracking problem into Markov decision process(MDP).The deep deterministic policy gradient algorithm(DDPG)based on continuous action space was selected to train the MPPT control algorithm to improve the control accuracy of output voltage.The DDPG algorithm was improved based on expert demonstration.The experience of traditional methods was added to the experience pool of reinforcement learning algorithm in advance,so as to accelerate the convergence speed of algorithm neural network training.This algorithm solved the problem of low dynamic efficiency of traditional MPPT algorithm in the face of complex environment with changing temperature and irradiance,and solved the problem of long training time and difficult convergence of deep deterministic policy gradient algorithm(DDPG)in solving MPPT problems.The example shows that the MPPT dynamic efficiency of this algorithm reaches 97.3%on average under EN50530 standard,and it has strong robustness.