基于深度强化学习的逆变器多频点控制参数优化

Optimization of Inverter Multi-Frequency Control Parameters Based on Deep Reinforcement Learning

传统逆变器闭环控制具有良好的静态和动态性能,但非常依赖精确的系统数学模型,难以适应逆变器接入不同负载或电网环境等原因带来的模型参数扰动。将该深度强化学习应用于逆变器多频点控制参数整定过程。文章首先建立了单相电压型逆变器的控制模型;在此基础上分析了逆变器参数调节过程中的不稳定特征,并设计了一种基于FFT的不稳定特征判断方法,实现逆变器参数调节过程中稳定状态的在线监测;其次对逆变器控制参数自适应过程进行马尔可夫过程建模,设计了智能体的状态、动作和奖励函数;针对智能体样本不平衡问题引入了经验优先级回放以及动作屏蔽机制提高智能体的学习效率;经过仿真学习训练,智能体实现比例谐振控制器参数自整定以获取最佳的多频点跟踪性能;最后在搭建的实验平台上进行实验,结果表明:训练后的智能体可以在较少次数训练后获得满足各频点控制精度要求的控制参数,同时整个训练过程系统都是稳定的。

Traditional inverter closed-loop control has good static and dynamic performance,but it is heavily dependent on the precise mathematical model of the system,and it is difficult to adapt to the model parameter disturbance caused by the inverter connected to different loads or power grid environment.In this paper,deep reinforcement learning is applied to the multi-frequency control parameter setting process of the inverter.Firstly,a method of approximate calculation of resonance coefficient based on steady-state error index is proposed.Based on this method,the resonance coefficient values under different frequency points and different load requirements are analyzed,which has guiding significance for the parameter design of resonance coefficient.Secondly,the stability characteristic judgment method based on FFT is designed to realize the on-line monitoring of the output stability state of the inverter.On this basis,Markov process modeling is carried out for the adaptive process of inverter control parameters,and the state,action and reward functions of the agent are designed.To address unbalanced agent samples,the experience priority playback and action shielding mechanism are introduced to improve the learning efficiency of the agent.After simulation learning training,the agent realizes the parameter self-tuning of the proportional resonance controller to obtain the best multifrequency tracking performance.Finally,experiments are carried out on the experimental platform,and the results show that the trained agent can obtain control parameters that meet the control accuracy requirements of each frequency after less training,and the whole training process system is stable.