基于深度Q学习的含用户侧储能微电网频率-电压数字化智能控制策略

Frequency-Voltage Digital Intelligent Control Strategy of Microgrid with User-Side Energy Storage Based on Deep Q-Learning

频率与电压是衡量电能指标的重要标准。针对微电网受到负荷波动而引起的频率/电压调控问题,提出基于深度Q学习(deep Q-learning,DQN)的含用户侧储能微电网智能监控-控制策略。首先,通过考虑用户行为的随机性,增加了用户侧储能输出的随机约束,并引入四象限充放电的模型,构建用户侧储能的集群充放电模型,从而搭建出微电网频率-电压的协同控制模型。其次,设计基于DQN的频率/电压控制器结构与数字化智能控制平台,以系统实时的频率偏差、电压偏差与用户侧储能输出功率的上、下限约束为状态空间,以系统各机组出力为动作空间,并基于频率及电压2个控制目标,完成包含2个本地奖励的全局奖励函数的设计。算例结果表明:与传统PID控制器相比,所提DQN控制器能同时满足频率与电压的控制需求,更有效地应对负荷波动所引起的电能质量问题。

Frequency and voltage are important criteria for measuring energy indicators.Considering the problem of frequency/voltage regulation caused by load fluctuations in microgrids,a smart monitoring-control strategy for microgrids with userside energy storage based on deep Q-learning(DQN)is proposed in this paper.Firstly,in view of the randomness of user behavior,the random constraints on the output of energy storage on the user side are added,and a four-quadrant charging and discharging model is introduced to build a charging and discharging model for user-side energy storage.On this basis,a cooperative control model of frequency and voltage in the microgrid is built.Secondly,the frequency/voltage controller structure and digital intelligent control platform based on DQN are designed.With the real-time frequency deviation and voltage deviation of the system and the upper and lower limit constraints of the output power of user-side energy storage as the state space and the output of each unit in the system as the action space,a global reward function including two local rewards is designed on the basis of the two control objectives of frequency and voltage.The results of the calculation examples indicate that compared with the traditional PID controller,the DQN controller proposed in this paper can simultaneously meet the control requirements of frequency and voltage and can more effectively deal with the power quality problems caused by load fluctuations.