分时电价下蓄冷空调系统需求响应强化学习控制

Demand Response Reinforcement Learning Control of Thermal Energy Storage Air-conditioning System under Time-of-use Pricing

电力需求的快速增长给电力供需平衡带来了很大的挑战。需求响应是1种削减峰值负荷、增强电网稳定性的有效手段。暖通空调系统的集中可控性和建筑的热惯性使其成为高效的需求响应参与者。由于建筑热力学的复杂性和外界环境的干扰,传统基于模型的方法在空调需求响应调控中往往受限。而强化学习是1种无模型、自适应的连续控制方法。本文将强化学习应用到具备储能罐的变风量空调系统中。搭建TRNSYS+MATLAB联合仿真平台,并在此基础上实现强化学习算法。该算法充分考虑了环境因素、分时电价和能耗等因素,以神经网络构建算法策略框架输出离散控制动作,实时学习室内温度设定点。选取3周的夏季高温工作日,运行强化学习控制策略仿真研究。仿真结果表明,所提的强化学习算法能够实现温度设定点控制,相对于固定设定点非蓄冷空调系统,可节约运行费用9.17%。

The rapid growth of electricity demand has brought great challenges to the balance of power supply and demand.Demand response(DR) can alleviate the peak load of the grid and enhances the stability of the grid.The centralized controllability of HVAC(Heating,Ventilation,Air-conditioning) system and the inertia of the building make it an efficient DR participant.Due to the complexity of building thermodynamics and interference of the external environment,traditional model-based methods are not efficient in practice.Reinforcement learning(RL) is a modeless,adaptive continuous control method.RL was applied to the control of Variable-Air volume air-conditioning system equipped with energy-storage tank.The RL algorithm was implemented on the established TRNSYS+MATLAB combined simulation platform.The algorithm fully took into account factors such as environmental factors,time-of-use pricing,and energy consumption,with policy frame constructed by neural network output discrete control action,and made real-time control of indoor temperature set points.Fifteen summer high temperature working days were selected to run the RL control method.The simulation results showed that the RL algorithm can realized indoor temperature set point control,which can save 9.17% of operating costs compared to fixed set point non-accumulation air-conditioning systems.