基于深度强化学习的分布式电采暖参与需求响应优化调度

Demand Response Optimal Scheduling for Distributed Electric Heating Based on Deep Reinforcement Learning

分布式电采暖具备可时移特性,能够作为需求响应资源,但其数量多、单体容量小,调度中心难以直接控制,且传统优化方法难以满足调度时效性。应用深度学习实现了无需热力学模型分析户用电采暖单元温变–功率动态关系,构建了包含负荷聚集商和楼宇级控制的调度架构。提出改进的深度确定性策略梯度(deepdeterministicpolicygradient,DDPG)算法作为楼宇级控制策略,构建了改进算法的框架及网络结构,网络训练收敛后可用于在线决策控制。日前调度场景下改进算法在线应用耗时仅耗时0.35s,且在多维度输入场景收敛能力更优。深度学习描述电采暖单元温变–功率关系的有效性通过仿真实验进行了证实。仿真结果表明所提架构相比现行模式更易通过实时电价动态增加/降低引导电采暖负荷减少/增多,提高了电采暖负荷侧需求响应能力;同时使等效负荷标准差由65.6kW降低到37.3kW,减小聚合负荷峰谷差;在保障用户热舒适前提下,用户费用由1031.4元降低到936.1元,减少了用户成本,实现了调度户用电采暖参与需求响应的有效性和经济性。

Distributed electric heating,being time-shifting,can be used as a demand response resource,but it is large in number and small in unit capacity,which makes it difficult for the dispatch center to control directly,and for the traditional optimization method to achieve the effectiveness.Applying deep learning makes it possible to analyze the temperature-power dynamic relationship of electric heating units without the need of a thermodynamic model,then a dispatching architecture including a load aggregator and building-level control is constructed.An improved Deep Deterministic Policy Gradient algorithm is proposed as a building-level control strategy to construct the framework and network structure of the improved algorithm,which can be used for online decision control after the network training convergence.The online application of the improved algorithm in the day-to-day scheduling scenario only takes 0.35 seconds and it has better convergence ability in multi-dimensional input scenarios.The effectiveness of deep learning to describe the temperature-power relationship of electric heating units is verified by simulation experiments.The simulation results show that compared with the current model,the proposed architecture can guide the reduction/increase of electric heating load through real-time electricity price dynamic increase/decrease,which improves the demand response ability of the electric heating load side.It lowers the standard deviation of the equivalent load from 65.6 kW to 37.3 kW,reducing the peak-to-valley difference in aggregation load.Under the premise of ensuring thermal comfort for users,the cost of each user lowers from 1031.4 yuan to 936.1 yuan.The effectiveness and economy of dispatching household electric heating to participate in demand response are realized.