知识与深度强化学习融合的家庭需求响应优化

Fusing Knowledge and Deep Reinforcement Learning for Home Demand Response Optimization

家庭能源系统需求响应(demand response,DR)能够有效促进节能减排。家庭能源系统具有强不确定性,要求DR优化策略具备自动、快速调适能力。该文研究知识与强化学习融合的DR优化方法。首先,建立两者融合的优化框架,形成知识与强化学习互补机制;然后,对家庭设备建模并建立其DR优化知识规则集;进而,以知识转化为学习样本为核心,设计了知识融入强化学习的模型,重点研究知识样本作用概率动态调节、多样化知识、随机探索概率动态调节等问题,并设计知识与DQN融合的算法和网络。算例表明:该文方法能够自动适应家庭的不确定性,用能成本比知识规则法降低11.1%;与标准DQN相比,在同等的收敛标准下,该文方法能源成本低3.3%,且收敛时间仅需标准DQN的1/6。

Demand response(DR)for the home energy system(HES)can effectively promote energy conservation and emission reduction.Uncertainties of HESs require that the DR optimization strategy should be adjusted automatically and quickly.This paper investigates knowledge integration in reinforcement learning for DR optimization.First,an optimization framework of knowledge and reinforcement learning is established to form a complementary mechanism.Then,the home device models and their rule-based DR optimization knowledge set are established.Furthermore,based on the core of knowledge conversion into learning samples,a model of knowledge integration in reinforcement learning is designed.The dynamic adjustment of knowledge-guided action sampling probability,diversified knowledge,random exploration probability has been emphatically studied.The network and DQN algorithm with knowledge integration are designed.The case studies demonstrate that the proposed method can automatically adapt to uncertainties in the HES,and the energy cost is 11.1%lower than the knowledge rule-based method;compared with the standard DQN,under the same convergence criterion,the proposed method has a 3.3%lower energy cost and the convergence time is only 1/6 of the standard DQN.