Combined peakreductionandself-consumptionusingproximalpolicy optimisation

Residential demand response programs aim to activate demand flexibility at the household level.In recent years,reinforcement learning(RL)has gained significant attention for these type of applications.A major challenge of RL algorithms is data efficiency.New RL algorithms,such as proximal policy optimisation(PPO),have tried to increase data efficiency.Addi tionally,combining RL with transfer learning has been proposed in an effort to mitigate this challenge.In this work,we further improve upon state-of-the-art transfer learning performance by incorporating demand response domain knowledge into the learning pipeline.We evaluate our approach on a demand response use case where peak shaving and self-consumption is incentivised by means of a capacity tariff.We show our adapted version of PPO,combined with transfer learming,reduces cost by 14.51%compared to a regular hysteresis controller and by 6.68%compared to traditional PPO.

Transfer learning in demand response: A review of algorithms for data-efficient modelling and control

A number of decarbonization scenarios for the energy sector are built on simultaneous electrification of energy demand,and decarbonization of electricity generation through renewable energy sources.However,increased electricity demand due to heat and transport electrification and the variability associated with renewables have the potential to disrupt stable electric grid operation.To address these issues using demand response,researchers and practitioners have increasingly turned towards automated decision support tools which utilize machine learning and optimization algorithms.However,when applied naively,these algorithms suffer from high sample complexity,which means that it is often impractical to fit sufficiently complex models because of a lack of observed data.Recent advances have shown that techniques such as transfer learning can address this problem and improve their performance considerably—both in supervised and reinforcement learning contexts.Such formulations allow models to leverage existing domain knowledge and human expertise in addition to sparse observational data.More formally,transfer learning embodies all techniques where one aims to increase(learning)performance in a target domain or task,by using knowledge gained in a source domain or task.This paper provides a detailed overview of state-of-the-art techniques on applying transfer learning in demand response,showing improvements that can exceed 30%in a variety of tasks.We observe that most research to date has focused on transfer learning in the context of electricity demand prediction,although reinforcement learning based controllers have also seen increasing attention.However,a number of limitations remain in these studies,including a lack of benchmarks,systematic performance improvement tracking,and consensus on techniques that can help avoid negative transfer.