Building energy demand response is projected to be important in decarbonizing energy use. A demand responseprogram that communicates ‘‘artificial’’ hourly price signals to workers as part of a social game has the ...Building energy demand response is projected to be important in decarbonizing energy use. A demand responseprogram that communicates ‘‘artificial’’ hourly price signals to workers as part of a social game has the potentialto elicit energy consumption changes that simultaneously reduce energy costs and emissions. The efficacy ofsuch a program depends on the pricing agent’s ability to learn how workers respond to prices and mitigatethe risk of high energy costs during this learning process. We assess the value of deep reinforcement learning(RL) for mitigating this risk. Specifically, we explore the value of combining: (i) a model-free RL method thatcan learn by posting price signals to workers, (ii) a supervisory ‘‘planning model’’ that provides a syntheticlearning environment, and (iii) a guardrail method that determines whether a price should be posted to realworkers or the planning environment for feedback. In a simulated medium-sized office building, we compareour pricing agent against existing model-free and model-based deep RL agents, and the simpler strategy ofpassing on the time-of-use price signal to workers. We find that our controller eliminates 175,000 US Dollarsin initial investment, decreases by 30% the energy cost, and curbs emissions by 32% compared to energyconsumption under the time-of-use rate. In contrast, the model-free and model-based deep RL benchmarksare unable to overcome initial learning costs. Our results bode well for risk-aware deep RL facilitating thedeployment of building demand response.展开更多
文摘Building energy demand response is projected to be important in decarbonizing energy use. A demand responseprogram that communicates ‘‘artificial’’ hourly price signals to workers as part of a social game has the potentialto elicit energy consumption changes that simultaneously reduce energy costs and emissions. The efficacy ofsuch a program depends on the pricing agent’s ability to learn how workers respond to prices and mitigatethe risk of high energy costs during this learning process. We assess the value of deep reinforcement learning(RL) for mitigating this risk. Specifically, we explore the value of combining: (i) a model-free RL method thatcan learn by posting price signals to workers, (ii) a supervisory ‘‘planning model’’ that provides a syntheticlearning environment, and (iii) a guardrail method that determines whether a price should be posted to realworkers or the planning environment for feedback. In a simulated medium-sized office building, we compareour pricing agent against existing model-free and model-based deep RL agents, and the simpler strategy ofpassing on the time-of-use price signal to workers. We find that our controller eliminates 175,000 US Dollarsin initial investment, decreases by 30% the energy cost, and curbs emissions by 32% compared to energyconsumption under the time-of-use rate. In contrast, the model-free and model-based deep RL benchmarksare unable to overcome initial learning costs. Our results bode well for risk-aware deep RL facilitating thedeployment of building demand response.
