Day-ahead scheduling based on reinforcement learning with hybrid action space

Driven by the improvement of the smart grid,the active distribution network(ADN)has attracted much attention due to its characteristic of active management.By making full use of electricity price signals for optimal scheduling,the total cost of the ADN can be reduced.However,the optimal dayahead scheduling problem is challenging since the future electricity price is unknown.Moreover,in ADN,some schedulable variables are continuous while some schedulable variables are discrete,which increases the difficulty of determining the optimal scheduling scheme.In this paper,the day-ahead scheduling problem of the ADN is formulated as a Markov decision process(MDP)with continuous-discrete hybrid action space.Then,an algorithm based on multi-agent hybrid reinforcement learning(HRL)is proposed to obtain the optimal scheduling scheme.The proposed algorithm adopts the structure of centralized training and decentralized execution,and different methods are applied to determine the selection policy of continuous scheduling variables and discrete scheduling variables.The simulation experiment results demonstrate the effectiveness of the algorithm.

Robust coordination of interdependent electricity and natural gas systems in day-ahead scheduling for facilitating volatile renewable generations via power-to-gas technology

The increasing interdependency of electricity and natural gas systems promotes coordination of the two systems for ensuring operational security and economics.This paper proposes a robust day-ahead scheduling model for the optimal coordinated operation of integrated energy systems while considering key uncertainties of the power system and natural gas system operation cost. Energy hub,with collocated gas-fired units, power-to-gas(Pt G) facilities, and natural gas storages, is considered to store or convert one type of energy(i.e., electricity or natural gas)into the other form, which could analogously function as large-scale electrical energy storages. The column-andconstraint generation(C&CG) is adopted to solve the proposed integrated robust model, in which nonlinear natural gas network constraints are reformulated via a set of linear constraints. Numerical experiments signify the effectiveness of the proposed model for handling volatile electrical loads and renewable generations via the coordinated scheduling of electricity and natural gas systems.

Day-ahead Scheduling of Multi-carrier Energy Systems with Multi-type Energy Storages and Wind Power

The integration of large-scale wind power brings challenges to the operation of integrated energy systems(IES).In this paper,a day-ahead scheduling model for IES with wind power and multi-type energy storage is proposed in a scenario-based stochastic programming framework.The structure of the IES consists of electricity,natural gas,and heating networks which are all included in the model.Operational constraints for IES incorporating multi-type energy storage devices are also considered.The constraints of the electricity network,natural gas network and heating network are formulated,and non-linear constraints are linearized.The calculation method for the correlation of wind speed between wind farms based on historical data is proposed.Uncertainties of correlated wind power were represented by creating multiple representative scenarios with different probabilities,and this was done using the Latin hyper-cube sampling(LHS)method.The stochastic scheduling model is formulated as a mixed integer linear programming(MILP)problem with the objective function of minimizing the total expected operation cost.Numerical results on a modified PJM 5-bus electricity system with a seven-node natural gas system and a six-node heating system validate the proposed model.The results demonstrate that multi-type energy storage devices can help reduce wind power curtailments and improve the operational flexibility of IES.

An economic and low-carbon day-ahead Pareto-optimal scheduling for wind farm integrated power systems with demand response

Demand response(DR)and wind power are beneficial to low-carbon electricity to deal with energy and environmental problems.However,the uncertain wind power generation(WG)which has anti-peaking characteristic would be hard to exert its ability in carbon reduction.This paper introduces DR into traditional unit commitment(UC)strategy and proposes a multi-objective day-ahead optimal scheduling model for wind farm integrated power systems,since incentive-based DR can accommodate excess wind power and can be used as a source of system spinning reserve to alleviate generation side reserve pressure during both peak and valley load periods.Firstly,net load curve is obtained by forecasting load and wind power output.Then,considering the behavior of DR,a day-ahead optimal dispatching scheme is proposed with objectives of minimum generating cost and carbon emission.Non-dominated sorting genetic algorithm-II(NSGA-II)and satisfaction-maximizing method are adopted to solve the multi-objective model with Pareto fronts and eclectic decision obtained.Finally,a case study is carried out to demonstrate that the approach can achieve economic and environmental aims and DR can help to accommodate the wind power.

Day-ahead scheduling of large numbers of thermostatically controlled loads based on equivalent energy storage model

Due to their heat/cool storage characteristics, thermostatically controlled loads(TCLs) play an important role in demand response programmers. However, the modeling of the heat/cool storage characteristic of large numbers of TCLs is not simple. In this paper, the heat exchange power is adopted to calculate the power instead of the average power, and the relationship between the heat exchange power and energy storage is considered to develop an equivalent storage model, based on which the time-varying power constraints and the energy storage constraints are developed to establish the overall day-ahead schedulingmodel. Finally, the proposed scheduling method is verified using the simulation results of a six-bus system.

Day-ahead optimal charging/discharging scheduling for electric vehicles in microgrids

Microgrid as an important part of smart grid comprises distributed generators(DGs),adjustable loads,energy storage systems(ESSs)and control units.It can be operated either connected with the external system or islanded with the support of ESSs.While the daily output of DGs strongly depends on the temporal distribution of natural resources such as wind and solar,unregulated electric vehicle(EV)charging demand will deteriorate the unbalance between the daily load curve and generation curve.In this paper,a statistic model is presented to describe daily EV charging/discharging behaviors considering the randomness of the initial state of charge(SOC)of EV batteries.The optimization problem is proposed to obtain the economic operation for the microgrid based on this model.In dayahead scheduling,with the estimated power generation and load demand,the optimal charging/discharging scheduling of EVs during 24 h is achieved by serial quadratic programming.With the optimal charging/discharging scheduling of EVs,the daily load curve can better track the generation curve.The network loss in grid-connected operation mode and required ESS capacity in islanded operation mode are both decreased.

Optimal Day-ahead Scheduling of Islanded Microgrid Considering Risk-based Reserve Decision

Due to the lack of support from the main grid,the intermittency of renewable energy sources(RESs)and the fluctuation of load will derive uncertainties to the operation of islanded microgrids(IMGs).It is crucial to allocate appropriate reserve capacity for the economic and reliable operation of IMGs.With the high penetration of RESs,it faces both economic and environmental challenges if we only use spinning reserve for reserve support.To solve these problems,a multi-type reserve scheme for IMGs is proposed according to different operation characteristics of generation,load,and storage.The operation risk due to reserve shortage is modeled by the conditional value-at-risk(CVaR)method.The correlation of input variables is considered for the forecasting error modeling of RES and load,and Latin hypercube sampling(LHS)is adopted to generate the random scenarios of the forecasting error,so as to avoid the dimension disaster caused by conventional large-scale scenario sampling approaches.Furthermore,an optimal day-ahead scheduling model of joint energy and reserve considering riskbased reserve decision is established to coordinate the security and economy of the operation of IMGs.Finally,the comparison of numerical results of different schemes demonstrate the rationality and effectiveness of the proposed scheme and model.

Day-ahead Optimization Schedule for Gas-electric Integrated Energy System Based on Second-order Cone Programming

This paper proposes an optimal day-ahead opti-mization schedule for gas-electric integrated energy system(IES)considering the bi-directional energy flow.The hourly topology of electric power system(EPS),natural gas system(NGS),energy hubs(EH)integrated power to gas(P2G)unit,are modeled to minimize the day-ahead operation cost of IES.Then,a second-order cone programming(SOCP)method is utilized to solve the optimization problem,which is actually a mixed integer nonconvex and nonlinear programming issue.Besides,cutting planes are added to ensure the exactness of the global optimal solution.Finally,simulation results demonstrate that the proposed optimization schedule can provide a safe,effective and economical day-ahead scheduling scheme for gas-electric IES.

Strategy of constructing virtual peaking unit by public buildings’central air conditioning loads for day-ahead power dispatching

With the gradually widely usage of the air conditioning(AC) loads in developing countries, the urban power grid load has swiftly increased over the past decade.Especially in China, the AC load has accounted for over30% of the maximum load in many cities during summer.This paper proposes a scheme of constructing a virtual peaking unit(VPU) by public buildings’ cool storage central AC(CSCAC) systems and non-CSCAC(NCSCAC)systems for the day-ahead power network dispatching(DAPND). Considering the accumulation effect of different meteorological parameters, a short term load forecasting method of public building’s central AC(CAC) baseline load is firstly discussed. Then, a second-order equivalent thermal parameters model is established for the public building’s CAC load. Moreover, the novel load reduction control strategies for the public building’s CSCAC system and the public building’s NCSCAC system are respectively presented. Furthermore, based on the multiple-rank control strategy, the model of the DAPND with the participation of a VPU is set up. The VPU is composed of large-scale regulated public building’s CAC loads. To demonstrate the effectiveness of the proposed strategy, results of a sample study on a region in Nanjing which involves 22 public buildings’ CAC loads are described in this paper. Simulated results show that, by adopting the proposed DAPND scheme, the power network peak load in the region obviously decreases with a small enough deviation between the regulated load value and the dispatching instruction of the VPU. The total electricity-saving amount accounts for7.78% of total electricity consumption of the VPU before regulation.

Storing Freshwater Versus Storing Electricity in Power Systems with High Freshwater Electric Demand

We consider a power system whose electric demand pertaining to freshwater production is high(high freshwater electric demand),as in the Middle East,and investigate the tradeoff of storing freshwater in tanks versus storing electricity in batteries at the day-ahead operation stage.Both storing freshwater and storing electricity increase the actual electric demand at valley hours and decrease it at peak hours,which is generally beneficial in term of cost and reliability.But,to what extent?We analyze this question considering three power systems with different generation-mix configurations,i.e.,a thermal-dominated mix,a renewable-dominated one,and a fully renewable one.These generation-mix configurations are inspired by how power systems may evolve in different countries in the Middle East.Renewable production uncertainty is compactly modeled using chance constraints.We draw conclusions on how both storage facilities(freshwater and electricity)complement each other to render an optimal operation of the power system.

Hybrid-integer algorithm for a multi-objective optimal home energy management system

Most of the energy produced in the world is consumed by commercial and residential buildings.With the growth in the global economy and world demographics,this energy demand has become increasingly important.This has led to higher unit electricity prices,frequent stresses on the main electricity grid and carbon emissions due to inefficient energy management.This paper presents an energy-consumption management system based on time-shifting of loads according to the dynamic day-ahead electricity pricing.This simultaneously reduces the electricity bill and the peaks,while maintaining user comfort in terms of the operating waiting time of appliances.The proposed optimization problem is formulated mathematically in terms of multi-objective integer non-linear programming,which involves constraints and consumer preferences.For optimal scheduling,the management problem is solved using the hybridization of the particle swarm optimization algorithm and the branch-and-bound algorithm.Two techniques are proposed to manage the trade-off between the conflicting objectives.The first technique is the Pareto-optimal solutions classification using supervised learning methods.The second technique is called the lexicographic method.The simulations were performed based on residential building energy consumption,time-of-use pricing(TOU)and critical peak pricing(CPP).The algorithms were implemented in Python.The results of the current work show that the proposed approach is effective and can reduce the electricity bill and the peak-to-average ratio(PAR)by 28% and 49.32%,respectively,for the TOU tariff rate,and 48.91% and 47.87% for the CPP tariff rate by taking into account the consumer’s comfort level.