Energy Management of Price-maker Community Energy Storage by Stochastic Dynamic Programming

In this paper,we propose an analytical stochastic dynamic programming(SDP)algorithm to address the optimal management problem of price-maker community energy storage.As a price-maker,energy storage smooths price differences,thus decreasing energy arbitrage value.However,this price-smoothing effect can result in significant external welfare changes by reduc-ing consumer costs and producer revenues,which is not negligible for the community with energy storage systems.As such,we formulate community storage management as an SDP that aims to maximize both energy arbitrage and community welfare.To incorporate market interaction into the SDP format,we propose a framework that derives partial but sufficient market information to approximate impact of storage operations on market prices.Then we present an analytical SDP algorithm that does not require state discretization.Apart from computational efficiency,another advantage of the analytical algorithm is to guide energy storage to charge/discharge by directly comparing its current marginal value with expected future marginal value.Case studies indicate community-owned energy storage that maximizes both arbitrage and welfare value gains more benefits than storage that maximizes only arbitrage.The proposed algorithm ensures optimality and largely reduces the computational complexity of the standard SDP.Index Terms-Analytical stochastic dynamic programming,energy management,energy storage,price-maker,social welfare.

Experimental Study of Methods of Scenario Lattice Construction for Stochastic Dual Dynamic Programming

The stochastic dual dynamic programming (SDDP) algorithm is becoming increasingly used. In this paper we present analysis of different methods of lattice construction for SDDP exemplifying a realistic variant of the newsvendor problem, incorporating storage of production. We model several days of work and compare the profits realized using different methods of the lattice construction and the corresponding computer time spent in lattice construction. Our case differs from the known one because we consider not only a multidimensional but also a multistage case with stage dependence. We construct scenario lattice for different Markov processes which play a crucial role in stochastic modeling. The novelty of our work is comparing different methods of scenario lattice construction. We considered a realistic variant of the newsvendor problem. The results presented in this article show that the Voronoi method slightly outperforms others, but the k-means method is much faster overall.

A NEW STOCHASTIC OPTIMAL CONTROL STRATEGY FOR HYSTERETIC MR DAMPERS

A new stochastic optimal control strategy for randomly excited quasi-integrable Hamiltonian systems using magneto-rheological (MR) dampers is proposed. The dynamic be- havior of an MR damper is characterized by the Bouc-Wen hysteretic model. The control force produced by the MR damper is separated into a passive part incorporated in the uncontrolled system and a semi-active part to be determined. The system combining the Bouc-Wen hysteretic force is converted into an equivalent non-hysteretic nonlinear stochastic control system. Then It?o stochastic di?erential equations are derived from the equivalent system by using the stochastic averaging method. A dynamical programming equation for the controlled di?usion processes is established based on the stochastic dynamical programming principle. The non-clipping nonlin- ear optimal control law is obtained for a certain performance index by minimizing the dynamical programming equation. Finally, an example is given to illustrate the application and e?ectiveness of the proposed control strategy.

Stochastic optimal control of cable vibration in plane by using axial support motion

A stochastic optimal control strategy for a slightly sagged cable using support motion in the cable axial direction is proposed. The nonlinear equation of cable motion in plane is derived and reduced to the equations for the first two modes of cable vibration by using the Galerkin method. The partially averaged Ito equation for controlled system energy is further derived by applying the stochastic averaging method for quasi-non-integrable Hamiltonian systems. The dynamical programming equation for the controlled system energy with a performance index is established by applying the stochastic dynamical programming principle and a stochastic optimal control law is obtained through solving the dynamical programming equation. A bilinear controller by using the direct method of Lyapunov is introduced. The comparison between the two controllers shows that the proposed stochastic optimal control strategy is superior to the bilinear control strategy in terms of higher control effectiveness and efficiency.

Two-Stage Stochastic Dual Dynamic Programming for Transmission Expansion Planning with Significant Renewable Generation and N-k Criterion

The large-scale integration of renewable energy sources(RES)is the global trend to deal with the energy crisis and greenhouse emissions.Due to the intermittent nature of RES together with the uncertainty of load demand,the problem of transmission expansion planning(TEP)is facing more and more challenges from uncertainties.In this paper,the TEP problem is modeled as a two-stage formulation,so as to minimize the total of investment costs and generation costs.To ensure the utilization level of the RES generation,the expansion plan is required to provide sufficient transmission capacity for the integration of RES.Also,N-k security criterion is considered into the model,so the expansion plan can meet the required security criteria.The stochastic dual dynamic programming(SDDP)approach is applied to consider the uncertainties,and the whole model is solved by Benders’decomposition technique.Two case studies are carried out to compare the performance of the SDDP approach and the deterministic approach.Results show that the expansion plan obtained by the SDDP approach has a better performance than that of the deterministic approach.

Loading-Balance Relay-Selective Strategy Based on Stochastic Dynamic Program

A Relay-Assisted （RA） network with relay selection is considered as a type of effective technology to improve the spectrum and energy efficiency of a cellular network. However, loading balance of the assisted relay node becomes an inevitable bottleneck in RA network development because users do not follow uniform distribution. Furthermore, the time-varying channel condition of wireless communication is also a major challenge for the RA network with relay selection. To solve these problems and improve the practicability of the RA network, a Loading Balance-Relay Selective （LBRS） strategy is proposed for the RA network in this paper. The proposed LBRS strategy formulates the relay selection of the RA network under imperfect channel state information assumption as a Multistage Decision （MD） problem. An optimal algorithm is also investigated to solve the proposed MD problem based on stochastic dynamic program. Numerical results show that the performance of the LBRS strategy is better than that of traditional greedy algorithm and the former is effective as an exhaustive search-based method.

Optimal bounded control for maximizing reliability of Duhem hysteretic systems

The optimal bounded control of stochastic-excited systems with Duhem hysteretic components for maximizing system reliability is investigated. The Duhem hysteretic force is transformed to energy-depending damping and stiffness by the energy dissipation balance technique. The controlled system is transformed to the equivalent non- hysteretic system. Stochastic averaging is then implemented to obtain the It5 stochastic equation associated with the total energy of the vibrating system, appropriate for eval- uating system responses. Dynamical programming equations for maximizing system re- liability are formulated by the dynamical programming principle. The optimal bounded control is derived from the maximization condition in the dynamical programming equation. Finally, the conditional reliability function and mean time of first-passage failure of the optimal Duhem systems are numerically solved from the Kolmogorov equations. The proposed procedure is illustrated with a representative example.

Stochastic Control for Optimal Execution:Fast Approximation Solution Scheme Under Nested Mean-semi Deviation and Conditional Value at Risk

When executing a large order of stocks in a market,one important factor in forming the optimal trading strategy is to consider the price impact of large-volume trading activity.Minimizing a risk measure of the implementation shortfall,i.e.,the difference between the value of a trader’s initial equity position and the sum of cash flow he receives from his trading process,is essentially a stochastic control problem.In this study,we investigate such a practical problem under a dynamic coherent risk measure in a market in which the stock price dynamics has a feature of momentum effect.We develop a fast approximation solution scheme,which is critical in highfrequency trading.We demonstrate some prominent features of our derived solution algorithm in providing useful guidance for real implementation.

Stochastic Differential Games with Reflection and Related Obstacle Problems for Isaacs Equations

In this paper we first investigate zero-sum two-player stochastic differential games with reflection, with the help of theory of Reflected Backward Stochastic Differential Equations （RBSDEs）. We will establish the dynamic programming principle for the upper and the lower value functions of this kind of stochastic differential games with reflection in a straightforward way. Then the upper and the lower value functions are proved to be the unique viscosity solutions to the associated upper and the lower Hamilton-Jacobi-Bettman-Isaacs equations with obstacles, respectively. The method differs significantly from those used for control problems with reflection, with new techniques developed of interest on its own. Further, we also prove a new estimate for RBSDEs being sharper than that in the paper of E1 Karoui, Kapoudjian, Pardoux, Peng and Quenez （1997）, which turns out to be very useful because it allows us to estimate the LP-distance of the solutions of two different RBSDEs by the p-th power of the distance of the initial values of the driving forward equations. We also show that the unique viscosity solution to the approximating Isaacs equation constructed by the penalization method converges to the viscosity solution of the Isaacs equation with obstacle.

Risk-Averse Two-Stage Distributionally Robust Economic Dispatch Model Under Uncertain Renewable Energy

With the participation of large quantities of renewable energy in power system operations,their volatility and intermittence increases the difficulties and challenges of power system economic scheduling.Considering the uncertainty of renewable energy generation,based on the distributionally robust optimization method,a two-stage economic dispatch model is proposed to minimize the total operation costs.In this paper,it is assumed that the fluctuating of renewable power generation follows the unknown probability distribution that is restricted in an ambiguity set,which is established by utilizing the first-order moment information of available historical data.Furthermore,the theory of conditional value-at-risk is introduced to transform the model into a tractable model,which we call robust counterpart formulation.Based on the stochastic dual dynamic programming method,an improved iterative algorithm is proposed to solve the robust counterpart problem.Specifically,the convergence optimum can be obtained by the improved iterative algorithm,which performs a forward pass and backward pass repeatedly in each iterative process.Finally,by comparing with other methods,the results on the modified IEEE 6-bus,118-bus,and 300-bus system show the effectiveness and advantages of the proposed model and method.

Evaluation of optimization operation models for cascaded hydropower reservoirs to utilize medium range forecasting inflow

This paper evaluates the performances of the models that incorporate forecasting inflow for cascaded hydropower reservoirs operation. These models are constructed separately on the concepts of explicit stochastic optimization （ESO） and implicit sto- chastic optimization （ISO） as well as parametefization-simulation-optimization （PSO）. Firstly, the aggregation-disaggregation method is implemented in ESO models to reduce the complexity of stochastic dynamic programming （SDP）. And the aggre- gate flow SDP （AF-SDP） and aggregation-disaggregation SDP （AD-SDP） are constructed respectively. Secondly, in ISO mod- el, decision tree is the well-known and widespread algorithm. The algorithm C 5.0 is selected to extract the if-then-else rules for reservoir operation. Thirdly, based on the PSO model, the hedging rule curves （HRCs） are pre-defined by fusing the storage and inflow as state variable. The parameters of the HRCs are determined by using the simulation-optimization model. Finally, China＇s Hun River cascade hydropower reservoirs system is taken as an example to illustrate the efficiency and reliability of the models. In addition, the values of quantitative precipitation forecasts of the global forecast system （10 days lead-time） are implemented to forecast the 10 days inflow.

TWO-STAGE ORDERING DECISION FOR A SHORT-LIFE-CYCLE PRODUCT

This paper investigates the ordering decision problem for a short-life-cycle product under Bayesian updating. For a product characterized by a single manufacturing cycle and two selling periods, we depict a Two-Stage （TS） ordering strategy with a stochastic dynamic programming model in the view of the whole system, and prove that the expected profit function of the whole system is concave on the first ordering quantity and the remedial ordering quantity, respectively. Then, the optimal ordering decision is developed. Finally, characteristics of the optimal ordering quantities are analyzed with several examples. Our results show that the suggested TS decision model is better than a Quick Response （QR） decision model.

Explicit Solution for Constrained Optimal Execution Problem with General Correlated Market Depth

This work studies the constrained optimal execution problem with a random market depth in the limit order market.Motivated from the real trading activities,our execution model considers the execution bounds and allows the random market depth to be statistically correlated in different periods.Usually,it is difficult to achieve the analytical solution for this class of constrained dynamic decision problem.Thanks to the special structure of this model,by applying the proposed state separation theorem and dynamic programming,we successfully obtain the analytical execution policy.The revealed policy is of feedback nature.Examples are provided to illustrate our solution methods.Simulation results demonstrate the advantages of our model comparing with the classical execution policy.

The Jointed Decision Model on a Newsboy-type Ordering and Pricing for Retailer

This paper deals with the jointed decision question on ordering and pricing for a short-life-cycle product under stochastic multiplicative demand depended selling price. According to the marketing practices, which retailers sell their products in different periods with the different marketing policies, we depict the jointed decision question with a stochastic dynamic programming model from the view of the centralized system. Then, we prove that the expected profit function are concave on decision vectors respectively, and develop the decision method for ordering and pricing. Lastly, we design the iterative search arithmetic to find the optimal decision vectors.

Optimal Control of Unknown Discrete-Time Linear Systems with Additive Noise

The optimal control problem with a long run average cost is investigated for unknown linear discrete-time systems with additive noise.The authors propose a value iteration-based stochastic adaptive dynamic programming(VI-based SADP)algorithm,based on which the optimal controller is obtained.Different from the existing relevant work,the algorithm does not need to estimate the expectation(conditional expectation)and variance(conditional variance)of states or other relevant variables,and the convergence of the algorithm can be proved rigorously.A simulation example is given to verify the effectiveness of the proposed approach.