Stochastic programming based multi-arm bandit offloading strategy for internet of things

In order to solve the high latency of traditional cloud computing and the processing capacity limitation of Internet of Things(IoT)users,Multi-access Edge Computing(MEC)migrates computing and storage capabilities from the remote data center to the edge of network,providing users with computation services quickly and directly.In this paper,we investigate the impact of the randomness caused by the movement of the IoT user on decision-making for offloading,where the connection between the IoT user and the MEC servers is uncertain.This uncertainty would be the main obstacle to assign the task accurately.Consequently,if the assigned task cannot match well with the real connection time,a migration(connection time is not enough to process)would be caused.In order to address the impact of this uncertainty,we formulate the offloading decision as an optimization problem considering the transmission,computation and migration.With the help of Stochastic Programming(SP),we use the posteriori recourse to compensate for inaccurate predictions.Meanwhile,in heterogeneous networks,considering multiple candidate MEC servers could be selected simultaneously due to overlapping,we also introduce the Multi-Arm Bandit(MAB)theory for MEC selection.The extensive simulations validate the improvement and effectiveness of the proposed SP-based Multi-arm bandit Method(SMM)for offloading in terms of reward,cost,energy consumption and delay.The results showthat SMMcan achieve about 20%improvement compared with the traditional offloading method that does not consider the randomness,and it also outperforms the existing SP/MAB based method for offloading.

Foreign exchange trading and management with the stochastic dual dynamic programming method

We present a novel tool for generating speculative and hedging foreign exchange(FX)trading policies.Our solution provides a schedule that determines trades in each rebalancing period based on future currency prices,net foreign account positions,and incoming(outgoing)flows from business operations.To obtain such policies,we construct a multistage stochastic programming(MSP)model and solve it using the stochastic dual dynamic programming(SDDP)numerical method,which specializes in solving high-dimensional MSP models.We construct our methodology within an open-source SDDP package,avoiding implementing the method from scratch.To measure the performance of our policies,we model FX prices as a mean-reverting stochastic process with random events that incorporate stochastic trends.We calibrate this price model on seven currency pairs,demonstrating that our trading policies not only outperform the benchmarks for each currency,but may also be close to ex-post optimal solutions.We also show how the tool can be used to generate more or less conservative strategies by adjusting the risk tolerance,and how it can be used in a vari-ety of contexts and time scales,ranging from intraday speculative trading to monthly hedging for business operations.Finally,we examine the impact of increasing trade policy uncertainty(TPU)levels on our findings.Our findings show that the volatility of currencies from emerging economies rises in comparison to currencies from devel-oped markets.We discover that an increase in the TPU level has no effect on the aver-age profit obtained by our method.However,the risk exposure of the policies increases(decreases)for the group of currencies from emerging(developed)markets.

Improved Unit Commitment with Accurate Dynamic Scenarios Clustering Based on Multi-Parametric Programming and Benders Decomposition

Stochastic unit commitment is one of the most powerful methods to address uncertainty. However, the existingscenario clustering technique for stochastic unit commitment cannot accurately select representative scenarios,which threatens the robustness of stochastic unit commitment and hinders its application. This paper providesa stochastic unit commitment with dynamic scenario clustering based on multi-parametric programming andBenders decomposition. The stochastic unit commitment is solved via the Benders decomposition, which decouplesthe primal problem into the master problem and two types of subproblems. In the master problem, the committedgenerator is determined, while the feasibility and optimality of generator output are checked in these twosubproblems. Scenarios are dynamically clustered during the subproblem solution process through the multiparametric programming with respect to the solution of the master problem. In other words, multiple scenariosare clustered into several representative scenarios after the subproblem is solved, and the Benders cut obtainedby the representative scenario is generated for the master problem. Different from the conventional stochasticunit commitment, the proposed approach integrates scenario clustering into the Benders decomposition solutionprocess. Such a clustering approach could accurately cluster representative scenarios that have impacts on theunit commitment. The proposed method is tested on a 6-bus system and the modified IEEE 118-bus system.Numerical results illustrate the effectiveness of the proposed method in clustering scenarios. Compared withthe conventional clustering method, the proposed method can accurately select representative scenarios whilemitigating computational burden, thus guaranteeing the robustness of unit commitment.

A Stochastic Programming Strategy in Microgrid Cyber Physical Energy System for Energy Optimal Operation

This paper focuses on the energy optimal operation problem of microgrids(MGs) under stochastic environment.The deterministic method of MGs operation is often uneconomical because it fails to consider the high randomness of unconventional energy resources.Therefore,it is necessary to develop a novel operation approach combining the uncertainty in the physical world with modeling strategy in the cyber system.This paper proposes an energy scheduling optimization strategy based on stochastic programming model by considering the uncertainty in MGs.The goal is to minimize the expected operation cost of MGs.The uncertainties are modeled based on autoregressive moving average(ARMA) model to expose the effects of physical world on cyber world.Through the comparison of the simulation results with deterministic method,it is shown that the effectiveness and robustness of proposed stochastic energy scheduling optimization strategy for MGs are valid.

A simulation-based two-stage interval-stochastic programming model for water resources management in Kaidu-Konqi watershed,China

This study presented a simulation-based two-stage interval-stochastic programming （STIP） model to support water resources management in the Kaidu-Konqi watershed in Northwest China. The modeling system coupled a distributed hydrological model with an interval two-stage stochastic programing （ITSP）. The distributed hydrological model was used for establishing a rainfall-runoff forecast system, while random parameters were pro- vided by the statistical analysis of simulation outcomes water resources management planning in Kaidu-Konqi The developed STIP model was applied to a real case of watershed, where three scenarios with different water re- sources management policies were analyzed. The results indicated that water shortage mainly occurred in agri- culture, ecology and forestry sectors. In comparison, the water demand from municipality, industry and stock- breeding sectors can be satisfied due to their lower consumptions and higher economic values. Different policies for ecological water allocation can result in varied system benefits, and can help to identify desired water allocation plans with a maximum economic benefit and a minimum risk of system disruption under uncertainty.

Optimal Reservoir Operation Using Stochastic Dynamic Programming

This paper focused on the applying stochastic dynamic programming (SDP) to reservoir operation. Based on the two stages decision procedure, we built an operation model for reservoir operation to derive operating rules. With a case study of the China’s Three Gorges Reservoir, long-term operating rules are obtained. Based on the derived operating rules, the reservoir is simulated with the inflow from 1882 to 2005, which the mean hydropower generation is 85.71 billion kWh. It is shown that the SDP works well in the reservoir operation.

Chance-Constrained Approaches for Multiobjective Stochastic Linear Programming Problems

Multiple objective stochastic linear programming is a relevant topic. As a matter of fact, many practical problems ranging from portfolio selection to water resource management may be cast into this framework. Severe limitations on objectivity are encountered in this field because of the simultaneous presence of randomness and conflicting goals. In such a turbulent environment, the mainstay of rational choice cannot hold and it is virtually impossible to provide a truly scientific foundation for an optimal decision. In this paper, we resort to the bounded rationality principle to introduce satisfying solution for multiobjective stochastic linear programming problems. These solutions that are based on the chance-constrained paradigm are characterized under the assumption of normality of involved random variables. Ways for singling out such solutions are also discussed and a numerical example provided for the sake of illustration.

Approximation-Exact Penalty Function Method for Solving a Class of Stochastic Programming

We present an approximation-exact penalty function method for solving the single stage stochastic programming problem with continuous random variable. The original problem is transformed into a determinate nonlinear programming problem with a discrete random variable sequence, which is obtained by some discrete method. We construct an exact penalty function and obtain an unconstrained optimization. It avoids the difficulty in solution by the rapid growing of the number of constraints for discrete precision. Under lenient conditions, we prove the equivalence of the minimum solution of penalty function and the solution of the determinate programming, and prove that the solution sequences of the discrete problem converge to a solution to the original problem.

An Interactive Fuzzy Satisficing Method for Multiobjective Stochastic Integer Programming with Simple Recourse

This paper considers multiobjective integer programming problems involving random variables in constraints. Using the concept of simple recourse, the formulated multiobjective stochastic simple recourse problems are transformed into deterministic ones. For solving transformed deterministic problems efficiently, we also introduce genetic algorithms with double strings for nonlinear integer programming problems. Taking into account vagueness of judgments of the decision maker, an interactive fuzzy satisficing method is presented. In the proposed interactive method, after determineing the fuzzy goals of the decision maker, a satisficing solution for the decision maker is derived efficiently by updating the reference membership levels of the decision maker. An illustrative numerical example is provided to demonstrate the feasibility and efficiency of the proposed method.

Stochastic level-value approximation for quadratic integer convex programming

We propose a stochastic level value approximation method for a quadratic integer convex minimizing problem in this paper. This method applies an importance sampling technique, and make use of the cross-entropy method to update the sample density functions. We also prove the asymptotic convergence of this algorithm, and report some numerical results to illuminate its effectiveness.

Visual Programming of Stochastic Weather Generator and Future Applications on Agroecological Study

Based on former studies on weather simulator modules in IPMist laboratory, study on visual programming of stochastic weather generator(VS-WGEN)was continued. In this study, Markov Chain, Monte Carlo, Fourier Series, and weak stationary process were used to generate the daily weather data in software Matlab 6. 0, with the data input from 40 years' weather data recorded by Beijing Weather Station. The generated data includes daily maximum temperature, minimum temperature, precipitation and solar radiation. It has been verified that the weather data generated by the VS-WGEN were more accurate than that by the old WGEN, when twenty new model parameters were included. VS-WGEN has wide software applications, such as pest risk analysis, pest forecast and management. It can be implemented in hardware development as well, such as weather control in weather chamber and greenhouse for researches on ecological adaptation of crop varieties to a given location over time and space. Overall, VS-WGEN is a very useful tool for studies on theoretical and applied ecology.

Gaining-Sharing Knowledge Based Algorithm for Solving Stochastic Programming Problems

This paper presents a novel application of metaheuristic algorithmsfor solving stochastic programming problems using a recently developed gaining sharing knowledge based optimization (GSK) algorithm. The algorithmis based on human behavior in which people gain and share their knowledgewith others. Different types of stochastic fractional programming problemsare considered in this study. The augmented Lagrangian method (ALM)is used to handle these constrained optimization problems by convertingthem into unconstrained optimization problems. Three examples from theliterature are considered and transformed into their deterministic form usingthe chance-constrained technique. The transformed problems are solved usingGSK algorithm and the results are compared with eight other state-of-the-artmetaheuristic algorithms. The obtained results are also compared with theoptimal global solution and the results quoted in the literature. To investigatethe performance of the GSK algorithm on a real-world problem, a solidstochastic fixed charge transportation problem is examined, in which theparameters of the problem are considered as random variables. The obtainedresults show that the GSK algorithm outperforms other algorithms in termsof convergence, robustness, computational time, and quality of obtainedsolutions.

A Literature Review of Stochastic Programming and Unit Commitment

The study of unit commitment (UC) aims to find reasonable schedules for generators to optimize power systems’ operation. Many papers have been published that solve UC through different methods. Articles that systematically summarize UC problems’ progress in order to update researchers interested in this field are needed. Because of its promising performance, stochastic programming (SP) has become increasingly researched. Most papers, however, present SP’s UC solving approaches differently, which masks their relationships and makes it hard for new researchers to quickly obtain a general idea. Therefore, this paper tries to give a structured bibliographic survey of SP’s applications in UC problems.

Solution of Stochastic Quadratic Programming with Imperfect Probability Distribution Using Nelder-Mead Simplex Method

Stochastic quadratic programming with recourse is one of the most important topics in the field of optimization. It is usually assumed that the probability distribution of random variables has complete information, but only part of the information can be obtained in practical situation. In this paper, we propose a stochastic quadratic programming with imperfect probability distribution based on the linear partial information (LPI) theory. A direct optimizing algorithm based on Nelder-Mead simplex method is proposed for solving the problem. Finally, a numerical example is given to demonstrate the efficiency of the algorithm.

Stochastic Programming For Order Allocation And Production Planning

Stochastic demand is an important factor that heavily affects production planning.It influences activities such as purchasing,manufacturing,and selling,and quick adaption is required.In production planning,for reasons such as reducing costs and obtaining supplier discounts,many decisions must be made in the initial stage when demand has not been realized.The effects of non-optimal decisions will propagate to later stages,which can lead to losses due to overstocks or out-of-stocks.To find the optimal solutions for the initial and later stage regarding demand realization,this study proposes a stochastic two-stage linear program-ming model for a multi-supplier,multi-material,and multi-product purchasing and production planning process.The objective function is the expected total cost after two stages,and the results include detailed plans for purchasing and production in each demand scenario.Small-scale problems are solved through a deterministic equivalent transformation technique.To solve the problems in the large scale,an algorithm combining metaheuristic and sample average approximation is suggested.This algorithm can be implemented in parallel to utilize the power of the solver.The algorithm based on the observation that if the remaining quantity of materials and number of units of products at the end of the initial stage are given,then the problems of the first and second stages can be decomposed.

Relationship between Maximum Principle and Dynamic Programming in Stochastic Differential Games and Applications

This paper is concerned with the relationship between maximum principle and dynamic programming in zero-sum stochastic differential games. Under the assumption that the value function is enough smooth, relations among the adjoint processes, the generalized Hamiltonian function and the value function are given. A portfolio optimization problem under model uncertainty in the financial market is discussed to show the applications of our result.

Experimental Study of Methods of Scenario Lattice Construction for Stochastic Dual Dynamic Programming

The stochastic dual dynamic programming (SDDP) algorithm is becoming increasingly used. In this paper we present analysis of different methods of lattice construction for SDDP exemplifying a realistic variant of the newsvendor problem, incorporating storage of production. We model several days of work and compare the profits realized using different methods of the lattice construction and the corresponding computer time spent in lattice construction. Our case differs from the known one because we consider not only a multidimensional but also a multistage case with stage dependence. We construct scenario lattice for different Markov processes which play a crucial role in stochastic modeling. The novelty of our work is comparing different methods of scenario lattice construction. We considered a realistic variant of the newsvendor problem. The results presented in this article show that the Voronoi method slightly outperforms others, but the k-means method is much faster overall.

Stochastic Programming Model for Discrete Lotsizing and Scheduling Problem on Parallel Machines

In recent years, it has been difficult for manufactures and suppliers to forecast demand from a market for a given product precisely. Therefore, it has become important for them to cope with fluctuations in demand. From this viewpoint, the problem of planning or scheduling in production systems can be regarded as a mathematical problem with stochastic elements. However, in many previous studies, such problems are formulated without stochastic factors, treating stochastic elements as deterministic variables or parameters. Stochastic programming incorporates such factors into the mathematical formulation. In the present paper, we consider a multi-product, discrete, lotsizing and scheduling problem on parallel machines with stochastic demands. Under certain assumptions, this problem can be formulated as a stochastic integer programming problem. We attempt to solve this problem by a scenario aggregation method proposed by Rockafellar and Wets. The results from computational experiments suggest that our approach is able to solve large-scale problems, and that, under the condition of uncertainty, incorporating stochastic elements into the model gives better results than formulating the problem as a deterministic model.

A Complex Algorithm for Solving a Kind of Stochastic Programming

Considering that the probability distribution of random variables in stochastic programming usually has incomplete information due to a perfect sample data in many real applications, this paper discusses a class of two-stage stochastic programming problems modeling with maximum minimum expectation compensation criterion (MaxEMin) under the probability distribution having linear partial information (LPI). In view of the nondifferentiability of this kind of stochastic programming modeling, an improved complex algorithm is designed and analyzed. This algorithm can effectively solve the nondifferentiable stochastic programming problem under LPI through the variable polyhedron iteration. The calculation and discussion of numerical examples show the effectiveness of the proposed algorithm.

Multiobjective Stochastic Linear Programming: An Overview

Many Optimization problems in engineering and economic involve the challenging task of pondering both conflicting goals and random data. In this paper, we give an up-to-date overview of how important ideas from optimization, probability theory and multicriteria decision analysis are interwoven to address situations where the presence of several objective functions and the stochastic nature of data are under one roof in a linear optimization context. In this way users of these models are not bound to caricature their problems by arbitrarily squeezing different objective functions into one and by blindly accepting fixed values in lieu of imprecise ones.