A CLASS OF STATIONARY MODELS OF SINGULAR STOCHASTIC CONTROL

A class of stationary models of singular stochastic control has been studied, in which the state is extended to solution of a class of S.D.E. from Wiener process. The existence of optimal control has been proved in all cases under some weaker conditions, and the structure of optimal control may be characterized.

Network selection policy in multi-radio access environment using stochastic control theory

Network selection is crucial in improving the performance of heterogeneous wireless access systems. Most of previous work on network selection or radio resource allocation concentrates on the capability of each available network and ignores the time-varying nature of wireless media due to channel fading. However, the channel condition determines the state of each wireless network and plays a vital role in ensuring quality of service in multi-radio access environment. In this article, we propose a network selection policy using stochastic control theory considering the time-varying and stochastic character of wireless channels. The proposed scheme selects one network among different alternatives in each decision epoch according to the channel state of each network, which is modeled as finite-state Markov channel, with the objectives of increasing the data-rate, decreasing the bit error rate and minishing the delay. The procedure of network selection is formulated as a stochastic control problem, which can be solved using linear programming and primal-dual index heuristic algorithm. Simulation results are presented to show that network selection has great impact on the system performance, and the proposed scheme can improve the performance significantly.

The generalization of a class of impulse stochastic control models of a geometric Brownian motion

Recently, international academic circles advanced a class of new stochastic control models of a geometric Brownian motion which is an important kind of impulse control models whose cost structure is different from the others before, and it has a broad applying background and important theoretical significance in financial control and management of investment.This paper generalizes substantially the above stochastic control models under quite extensive conditions and describes the models more exactly under more normal theoretical system of stochastic process.By establishing a set of proper variational equations and proving the existence of its solution, and applying the means of stochastic analysis, this paper proves that the generalized stochastic control models have optimal controls.Meanwhile, we also analyze the structure of optimal controls carefully.Besides, we study the solution function of variational equations in a relatively deep-going way, which constitutes the value function of control models to some extent.Because the analysis methods of this paper are greatly different from those of original reference, this paper possesses considerable originality to some extent.In addition, this paper gives the strict proof to the part of original reference which is not fairly well-knit in analyses, and makes analyses and discussions of the model have the exactitude of mathematical sense.

Dynamic Programming for Multidimensional Stochastic Control Problems

In this paper we study a general multidimensional diffusion-type stochastic control problem. Our model contains the usual regular control problem,singular control problem and impulse control problem as special cases.Using a unified treatment of dynamic programming,we show that the value function of the problem is a viscosity solution of certain Hamilton-Jacobi-Bellman (HJB) quasi- variational inequality.The uniqueness of such a quasi-variational inequality is proved.

High throughput relay policy in wireless cooperative relaying networks based on stochastic control theory

This paper proposes a distributed relay and modulation and coding scheme （MCS） selection in wireless cooperative relaying networks where the adaptive modulation and coding （AMC） scheme is applied. First-order finite-state Markov channels （FSMCs） are used to model the wireless channels and make prediction. The objective of the relay policy is to select one relay and MCS among different alternatives in each time-slot according to their channel state information （CSI） with the goal of maximizing the throughput of the whole transmission period. The procedure of relay and MCS selection can be formulated as a discounted Markov decision chain, and the relay policy can be obtained with recent advances in stochastic control algorithms. Simulation results are presented to show the effectiveness of the proposed scheme.

Tensor Decomposition and High-Performance Computing for Solving High-Dimensional Stochastic Control System Numerically

The paper presents a numerical method for solving a class of high-dimensional stochastic control systems based on tensor decomposition and parallel computing.The HJB solution provides a globally optimal controller to the associated dynamical system.Variable substitution is used to simplify the nonlinear HJB equation.The curse of dimensionality is avoided by representing the HJB equation using separated representation.Alternating least squares(ALS)is used to reduced the separation rank.The experiment is conducted and the numerical solution is obtained.A high-performance algorithm is designed to reduce the separation rank in the parallel environment,solving the high-dimensional HJB equation with high efficiency.

The Stochastic Control Model for Use Conversion of Land

In this paper, the authors investigate the optimal conversion rate at which land use is irreversibly converted from biodiversity conservation to agricultural production. This problem is formulated as a stochastic control model, then transformed into a HJB equation involving free boundary. Since the state equation has singularity, it is difficult to directly derive the boundary value condition for the HJB equation. They provide a new method to overcome the difficulty via constructing another auxiliary stochastic control problem,and impose a proper boundary value condition. Moreover, they establish the existence and uniqueness of the viscosity solution of the HJB equation. Finally, they propose a stable numerical method for the HJB equation involving free boundary, and show some numerical results.

The Optimal Control for the Output Feedback Stochastic System at the Risk-Sensitive Cost

The optimal control of the partially observable stochastic system at the risk-sensitive cost is considered in this paper. The system dynamics has a general correlation between system and measurement noise. And the risk-sensitive cost contains a general quadratic term (with cross terms and extra linear terms). The explicit solution of such a problem is presented here using the output feedback control method. This clean and direct derivation enables one to convert such partial observable problems into the equivalent complete observable control problems and use the routine ways to solve them.

A Stochastic Optimal Control Theory to Model Spontaneous Breathing

Respiratory variables, including tidal volume and respiratory rate, display significant variability. The probability density function (PDF) of respiratory variables has been shown to contain clinical information and can predict the risk for exacerbation in asthma. However, it is uncertain why this PDF plays a major role in predicting the dynamic conditions of the respiratory system. This paper introduces a stochastic optimal control model for noisy spontaneous breathing, and obtains a Shrödinger’s wave equation as the motion equation that can produce a PDF as a solution. Based on the lobules-bronchial tree model of the lung system, the tidal volume variable was expressed by a polar coordinate, by use of which the Shrödinger’s wave equation of inter-breath intervals (IBIs) was obtained. Through the wave equation of IBIs, the respiratory rhythm generator was characterized by the potential function including the PDF and the parameter concerning the topographical distribution of regional pulmonary ventilations. The stochastic model in this study was assumed to have a common variance parameter in the state variables, which would originate from the variability in metabolic energy at the cell level. As a conclusion, the PDF of IBIs would become a marker of neuroplasticity in the respiratory rhythm generator through Shr?dinger’s wave equation for IBIs.

A NEW STOCHASTIC OPTIMAL CONTROL STRATEGY FOR HYSTERETIC MR DAMPERS

A new stochastic optimal control strategy for randomly excited quasi-integrable Hamiltonian systems using magneto-rheological (MR) dampers is proposed. The dynamic be- havior of an MR damper is characterized by the Bouc-Wen hysteretic model. The control force produced by the MR damper is separated into a passive part incorporated in the uncontrolled system and a semi-active part to be determined. The system combining the Bouc-Wen hysteretic force is converted into an equivalent non-hysteretic nonlinear stochastic control system. Then It?o stochastic di?erential equations are derived from the equivalent system by using the stochastic averaging method. A dynamical programming equation for the controlled di?usion processes is established based on the stochastic dynamical programming principle. The non-clipping nonlin- ear optimal control law is obtained for a certain performance index by minimizing the dynamical programming equation. Finally, an example is given to illustrate the application and e?ectiveness of the proposed control strategy.

STOCHASTIC OPTIMAL CONTROL OF HYSTERETIC SYSTEMS UNDER EXTERNALLY AND PARAMETRICALLY RANDOM EXCITATIONS

A stochastic optimal control method for nonlinear hysteretic systems under externally and/or parametrically random excitations is presented and illustrated with an example of hysteretic column system. A hysteretic system subject to random excitation is first replaced by a nonlinear non-hysteretic stochastic system. An It$\hat {\rm o}$ stochastic differential equation for the total energy of the system as a one-dimensional controlled diffusion process is derived by using the stochastic averaging method of energy envelope. A dynamical programming equation is then established based on the stochastic dynamical programming principle and solved to yield the optimal control force. Finally, the responses of uncontrolled and controlled systems are evaluated to determine the control efficacy. It is shown by numerical results that the proposed stochastic optimal control method is more effective and efficient than other optimal control methods.

Stochastic optimal control for norovirus transmission dynamics by contaminated food and water

Norovirus is one of the most common causes of viral gastroenteritis in the world,causing significant morbidity,deaths,and medical costs.In this work,we look at stochastic modelling methodologies for norovirus transmission by water,human to human transmission and food.To begin,the proposed stochastic model is shown to have a single global positive solution.Second,we demonstrate adequate criteria for the existence of a unique ergodic stationary distribution R0 s>1 by developing a Lyapunov function.Thirdly,we find sufficient criteria Rs<1 for disease extinction.Finally,two simulation examples are used to exemplify the analytical results.We employed optimal control theory and examined stochastic control problems to regulate the spread of the disease using some external measures.Additional graphical solutions have been produced to further verify the acquired analytical results.This research could give a solid theoretical foundation for understanding chronic communicable diseases around the world.Our approach also focuses on offering a way of generating Lyapunov functions that can be utilized to investigate the stationary distribution of epidemic models with nonlinear stochastic disturbances.

Stochastic impulsive control for the stabilization of Lorenz system

This paper derives some sufficient conditions for the stabilization of Lorenz system with stochastic impulsive control. The estimate of the upper bound of impulse interval for asymptotically stable control is obtained. Some differences between the system with stochastic impulsive control and with deterministic impulsive control are presented. Computer simulation is given to show the effectiveness of the proposed method.

Input-to-state stability of Euler-Maruyama method for stochastic delay control systems

This paper develops the mean-square exponential input-to-state stability（exp-ISS） of the Euler-Maruyama（EM） method for stochastic delay control systems（SDCSs）.The definition of mean-square exp-ISS of numerical methods is established.The conditions of the exact and EM method for an SDCS with the property of mean-square exp-ISS are obtained without involving control Lyapunov functions or functional.Under the global Lipschitz coefficients and mean-square continuous measurable inputs,it is proved that the mean-square exp-ISS of an SDCS holds if and only if that of the EM method is preserved for a sufficiently small step size.The proposed results are evaluated by using numerical experiments to show their effectiveness.

Robust H_∞ control for neutral stochastic uncertain systems with time-varying delay

The problem of robust H_∞ control for uncertain neutral stochastic systems with time-varying delay is discussed.The parameter uncertaintie is assumed to be time varying norm-bounded.First,the stochastic robust stabilization of the stochastic system without disturbance input is investigated by nonlinear matrix inequality method.Then,a full-order stochastic dynamic output feedback controller is designed by solving a bilinear matrix inequality（BMI）,which ensures a prescribed stochastic robust H_∞ performance level for the resulting closed-loop system with nonzero disturbance input and for all admissible uncertainties.An illustrative example is provided to show the feasibility of the controller and the potential of the proposed technique.

A Mean-Field Necessary and Sufficient Conditions for Optimal Singular Stochastic Control

This paper studies singular optimal control problems for systems described by nonlinear-controlled stochastic differential equations of mean-field type(MFSDEs in short),in which the coefficients depend on the state of the solution process as well as of its expected value.Moreover,the cost functional is also of mean-field type.The control variable has two components,the first being absolutely continuous and the second singular.We establish necessary as well as sufficient conditions for optimal singular stochastic control where the system evolves according to MFSDEs.These conditions of optimality differs from the classical one in the sense that here the adjoint equation turns out to be a linear mean-field backward stochastic differential equation.The proof of our result is based on convex perturbation method of a given optimal control.The control domain is assumed to be convex.A linear quadratic stochastic optimal control problem of mean-field type is discussed as an illustrated example.

Maximum principle for anticipated recursive stochastic optimal control problem with delay and Lvy processes

In this paper, we study the stochastic maximum principle for optimal control prob- lem of anticipated forward-backward system with delay and Lovy processes as the random dis- turbance. This control system can be described by the anticipated forward-backward stochastic differential equations with delay and L^vy processes （AFBSDEDLs）, we first obtain the existence and uniqueness theorem of adapted solutions for AFBSDEDLs; combining the AFBSDEDLs＇ preliminary result with certain classical convex variational techniques, the corresponding maxi- mum principle is proved.

A NOTE ON STOCHASTIC OPTIMAL CONTROL OF REFLECTED DIFFUSIONS WITH JUMPS

Stochastic optimal control problems for a class of reflected diffusion with Poisson jumps in a half-space are considered. The nonlinear Nisio' s semigroup for such optimal control problems was constructed. A Hamilton-Jacobi-Bellman equation with the Neumann boundary condition associated with this semigroup was obtained. Then, viscosity solutions of this equation were defined and discussed, and various uniqueness of this equation was also considered. Finally, the value function was such optimal control problems is shown to be a viscosity solution of this equation.

Stochastic optimal control of cable vibration in plane by using axial support motion

A stochastic optimal control strategy for a slightly sagged cable using support motion in the cable axial direction is proposed. The nonlinear equation of cable motion in plane is derived and reduced to the equations for the first two modes of cable vibration by using the Galerkin method. The partially averaged Ito equation for controlled system energy is further derived by applying the stochastic averaging method for quasi-non-integrable Hamiltonian systems. The dynamical programming equation for the controlled system energy with a performance index is established by applying the stochastic dynamical programming principle and a stochastic optimal control law is obtained through solving the dynamical programming equation. A bilinear controller by using the direct method of Lyapunov is introduced. The comparison between the two controllers shows that the proposed stochastic optimal control strategy is superior to the bilinear control strategy in terms of higher control effectiveness and efficiency.

A MAXIMUM PRINCIPLE APPROACH TO STOCHASTIC H_2/H_∞ CONTROL WITH RANDOM JUMPS

A necessary maximum principle is given for nonzero-sum stochastic Oltterential games with random jumps. The result is applied to solve the H2/H∞ control problem of stochastic systems with random jumps. A necessary and sufficient condition for the existence of a unique solution to the H2/H∞ control problem is derived. The resulting solution is given by the solution of an uncontrolled forward backward stochastic differential equation with random jumps.