A CLASS OF STATIONARY MODELS OF SINGULAR STOCHASTIC CONTROL

A class of stationary models of singular stochastic control has been studied, in which the state is extended to solution of a class of S.D.E. from Wiener process. The existence of optimal control has been proved in all cases under some weaker conditions, and the structure of optimal control may be characterized.

The State Equations Methods for Stochastic Control Problems

The state equations of stochastic control problems,which are controlled stochastic differential equations,are proposed to be discretized by the weak midpoint rule and predictor-corrector methods for the Markov chain approximation approach. Local consistency of the methods are proved.Numerical tests on a simplified Merton's portfolio model show better simulation to feedback control rules by these two methods, as compared with the weak Euler-Maruyama discretisation used by Krawczyk.This suggests a new approach of improving accuracy of approximating Markov chains for stochastic control problems.

Stochastic Maximum Principle for Optimal Advertising Models with Delay and Non-Convex Control Spaces

In this paper we study optimal advertising problems that model the introduction of a new product into the market in the presence of carryover effects of the advertisement and with memory effects in the level of goodwill. In particular, we let the dynamics of the product goodwill to depend on the past, and also on past advertising efforts. We treat the problem by means of the stochastic Pontryagin maximum principle, that here is considered for a class of problems where in the state equation either the state or the control depend on the past. Moreover the control acts on the martingale term and the space of controls U can be chosen to be non-convex but now the space of controls U can be chosen to be non-convex. The maximum principle is thus formulated using a first-order adjoint Backward Stochastic Differential Equations (BSDEs), which can be explicitly computed due to the specific characteristics of the model, and a second-order adjoint relation.

MAXIMUM PRINCIPLE FOR FORWARD-BACKWARD STOCHASTIC CONTROL SYSTEM WITH RANDOM JUMPS AND APPLICATIONS TO FINANCE

Both necessary and sufficient maximum principles for optimal control of stochastic systemwith random jumps consisting of forward and backward state variables are proved.The control variableis allowed to enter both diffusion and jump coefficients.The result is applied to a mean-varianceportfolio selection mixed with a recursive utility functional optimization problem.Explicit expressionof the optimal portfolio selection strategy is obtained in the state feedback form.

ON SUFFICIENT AND NECESSARY OF EXISTENCE FOR A CLASS OF SINGULAR OPTIMAL STOCHASTIC CONTROL

We study a class of discounted models of singular stochastic control. In thiskind of models, not only the structure of cost function has been extended to some general type, butalso the state can be represented as the solution of a class of stochastic differential equationswith nonlinear drift and diffusion term. By the various methods of stochastic analysis, we derivethe sufficient and necessary conditions of the existence of optimal control.

Iterative Learning Control for Discrete-time Stochastic Systems with Quantized Information

An iterative learning control (ILC) algorithm using quantized error information is given in this paper for both linear and nonlinear discrete-time systems with stochastic noises. A logarithmic quantizer is used to guarantee an adaptive improvement in tracking performance. A decreasing learning gain is introduced into the algorithm to suppress the effects of stochastic noises and quantization errors. The input sequence is proved to converge strictly to the optimal input under the given index. Illustrative simulations are given to verify the theoretical analysis. © 2014 Chinese Association of Automation.

Optimal variational principle for backward stochastic control systems associated with Lévy processes

The paper is concerned with optimal control of backward stochastic differentiM equation （BSDE） driven by Teugel＇s martingales and an independent multi-dimensional Brownian motion, where Teugel＇s martingales are a family of pairwise strongly orthonormal martingales associated with L6vy processes （see e.g., Nualart and Schoutens＇ paper in 2000）. We derive the necessary and sufficient conditions for the existence of the optimal control by means of convex variation methods and duality techniques. As an application, the optimal control problem of linear backward stochastic differential equation with a quadratic cost criteria （or backward linear-quadratic problem, or BLQ problem for short） is discussed and characterized by a stochastic Hamilton system.

Network selection policy in multi-radio access environment using stochastic control theory

Network selection is crucial in improving the performance of heterogeneous wireless access systems. Most of previous work on network selection or radio resource allocation concentrates on the capability of each available network and ignores the time-varying nature of wireless media due to channel fading. However, the channel condition determines the state of each wireless network and plays a vital role in ensuring quality of service in multi-radio access environment. In this article, we propose a network selection policy using stochastic control theory considering the time-varying and stochastic character of wireless channels. The proposed scheme selects one network among different alternatives in each decision epoch according to the channel state of each network, which is modeled as finite-state Markov channel, with the objectives of increasing the data-rate, decreasing the bit error rate and minishing the delay. The procedure of network selection is formulated as a stochastic control problem, which can be solved using linear programming and primal-dual index heuristic algorithm. Simulation results are presented to show that network selection has great impact on the system performance, and the proposed scheme can improve the performance significantly.

The generalization of a class of impulse stochastic control models of a geometric Brownian motion

Recently, international academic circles advanced a class of new stochastic control models of a geometric Brownian motion which is an important kind of impulse control models whose cost structure is different from the others before, and it has a broad applying background and important theoretical significance in financial control and management of investment.This paper generalizes substantially the above stochastic control models under quite extensive conditions and describes the models more exactly under more normal theoretical system of stochastic process.By establishing a set of proper variational equations and proving the existence of its solution, and applying the means of stochastic analysis, this paper proves that the generalized stochastic control models have optimal controls.Meanwhile, we also analyze the structure of optimal controls carefully.Besides, we study the solution function of variational equations in a relatively deep-going way, which constitutes the value function of control models to some extent.Because the analysis methods of this paper are greatly different from those of original reference, this paper possesses considerable originality to some extent.In addition, this paper gives the strict proof to the part of original reference which is not fairly well-knit in analyses, and makes analyses and discussions of the model have the exactitude of mathematical sense.

Peng's maximum principle for a stochastic control problem driven by a fractional and a standard Brownian motion

We study a stochastic control system involving both a standard and a fractional Brownian motion with Hurst parameter less than 1/2.We apply an anticipative Girsanov transformation to transform the system into another one,driven only by the standard Brownian motion with coefficients depending on both the fractional Brownian motion and the standard Brownian motion.We derive a maximum principle and the associated stochastic variational inequality,which both are generalizations of the classical case.

Dynamic Programming for Multidimensional Stochastic Control Problems

In this paper we study a general multidimensional diffusion-type stochastic control problem. Our model contains the usual regular control problem,singular control problem and impulse control problem as special cases.Using a unified treatment of dynamic programming,we show that the value function of the problem is a viscosity solution of certain Hamilton-Jacobi-Bellman (HJB) quasi- variational inequality.The uniqueness of such a quasi-variational inequality is proved.

ON CONTROLLABILITY FOR STOCHASTIC CONTROL SYSTEMS WHEN THE COEFFICIENT IS TIME-VARIANT

This paper investigates the controllability problem of time-variant linear stochastic controlsystems.A sufficient and necessary condition is established for stochastic exact controllability,whichprovides a useful algebraic criterion for stochastic control systems.Furthermore,when the stochasticsystems degenerate to deterministic systems,the algebraic criterion becomes the counterpart for thecomplete controllability of deterministic control systems.

High throughput relay policy in wireless cooperative relaying networks based on stochastic control theory

This paper proposes a distributed relay and modulation and coding scheme （MCS） selection in wireless cooperative relaying networks where the adaptive modulation and coding （AMC） scheme is applied. First-order finite-state Markov channels （FSMCs） are used to model the wireless channels and make prediction. The objective of the relay policy is to select one relay and MCS among different alternatives in each time-slot according to their channel state information （CSI） with the goal of maximizing the throughput of the whole transmission period. The procedure of relay and MCS selection can be formulated as a discounted Markov decision chain, and the relay policy can be obtained with recent advances in stochastic control algorithms. Simulation results are presented to show the effectiveness of the proposed scheme.

A Maximum Principle for Fully Coupled Forward-Backward Stochastic Control System Driven by Lvy Process with Terminal State Constraints

This paper is concerned with a fully coupled forward-backward stochastic optimal control problem where the controlled system is driven by Levy process, while the forward state is constrained in a convex set at the terminal time. The authors use an equivalent backward formulation to deal with the terminal state constraint, and then obtain a stochastic maximum principle by Ekeland＇s variational principle. Finally, the result is applied to the utility optimization problem in a financial market.

Tensor Decomposition and High-Performance Computing for Solving High-Dimensional Stochastic Control System Numerically

The paper presents a numerical method for solving a class of high-dimensional stochastic control systems based on tensor decomposition and parallel computing.The HJB solution provides a globally optimal controller to the associated dynamical system.Variable substitution is used to simplify the nonlinear HJB equation.The curse of dimensionality is avoided by representing the HJB equation using separated representation.Alternating least squares(ALS)is used to reduced the separation rank.The experiment is conducted and the numerical solution is obtained.A high-performance algorithm is designed to reduce the separation rank in the parallel environment,solving the high-dimensional HJB equation with high efficiency.

The Stochastic Control Model for Use Conversion of Land

In this paper, the authors investigate the optimal conversion rate at which land use is irreversibly converted from biodiversity conservation to agricultural production. This problem is formulated as a stochastic control model, then transformed into a HJB equation involving free boundary. Since the state equation has singularity, it is difficult to directly derive the boundary value condition for the HJB equation. They provide a new method to overcome the difficulty via constructing another auxiliary stochastic control problem,and impose a proper boundary value condition. Moreover, they establish the existence and uniqueness of the viscosity solution of the HJB equation. Finally, they propose a stable numerical method for the HJB equation involving free boundary, and show some numerical results.

The Optimal Control for the Output Feedback Stochastic System at the Risk-Sensitive Cost

The optimal control of the partially observable stochastic system at the risk-sensitive cost is considered in this paper. The system dynamics has a general correlation between system and measurement noise. And the risk-sensitive cost contains a general quadratic term (with cross terms and extra linear terms). The explicit solution of such a problem is presented here using the output feedback control method. This clean and direct derivation enables one to convert such partial observable problems into the equivalent complete observable control problems and use the routine ways to solve them.

A Stochastic Optimal Control Theory to Model Spontaneous Breathing

Respiratory variables, including tidal volume and respiratory rate, display significant variability. The probability density function (PDF) of respiratory variables has been shown to contain clinical information and can predict the risk for exacerbation in asthma. However, it is uncertain why this PDF plays a major role in predicting the dynamic conditions of the respiratory system. This paper introduces a stochastic optimal control model for noisy spontaneous breathing, and obtains a Shrödinger’s wave equation as the motion equation that can produce a PDF as a solution. Based on the lobules-bronchial tree model of the lung system, the tidal volume variable was expressed by a polar coordinate, by use of which the Shrödinger’s wave equation of inter-breath intervals (IBIs) was obtained. Through the wave equation of IBIs, the respiratory rhythm generator was characterized by the potential function including the PDF and the parameter concerning the topographical distribution of regional pulmonary ventilations. The stochastic model in this study was assumed to have a common variance parameter in the state variables, which would originate from the variability in metabolic energy at the cell level. As a conclusion, the PDF of IBIs would become a marker of neuroplasticity in the respiratory rhythm generator through Shr?dinger’s wave equation for IBIs.

On stochastic optimal control of partially observable nonlinear quasi Hamiltonian systems

A stochastic optimal control strategy for partially observable nonlinear quasi Hamiltonian systems is proposed. The optimal control forces consist of two parts. The first part is determined by the conditions under which the stochastic optimal control problem of a partially observable nonlinear system is converted into that of a completely observable linear system. The second part is determined by solving the dynamical programming equation derived by applying the stochastic averaging method and stochastic dynamical programming principle to the completely observable linear control system. The response of the optimally controlled quasi Hamiltonian system is predicted by solving the averaged Fokker-Planck-Kolmogorov equation associated with the optimally controlled completely observable linear system and solving the Riccati equation for the estimated error of system states. An example is given to illustrate the procedure and effectiveness of the proposed control strategy.

A NEW STOCHASTIC OPTIMAL CONTROL STRATEGY FOR HYSTERETIC MR DAMPERS

A new stochastic optimal control strategy for randomly excited quasi-integrable Hamiltonian systems using magneto-rheological (MR) dampers is proposed. The dynamic be- havior of an MR damper is characterized by the Bouc-Wen hysteretic model. The control force produced by the MR damper is separated into a passive part incorporated in the uncontrolled system and a semi-active part to be determined. The system combining the Bouc-Wen hysteretic force is converted into an equivalent non-hysteretic nonlinear stochastic control system. Then It?o stochastic di?erential equations are derived from the equivalent system by using the stochastic averaging method. A dynamical programming equation for the controlled di?usion processes is established based on the stochastic dynamical programming principle. The non-clipping nonlin- ear optimal control law is obtained for a certain performance index by minimizing the dynamical programming equation. Finally, an example is given to illustrate the application and e?ectiveness of the proposed control strategy.