On stochastic optimal control of partially observable nonlinear quasi Hamiltonian systems

A stochastic optimal control strategy for partially observable nonlinear quasi Hamiltonian systems is proposed. The optimal control forces consist of two parts. The first part is determined by the conditions under which the stochastic optimal control problem of a partially observable nonlinear system is converted into that of a completely observable linear system. The second part is determined by solving the dynamical programming equation derived by applying the stochastic averaging method and stochastic dynamical programming principle to the completely observable linear control system. The response of the optimally controlled quasi Hamiltonian system is predicted by solving the averaged Fokker-Planck-Kolmogorov equation associated with the optimally controlled completely observable linear system and solving the Riccati equation for the estimated error of system states. An example is given to illustrate the procedure and effectiveness of the proposed control strategy.

A NEW STOCHASTIC OPTIMAL CONTROL STRATEGY FOR HYSTERETIC MR DAMPERS

A new stochastic optimal control strategy for randomly excited quasi-integrable Hamiltonian systems using magneto-rheological (MR) dampers is proposed. The dynamic be- havior of an MR damper is characterized by the Bouc-Wen hysteretic model. The control force produced by the MR damper is separated into a passive part incorporated in the uncontrolled system and a semi-active part to be determined. The system combining the Bouc-Wen hysteretic force is converted into an equivalent non-hysteretic nonlinear stochastic control system. Then It?o stochastic di?erential equations are derived from the equivalent system by using the stochastic averaging method. A dynamical programming equation for the controlled di?usion processes is established based on the stochastic dynamical programming principle. The non-clipping nonlin- ear optimal control law is obtained for a certain performance index by minimizing the dynamical programming equation. Finally, an example is given to illustrate the application and e?ectiveness of the proposed control strategy.

STOCHASTIC OPTIMAL CONTROL OF HYSTERETIC SYSTEMS UNDER EXTERNALLY AND PARAMETRICALLY RANDOM EXCITATIONS

A stochastic optimal control method for nonlinear hysteretic systems under externally and/or parametrically random excitations is presented and illustrated with an example of hysteretic column system. A hysteretic system subject to random excitation is first replaced by a nonlinear non-hysteretic stochastic system. An It$\hat {\rm o}$ stochastic differential equation for the total energy of the system as a one-dimensional controlled diffusion process is derived by using the stochastic averaging method of energy envelope. A dynamical programming equation is then established based on the stochastic dynamical programming principle and solved to yield the optimal control force. Finally, the responses of uncontrolled and controlled systems are evaluated to determine the control efficacy. It is shown by numerical results that the proposed stochastic optimal control method is more effective and efficient than other optimal control methods.

Stochastic optimal control for norovirus transmission dynamics by contaminated food and water

Norovirus is one of the most common causes of viral gastroenteritis in the world,causing significant morbidity,deaths,and medical costs.In this work,we look at stochastic modelling methodologies for norovirus transmission by water,human to human transmission and food.To begin,the proposed stochastic model is shown to have a single global positive solution.Second,we demonstrate adequate criteria for the existence of a unique ergodic stationary distribution R0 s>1 by developing a Lyapunov function.Thirdly,we find sufficient criteria Rs<1 for disease extinction.Finally,two simulation examples are used to exemplify the analytical results.We employed optimal control theory and examined stochastic control problems to regulate the spread of the disease using some external measures.Additional graphical solutions have been produced to further verify the acquired analytical results.This research could give a solid theoretical foundation for understanding chronic communicable diseases around the world.Our approach also focuses on offering a way of generating Lyapunov functions that can be utilized to investigate the stationary distribution of epidemic models with nonlinear stochastic disturbances.

Maximum principle for anticipated recursive stochastic optimal control problem with delay and Lvy processes

In this paper, we study the stochastic maximum principle for optimal control prob- lem of anticipated forward-backward system with delay and Lovy processes as the random dis- turbance. This control system can be described by the anticipated forward-backward stochastic differential equations with delay and L^vy processes （AFBSDEDLs）, we first obtain the existence and uniqueness theorem of adapted solutions for AFBSDEDLs; combining the AFBSDEDLs＇ preliminary result with certain classical convex variational techniques, the corresponding maxi- mum principle is proved.

Stochastic Optimal Control of First-Passage Failure for Rectangular Thin Plate Vibration Model under Gaussian White-Noise Excitations

A rectangular thin plate vibration model subjected to inplane stochastic excitation is simplified to a quasinonintegrable Hamiltonian system with two degrees of freedom. Subsequently a one-dimensional Ito stochastic differential equation for the system is obtained by applying the stochastic averaging method for quasi-nonintegrable Hamiltonian systems. The conditional reliability function and conditional probability density are both gained by solving the backward Kolmogorov equation numerically. Finally, a stochastic optimal control model is proposed and solved. The numerical results show the effectiveness of this method.

A NOTE ON STOCHASTIC OPTIMAL CONTROL OF REFLECTED DIFFUSIONS WITH JUMPS

Stochastic optimal control problems for a class of reflected diffusion with Poisson jumps in a half-space are considered. The nonlinear Nisio' s semigroup for such optimal control problems was constructed. A Hamilton-Jacobi-Bellman equation with the Neumann boundary condition associated with this semigroup was obtained. Then, viscosity solutions of this equation were defined and discussed, and various uniqueness of this equation was also considered. Finally, the value function was such optimal control problems is shown to be a viscosity solution of this equation.

A Stochastic Optimal Control Theory to Model Spontaneous Breathing

Respiratory variables, including tidal volume and respiratory rate, display significant variability. The probability density function (PDF) of respiratory variables has been shown to contain clinical information and can predict the risk for exacerbation in asthma. However, it is uncertain why this PDF plays a major role in predicting the dynamic conditions of the respiratory system. This paper introduces a stochastic optimal control model for noisy spontaneous breathing, and obtains a Shr&ouml;dinger’s wave equation as the motion equation that can produce a PDF as a solution. Based on the lobules-bronchial tree model of the lung system, the tidal volume variable was expressed by a polar coordinate, by use of which the Shr&ouml;dinger’s wave equation of inter-breath intervals (IBIs) was obtained. Through the wave equation of IBIs, the respiratory rhythm generator was characterized by the potential function including the PDF and the parameter concerning the topographical distribution of regional pulmonary ventilations. The stochastic model in this study was assumed to have a common variance parameter in the state variables, which would originate from the variability in metabolic energy at the cell level. As a conclusion, the PDF of IBIs would become a marker of neuroplasticity in the respiratory rhythm generator through Shr?dinger’s wave equation for IBIs.

Stochastic Maximum Principle for Optimal Advertising Models with Delay and Non-Convex Control Spaces

In this paper we study optimal advertising problems that model the introduction of a new product into the market in the presence of carryover effects of the advertisement and with memory effects in the level of goodwill. In particular, we let the dynamics of the product goodwill to depend on the past, and also on past advertising efforts. We treat the problem by means of the stochastic Pontryagin maximum principle, that here is considered for a class of problems where in the state equation either the state or the control depend on the past. Moreover the control acts on the martingale term and the space of controls U can be chosen to be non-convex but now the space of controls U can be chosen to be non-convex. The maximum principle is thus formulated using a first-order adjoint Backward Stochastic Differential Equations (BSDEs), which can be explicitly computed due to the specific characteristics of the model, and a second-order adjoint relation.

A Sample-Wise Data Driven Control Solver for the Stochastic Optimal Control Problem with Unknown Model Parameters

In this work,an efficient sample-wise data driven control solver will be developed to solve the stochastic optimal control problem with unknown model parameters.A direct filter method will be applied as an online parameter estimation method that dynamically estimates the target model parameters upon receiving the data,and a sample-wise optimal control solver will be provided to efficiently search for the optimal control.Then,an effective overarching algorithm will be introduced to combine the parameter estimator and the optimal control solver.Numerical experiments will be carried out to demonstrate the effectiveness and the efficiency of the sample-wise data driven control method.

FORWARD-BACKWARD STOCHASTIC DIFFERENTIAL EQUATIONS, LINEAR QUADRATIC STOCHASTIC OPTIMAL CONTROL AND NONZERO SUM DIFFERENTIAL GAMES

In this paper, we use the solutions of forward-backward stochastic differential equations to get the explicit form of the optimal control for linear quadratic stochastic optimal control problem and the open-loop Nash equilibrium point for nonzero sum differential games problem. We also discuss the solvability of the generalized Riccati equation system and give the linear feedback regulator for the optimal control problem using the solution of this kind of Riccati equation system.

Some results on pointwise second-order necessary conditions for stochastic optimal controls

The purpose of this paper is to derive some pointwise second-order necessary conditions for stochastic optimal controls in the general case that the control variable enters into both the drift and the diffusion terms.When the control region is convex, a pointwise second-order necessary condition for stochastic singular optimal controls in the classical sense is established; while when the control region is allowed to be nonconvex, we obtain a pointwise second-order necessary condition for stochastic singular optimal controls in the sense of Pontryagin-type maximum principle. It is found that, quite different from the first-order necessary conditions,the correction part of the solution to the second-order adjoint equation appears in the pointwise second-order necessary conditions whenever the diffusion term depends on the control variable, even if the control region is convex.

Meshfree Approximation for Stochastic Optimal Control Problems

In this work,we study the gradient projection method for solving a class of stochastic control problems by using a mesh free approximation ap-proach to implement spatial dimension approximation.Our main contribu-tion is to extend the existing gradient projection method to moderate high-dimensional space.The moving least square method and the general radial basis function interpolation method are introduced as showcase methods to demonstrate our computational framework,and rigorous numerical analysis is provided to prove the convergence of our meshfree approximation approach.We also present several numerical experiments to validate the theoretical re-sults of our approach and demonstrate the performance meshfree approxima-tion in solving stochastic optimal control problems.

Moving least-squares approximations for linearly-solvable stochastic optimal control problems

Nonlinear stochastic optimal control problems are fundamental in control theory.A general class of such problems can be reduced to computing the principal eigenfunction of a linear operator.Here,we describe a new method for finding this eigenfunction using a moving least-squares function approximation.We use efficient iterative solvers that do not require matrix factorization,thereby allowing us to handle large numbers of basis functions.The bases are evaluated at collocation states that change over iterations of the algorithm,so as to provide higher resolution at the regions of state space that are visited more often.The shape of the bases is automatically defined given the collocation states,in a way that avoids gaps in the coverage.Numerical results on test problems are provided.

The State Equations Methods for Stochastic Control Problems

The state equations of stochastic control problems,which are controlled stochastic differential equations,are proposed to be discretized by the weak midpoint rule and predictor-corrector methods for the Markov chain approximation approach. Local consistency of the methods are proved.Numerical tests on a simplified Merton's portfolio model show better simulation to feedback control rules by these two methods, as compared with the weak Euler-Maruyama discretisation used by Krawczyk.This suggests a new approach of improving accuracy of approximating Markov chains for stochastic control problems.

THE OPTIMAL DEDUCTIBLE AND COVERAGE IN INSURANCE CONTRACTS AND EQUILIBRIUM RISK SHARING POLICIES

In this paper, we consider the optimal risk sharing problem between two parties in the insurance business: the insurer and the insured. The risk is allocated between the insurer and the insured by setting a deductible and coverage in the insurance contract. We obtain the optimal deductible and coverage by considering the expected product of the two parties' utilities of terminal wealth according to stochastic optimal control theory. An equilibrium policy is also derived for when there are both a deductible and coverage;this is done by modelling the problem as a stochastic game in a continuous-time framework. A numerical example is provided to illustrate the results of the paper.

Active Vibration Control of Macroscopically Smart Mechanisms

This paper presents an investigation on the active vibration control of flexible linkage mechanisms featuring piezoceramic actuators and strain gauge sensors. The dynamic equation of the macroscopically smart mechanism is decoupled by means of the complex mode theory. The state-space expression of the controlled system is developed, which includes the system noise and the observation noise. Moreover, a discrete linear quadratic Gaussian （LQG） state feedback controller and a discrete Kalman filter are designed separately. Finally, the proposed method is applied to the on-line vibration control of a macroscopically smart mechanism. The experimental results reveal that the strain amplitude of the flexible link ig suppressed by 80% and the dynamic performance of mechanism has been ameliorated significantly.

A maximum principle for optimal control problem of fully coupled forward-backward stochastic systems with partial information

The paper is concerned with a stochastic optimal control problem in which the controlled system is described by a fully coupled nonlinear forward-backward stochastic differential equation driven by a Brownian motion.It is required that all admissible control processes are adapted to a given subfiltration of the filtration generated by the underlying Brownian motion.For this type of partial information control,one sufficient(a verification theorem) and one necessary conditions of optimality are proved.The control domain need to be convex and the forward diffusion coefficient of the system can contain the control variable.

G-stochastic maximum principle for risk-sensitive control problem and its applications

This study advances the G-stochastic maximum principle(G-SMP)from a risk-neutral framework to a risk-sensitive one.A salient feature of this advancement is its applicability to systems governed by stochastic differential equations under G-Brownian motion(G-SDEs),where the control variable may influence all terms.We aim to generalize our findings from a risk-neutral context to a risk-sensitive performance cost.Initially,we introduced an auxiliary process to address risk-sensitive performance costs within the G-expectation framework.Subsequently,we established and validated the correlation between the G-expected exponential utility and the G-quadratic backward stochastic differential equation.Furthermore,we simplified the G-adjoint process from a dual-component structure to a singular component.Moreover,we explained the necessary optimality conditions for this model by considering a convex set of admissible controls.To describe the main findings,we present two examples:the first addresses the linear-quadratic problem and the second examines a Merton-type problem characterized by power utility.

Turnpike Properties for Stochastic Linear-Quadratic Optimal Control Problems

This paper analyzes the limiting behavior of stochastic linear-quadratic optimal control problems in finite time-horizon[0,T]as T→∞.The so-called turnpike properties are established for such problems,under stabilizability condition which is weaker than the controllability,normally imposed in the similar problem for ordinary differential systems.In dealing with the turnpike problem,a crucial issue is to determine the corresponding static optimization problem.Intuitively mimicking the deterministic situations,it seems to be natural to include both the drift and the diffusion expressions of the state equation to be zero as constraints in the static optimization problem.However,this would lead us to a wrong direction.It is found that the correct static problem should contain the diffusion as a part of the objective function,which reveals a deep feature of the stochastic turnpike problem.