Matrix Riccati Equations in Optimal Control

In this paper, the matrix Riccati equation is considered. There is no general way for solving the matrix Riccati equation despite the many fields to which it applies. While scalar Riccati equation has been studied thoroughly, matrix Riccati equation of which scalar Riccati equations is a particular case, is much less investigated. This article proposes a change of variable that allows to find explicit solution of the Matrix Riccati equation. We then apply this solution to Optimal Control.

On stochastic optimal control of partially observable nonlinear quasi Hamiltonian systems

A stochastic optimal control strategy for partially observable nonlinear quasi Hamiltonian systems is proposed. The optimal control forces consist of two parts. The first part is determined by the conditions under which the stochastic optimal control problem of a partially observable nonlinear system is converted into that of a completely observable linear system. The second part is determined by solving the dynamical programming equation derived by applying the stochastic averaging method and stochastic dynamical programming principle to the completely observable linear control system. The response of the optimally controlled quasi Hamiltonian system is predicted by solving the averaged Fokker-Planck-Kolmogorov equation associated with the optimally controlled completely observable linear system and solving the Riccati equation for the estimated error of system states. An example is given to illustrate the procedure and effectiveness of the proposed control strategy.

STOCHASTIC OPTIMAL CONTROL OF HYSTERETIC SYSTEMS UNDER EXTERNALLY AND PARAMETRICALLY RANDOM EXCITATIONS

A stochastic optimal control method for nonlinear hysteretic systems under externally and/or parametrically random excitations is presented and illustrated with an example of hysteretic column system. A hysteretic system subject to random excitation is first replaced by a nonlinear non-hysteretic stochastic system. An It$\hat {\rm o}$ stochastic differential equation for the total energy of the system as a one-dimensional controlled diffusion process is derived by using the stochastic averaging method of energy envelope. A dynamical programming equation is then established based on the stochastic dynamical programming principle and solved to yield the optimal control force. Finally, the responses of uncontrolled and controlled systems are evaluated to determine the control efficacy. It is shown by numerical results that the proposed stochastic optimal control method is more effective and efficient than other optimal control methods.

Stochastic Optimal Control of First-Passage Failure for Rectangular Thin Plate Vibration Model under Gaussian White-Noise Excitations

A rectangular thin plate vibration model subjected to inplane stochastic excitation is simplified to a quasinonintegrable Hamiltonian system with two degrees of freedom. Subsequently a one-dimensional Ito stochastic differential equation for the system is obtained by applying the stochastic averaging method for quasi-nonintegrable Hamiltonian systems. The conditional reliability function and conditional probability density are both gained by solving the backward Kolmogorov equation numerically. Finally, a stochastic optimal control model is proposed and solved. The numerical results show the effectiveness of this method.

A NOTE ON STOCHASTIC OPTIMAL CONTROL OF REFLECTED DIFFUSIONS WITH JUMPS

Stochastic optimal control problems for a class of reflected diffusion with Poisson jumps in a half-space are considered. The nonlinear Nisio' s semigroup for such optimal control problems was constructed. A Hamilton-Jacobi-Bellman equation with the Neumann boundary condition associated with this semigroup was obtained. Then, viscosity solutions of this equation were defined and discussed, and various uniqueness of this equation was also considered. Finally, the value function was such optimal control problems is shown to be a viscosity solution of this equation.

Stability in Probability and Inverse Optimal Control of Evolution Systems Driven by Levy Processes

This paper first develops a Lyapunov-type theorem to study global well-posedness(existence and uniqueness of the strong variational solution)and asymptotic stability in probability of nonlinear stochastic evolution systems(SESs)driven by a special class of Levy processes,which consist of Wiener and compensated Poisson processes.This theorem is then utilized to develop an approach to solve an inverse optimal stabilization problem for SESs driven by Levy processes.The inverse optimal control design achieves global well-posedness and global asymptotic stability of the closed-loop system,and minimizes a meaningful cost functional that penalizes both states and control.The approach does not require to solve a Hamilton-Jacobi-Bellman equation(HJBE).An optimal stabilization of the evolution of the frequency of a certain genetic character from the population is included to illustrate the theoretical developments.

STOCHASTIC OPTIMAL CONTROL OF STRONGLY NONLINEAR SYSTEMS UNDER WIDE-BAND RANDOM EXCITATION WITH ACTUATOR SATURATION

A bounded optimal control strategy for strongly non-linear systems under non-white wide-band random excitation with actuator saturation is proposed. First, the stochastic averaging method is introduced for controlled strongly non-linear systems under wide-band random excitation using generalized harmonic functions. Then, the dynamical programming equation for the saturated control problem is formulated from the partially averaged Itō equation based on the dynamical programming principle. The optimal control consisting of the unbounded optimal control and the bounded bang-bang control is determined by solving the dynamical programming equation. Finally, the response of the optimally controlled system is predicted by solving the reduced Fokker-Planck-Kolmogorov （FPK） equation associated with the completed averaged Itō equation. An example is given to illustrate the proposed control strategy. Numerical results show that the proposed control strategy has high control effectiveness and efficiency and the chattering is reduced significantly comparing with the bang-bang control strategy.

Nonlinear dynamic characteristics and optimal control of a giant magnetostrictive film-shaped memory alloy composite plate subjected to in-plane stochastic excitation

The nonlinear dynamic characteristics and optimal control of a giant magnetostrictive film （GMF）-shaped memory alloy （SMA） composite plate subjected to in-plane stochastic excitation are studied. GMF is prepared based on an SMA plate, and combined into a GMF-SMA composite plate. The Van der Pol item is improved to explain the hysteretic phenomena of GMF and SMA, and the nonlinear dynamics model of a GMF-SMA composite cantilever plate subjected to in-plane stochastic excitation is developed. The stochastic stability of the system is analyzed, and the steady-state probability density function of the dynamic response of the system is obtained. The condition of stochastic Hopf bifurcation is discussed, the reliability function of the system is provided, and then the probability density of the first-passage time is given. Finally, the stochastic optimal control strategy is proposed by the stochastic dynamic programming method. Numerical simulation shows that the stability of the trivial solution varies with bifurcation parameters, and stochastic Hopf bifurcation appears in the process; the system＇s reliability is improved through stochastic optimal control, and the first- passage time is delayed. A GMF-SMA composite plate combines the advantages of GMF and SMA, and can reduce vibration through passive control and active control effectively. The results are helpful for the engineering applications of GMF-SMA composite plates.

STOCHASTIC OPTIMAL CONTROL FOR THE RESPONSE OF QUASI NON-INTEGRABLE HAMILTONIAN SYSTEMS

A strategy is proposed based on the stochastic averaging method for quasi non- integrable Hamiltonian systems and the stochastic dynamical programming principle.The pro- posed strategy can be used to design nonlinear stochastic optimal control to minimize the response of quasi non-integrable Hamiltonian systems subject to Gaussian white noise excitation.By using the stochastic averaging method for quasi non-integrable Hamiltonian systems the equations of motion of a controlled quasi non-integrable Hamiltonian system is reduced to a one-dimensional av- eraged It stochastic differential equation.By using the stochastic dynamical programming princi- ple the dynamical programming equation for minimizing the response of the system is formulated. The optimal control law is derived from the dynamical programming equation and the bounded control constraints.The response of optimally controlled systems is predicted through solving the FPK equation associated with It stochastic differential equation.An example is worked out in detail to illustrate the application of the control strategy proposed.

A NEW STOCHASTIC OPTIMAL CONTROL STRATEGY FOR HYSTERETIC MR DAMPERS

A new stochastic optimal control strategy for randomly excited quasi-integrable Hamiltonian systems using magneto-rheological (MR) dampers is proposed. The dynamic be- havior of an MR damper is characterized by the Bouc-Wen hysteretic model. The control force produced by the MR damper is separated into a passive part incorporated in the uncontrolled system and a semi-active part to be determined. The system combining the Bouc-Wen hysteretic force is converted into an equivalent non-hysteretic nonlinear stochastic control system. Then It?o stochastic di?erential equations are derived from the equivalent system by using the stochastic averaging method. A dynamical programming equation for the controlled di?usion processes is established based on the stochastic dynamical programming principle. The non-clipping nonlin- ear optimal control law is obtained for a certain performance index by minimizing the dynamical programming equation. Finally, an example is given to illustrate the application and e?ectiveness of the proposed control strategy.

Maximum principle for anticipated recursive stochastic optimal control problem with delay and Lvy processes

In this paper, we study the stochastic maximum principle for optimal control prob- lem of anticipated forward-backward system with delay and Lovy processes as the random dis- turbance. This control system can be described by the anticipated forward-backward stochastic differential equations with delay and L^vy processes （AFBSDEDLs）, we first obtain the existence and uniqueness theorem of adapted solutions for AFBSDEDLs; combining the AFBSDEDLs＇ preliminary result with certain classical convex variational techniques, the corresponding maxi- mum principle is proved.

Stochastic optimal control for norovirus transmission dynamics by contaminated food and water

Norovirus is one of the most common causes of viral gastroenteritis in the world,causing significant morbidity,deaths,and medical costs.In this work,we look at stochastic modelling methodologies for norovirus transmission by water,human to human transmission and food.To begin,the proposed stochastic model is shown to have a single global positive solution.Second,we demonstrate adequate criteria for the existence of a unique ergodic stationary distribution R0 s>1 by developing a Lyapunov function.Thirdly,we find sufficient criteria Rs<1 for disease extinction.Finally,two simulation examples are used to exemplify the analytical results.We employed optimal control theory and examined stochastic control problems to regulate the spread of the disease using some external measures.Additional graphical solutions have been produced to further verify the acquired analytical results.This research could give a solid theoretical foundation for understanding chronic communicable diseases around the world.Our approach also focuses on offering a way of generating Lyapunov functions that can be utilized to investigate the stationary distribution of epidemic models with nonlinear stochastic disturbances.

The Optimal Control for the Output Feedback Stochastic System at the Risk-Sensitive Cost

The optimal control of the partially observable stochastic system at the risk-sensitive cost is considered in this paper. The system dynamics has a general correlation between system and measurement noise. And the risk-sensitive cost contains a general quadratic term (with cross terms and extra linear terms). The explicit solution of such a problem is presented here using the output feedback control method. This clean and direct derivation enables one to convert such partial observable problems into the equivalent complete observable control problems and use the routine ways to solve them.

A Stochastic Optimal Control Theory to Model Spontaneous Breathing

Respiratory variables, including tidal volume and respiratory rate, display significant variability. The probability density function (PDF) of respiratory variables has been shown to contain clinical information and can predict the risk for exacerbation in asthma. However, it is uncertain why this PDF plays a major role in predicting the dynamic conditions of the respiratory system. This paper introduces a stochastic optimal control model for noisy spontaneous breathing, and obtains a Shr&ouml;dinger’s wave equation as the motion equation that can produce a PDF as a solution. Based on the lobules-bronchial tree model of the lung system, the tidal volume variable was expressed by a polar coordinate, by use of which the Shr&ouml;dinger’s wave equation of inter-breath intervals (IBIs) was obtained. Through the wave equation of IBIs, the respiratory rhythm generator was characterized by the potential function including the PDF and the parameter concerning the topographical distribution of regional pulmonary ventilations. The stochastic model in this study was assumed to have a common variance parameter in the state variables, which would originate from the variability in metabolic energy at the cell level. As a conclusion, the PDF of IBIs would become a marker of neuroplasticity in the respiratory rhythm generator through Shr?dinger’s wave equation for IBIs.

Least Squares Solution for Discrete Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences

In this paper, an efficient computational approach is proposed to solve the discrete time nonlinear stochastic optimal control problem. For this purpose, a linear quadratic regulator model, which is a linear dynamical system with the quadratic criterion cost function, is employed. In our approach, the model-based optimal control problem is reformulated into the input-output equations. In this way, the Hankel matrix and the observability matrix are constructed. Further, the sum squares of output error is defined. In these point of views, the least squares optimization problem is introduced, so as the differences between the real output and the model output could be calculated. Applying the first-order derivative to the sum squares of output error, the necessary condition is then derived. After some algebraic manipulations, the optimal control law is produced. By substituting this control policy into the input-output equations, the model output is updated iteratively. For illustration, an example of the direct current and alternating current converter problem is studied. As a result, the model output trajectory of the least squares solution is close to the real output with the smallest sum squares of output error. In conclusion, the efficiency and the accuracy of the approach proposed are highly presented.

Forward-backward Stochastic Differential Equations and Backward Linear Quadratic Stochastic Optimal Control Problem

In this paper, we use the solutions of forward-backward stochastic differential equations to get the optimal control for backward stochastic linear quadratic optimal control problem. And we also give the linear feedback regulator for the optimal control problem by using the solutions of a group of Riccati equations.

MAXIMUM PRINCIPLE FOR STOCHASTIC OPTIMAL CONTROL PROBLEM WITH DISTRIBUTED DELAYS

This paper is concerned with a Pontryagin's maximum principle for the stochastic optimal control problem with distributed delays given by integrals of not necessarily linear functions of state or control variables.By virtue of the duality method and the generalized anticipated backward stochastic differential equations,we establish a necessary maximum principle and a sufficient verification theorem.In particular,we deal with the controlled stochastic system where the distributed delays enter both the state and the control.To explain the theoretical results,we apply them to a dynamic advertising problem.

Discrete-Time Nonlinear Stochastic Optimal Control Problem Based on Stochastic Approximation Approach

In this paper, a computational approach is proposed for solving the discrete-time nonlinear optimal control problem, which is disturbed by a sequence of random noises. Because of the exact solution of such optimal control problem is impossible to be obtained, estimating the state dynamics is currently required. Here, it is assumed that the output can be measured from the real plant process. In our approach, the state mean propagation is applied in order to construct a linear model-based optimal control problem, where the model output is measureable. On this basis, an output error, which takes into account the differences between the real output and the model output, is defined. Then, this output error is minimized by applying the stochastic approximation approach. During the computation procedure, the stochastic gradient is established, so as the optimal solution of the model used can be updated iteratively. Once the convergence is achieved, the iterative solution approximates to the true optimal solution of the original optimal control problem, in spite of model-reality differences. For illustration, an example on a continuous stirred-tank reactor problem is studied, and the result obtained shows the applicability of the approach proposed. Hence, the efficiency of the approach proposed is highly recommended.

Stochastic Maximum Principle for Square-Integrable Optimal Control of Linear Stochastic Systems

The authors give a stochastic maximum principle for square-integrable optimal control of linear stochastic systems.The control domain is not necessarily convex and the cost functional can have a quadratic growth.In particular,they give a stochastic maximum principle for the linear quadratic optimal control problem.

Stochastic Maximum Principle for Optimal Advertising Models with Delay and Non-Convex Control Spaces

In this paper we study optimal advertising problems that model the introduction of a new product into the market in the presence of carryover effects of the advertisement and with memory effects in the level of goodwill. In particular, we let the dynamics of the product goodwill to depend on the past, and also on past advertising efforts. We treat the problem by means of the stochastic Pontryagin maximum principle, that here is considered for a class of problems where in the state equation either the state or the control depend on the past. Moreover the control acts on the martingale term and the space of controls U can be chosen to be non-convex but now the space of controls U can be chosen to be non-convex. The maximum principle is thus formulated using a first-order adjoint Backward Stochastic Differential Equations (BSDEs), which can be explicitly computed due to the specific characteristics of the model, and a second-order adjoint relation.