Variance minimization for continuous-time Markov decision processes: two approaches

This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.

Development of Optimal Maintenance Policies for Offshore Wind Turbine Gearboxes Based on the Non-homogeneous Continuous-Time Markov Process

Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.

Variance Optimization for Continuous-Time Markov Decision Processes

This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.

MODERATE DEVIATIONS FOR PARAMETER ESTIMATORS IN FRACTIONAL ORNSTEIN-UHLENBECK PROCESS

We study moderate deviations for estimators of the drift parameter of the fractional Ornstein-Uhlenbeck process. Two moderate deviation principles are obtained.

PARAMETER ESTIMATION FOR AN ORNSTEIN-UHLENBECK PROCESS DRIVEN BY A GENERAL GAUSSIAN NOISE

In this paper,we consider an inference problem for an Ornstein-Uhlenbeck process driven by a general one-dimensional centered Gaussian process(G_(t))t≥0.The second order mixed partial derivative of the covariance function R(t,s)=E[GtGs]can be decomposed into two parts,one of which coincides with that of fractional Brownian motion and the other of which is bounded by(ts)^(β-1)up to a constant factor.This condition is valid for a class of continuous Gaussian processes that fails to be self-similar or to have stationary increments;some examples of this include the subfractional Brownian motion and the bi-fractional Brownian motion.Under this assumption,we study the parameter estimation for a drift parameter in the Ornstein-Uhlenbeck process driven by the Gaussian noise(G_(t))t≥0.For the least squares estimator and the second moment estimator constructed from the continuous observations,we prove the strong consistency and the asympotic normality,and obtain the Berry-Esséen bounds.The proof is based on the inner product's representation of the Hilbert space(h)associated with the Gaussian noise(G_(t))t≥0,and the estimation of the inner product based on the results of the Hilbert space associated with the fractional Brownian motion.

Parameter Estimation for Complex Ornstein-Uhlenbeck Processes

The drift parameter estimation problem of the complex Ornstein-Uhlenbeck process driven by a complexα-stable motion is considered.Based on discrete observations,an estimator of the unknown drift parameter is constructed by using the least squares method.Moreover,the strong consistency and the asymptotic distribution of the least squares estimator are derived under some assumptions.

ON COLLISION LOCAL TIME OF TWO INDEPENDENT FRACTIONAL ORNSTEIN-UHLENBECK PROCESSES

In this article, we study the existence of collision local time of two indepen- dent d-dimensional fractional Ornstein-Uhlenbeck processes X＋^H1 and Xt^H2 with different parameters Hi ∈ （0, 1）,i = 1, 2. Under the canonical framework of white noise analysis, we characterize the collision local time as a Hida distribution and obtain its＇ chaos expansion. Key words Collision local time; fractional Ornstein-Uhlenbeck processes; generalized white noise functionals; choas expansion

Stability of Stochastic Logistic Model with Ornstein-Uhlenbeck Process for Cell Growth of Microorganism in Fermentation Process

Current research is concerned with the stability of stochastic logistic equation with Ornstein-Uhlenbeck process. First, this research proves that the stochastic logistic model with Ornstein-Uhlenbeck process has a positive solution. After that, it also introduces the sufficient conditions for stochastically stability of stochastic logistic model for cell growth of microorganism in fermentation process for positive equilibrium point by using Lyapunov function. In addition, this research establishes the sufficient conditions for zero solution as mentioned in Appendix A due to the cell growth of microorganism μmax, which cannot be negative in fermentation process. Furthermore, for numerical simulation, current research uses the 4-stage stochastic Runge-Kutta (SRK4) method to show the reality of the results.

LEAST SQUARES TYPE ESTIMATION FOR DISCRETELY OBSERVED NON-ERGODIC GAUSSIAN ORNSTEIN-UHLENBECK PROCESSES

In this article, we consider the drift parameter estimation problem for the nonergodic Ornstein-Uhlenbeck process defined as dXt = OXtdt + dGt, i > 0 with an unknown parameter θ> 0, where G is a Gaussian process. We assume that the process {xt,t≥ 0} is observed at discrete time instants t1=△n,…, tn = n△n, and we construct two least squares type estimators θn and θn for θ on the basis of the discrete observations ,{xti,i= 1,…, n} as →∞. Then, we provide sufficient conditions, based on properties of G, which ensure that θn and θn are strongly consistent and the sequences √n△n(θn-θ) and √n△n(θn-θ) are tight. Our approach offers an elementary proof of [11], which studied the case when G is a fractional Brownian motion with Hurst parameter H∈(1/2, 1). As such, our results extend the recent findings by [11] to the case of general Hurst parameter H∈(0,1). We also apply our approach to study subfractional Ornstein-Uhlenbeck and bifractional Ornstein-Uhlenbeck processes.

THE LEAST SQUARES ESTIMATOR FOR AN ORNSTEIN-UHLENBECK PROCESS DRIVEN BY A HERMITE PROCESS WITH A PERIODIC MEAN

We consider the least square estimator for the parameters of Ornstein-Uhlenbeck processes dY_(s)=(∑_(j=1)^(k)μ_(j)φ_(j)(s)-βY_(s))ds+dZ_(s)^(q,H),driven by the Hermite process Z_(s)^(q,H)with order q≥1 and a Hurst index H∈(1/2,1),where the periodic functionsφ_(j)(s),,j=1,...,κare bounded,and the real numbersμ_(j),,j=1,...,κtogether withβ>0 are unknown parameters.We establish the consistency of a least squares estimation and obtain the asymptotic behavior for the estimator.We also introduce alternative estimators,which can be looked upon as an application of the least squares estimator.In terms of the fractional Ornstein-Uhlenbeck processes with periodic mean,our work can be regarded as its non-Gaussian extension.

ERRATUM TO： LEAST SQUARES ESTIMATION FOR ORNSTEIN-UHLENBECK PROCESSES DRIVEN BY THE WEIGHTED FRACTIONAL BROWNIAN MOTION （ACTA MATHEMATICA SCIENTIA 2016,36B （2） ：394-408）

We give a correction of Theorem 2.2 of Shen, Yin and Yan （2016）.

SOME CHARACTERS OF A CLASS OF ORNSTEIN-UHLENBECK TYPE MARKOV PROCESSES WITH STABLE PROCESSES

This article concerns a class of Ornstein-Uhlenbeck type Markov processes and for which the level sets will be approached. By constructing a new class f processes, we shall obtain an inequality on the Hausdorff dimensions of the level sets for the Ornstein-Uhlenbeck type Markov processes. Based on this result, we finally verify that any two independent O-U.M.P with alpha-stable processes could collide with probability one.

DIMENSION RESULTS FORORNSTEIN-UHLENBECK TYPE MARKOVPROCESS

Let {X-t, t greater than or equal to 0} be an Ornstein-Uhlenbeck type Markov process with Levy process A(t), the authors consider the fractal properties of its ranges, give the upper and lower bounds of the Hausdorff dimensions of the ranges and the estimate of the dimensions of the level sets for the process. The existence of local times and occupation times of X-t are considered in some special situations.

QUASI-STATIONARY DISTRIBUTIONS FOR THE RADIAL ORNSTEIN-UHLENBECK PROCESSES

The purpose of this article is to obtain the quasi-stationary distributions of the δ（δ 〈 2）-dimensional radial Ornstein-Uhlenbeck process with parameter -λ by using the methods of Martinez and San Martin （2001）. It is described that the law of this process conditioned on first hitting 0 is just the probability measure induced by a （4 - δ）- dimensional radial Ornstein-Uhlenbeck process with parameter -λ. Moreover, it is shown that the law of the conditioned process associated with the left eigenfunction of the process conditioned on first hitting 0 is induced by a one-parameter diffusion.

Asymptotic Properties of Estimators for Ornstein-Uhlenbeck Processes with Small Symmetricα-Stable Motions

The asymptotic behaviors for estimators of the drift parameters in the Ornstein-Uhlenbeck process driven by small symmetricα-stable motion are studied in this paper.Based on the discrete observations,the conditional least squares estimators(CLSEs)of all the parameters involved in the Ornstein–Uhlenbeck process are proposed.We establish the consistency and the asymptotic distributions of our estimators asεgoes to 0 and n goes to∞simultaneously.

Estimation for nearly unit root processes with GARCH errors

In this paper the limiting distribution of the least square estimate for the autoregressive coefficient of a nearly unit root model with GARCH errors is derived. Since the limiting distribution depends on the unknown variance of the errors, an empirical likelihood ratio statistic is proposed from which confidence intervals can be constructed for the nearly unit root model without knowing the variance. To gain an intuitive sense for the empirical likelihood ratio, a small simulation for the asymptotic distribution is given.

ON MODULI OF NON-DIFFERENTIABILITY FOR A TWO-PARAMETER ORNSTEIN-UHLENBECK PRCESS

In the paper, which is a continuation of [2], we establish moduli of non-differentiablity for a two-parameter Ornstein-Uhlenbeck process.

ON THE FRACTAL NATURE OF INCREMENTS OF THE INFINITE SERIES OF OU PROCESSES RELATED TO THE CHUNG LIL

Abstract The author proves that the set of points where the Chung type LIL fails for the path of the infinite series of independent Ornstein Uhlenbeck processes is a random fractal, and evaluates its Hausdorff dimension.

Stochastic Model of Dengue: Analysing the Probability of Extinction and LLN

In this article, we develop and analyze a continuous-time Markov chain (CTMC) model to study the resurgence of dengue. We also explore the large population asymptotic behavior of probabilistic model of dengue using the law of large numbers (LLN). Initially, we calculate and estimate the probabilities of dengue extinction and major outbreak occurrence using multi-type Galton-Watson branching processes. Subsequently, we apply the LLN to examine the convergence of the stochastic model towards the deterministic model. Finally, theoretical numerical simulations are conducted exploration to validate our findings. Under identical conditions, our numerical results demonstrate that dengue could vanish in the stochastic model while persisting in the deterministic model. The highlighting of the law of large numbers through numerical simulations indicates from what population size a deterministic model should be considered preferable.

STRONG N-DISCOUNT AND FINITE-HORIZON OPTIMALITY FOR CONTINUOUS-TIME MARKOV DECISION PROCESSES

This paper studies the strong n(n =—1,0)-discount and finite horizon criteria for continuoustime Markov decision processes in Polish spaces.The corresponding transition rates are allowed to be unbounded,and the reward rates may have neither upper nor lower bounds.Under mild conditions,the authors prove the existence of strong n(n =—1,0)-discount optimal stationary policies by developing two equivalence relations:One is between the standard expected average reward and strong—1-discount optimality,and the other is between the bias and strong 0-discount optimality.The authors also prove the existence of an optimal policy for a finite horizon control problem by developing an interesting characterization of a canonical triplet.