Hausdorff Dimension of Range and Graph for General Markov Processes

We establish the Hausdorff dimension of the graph of general Markov processes on Rd based on some probability estimates of the processes staying or leaving small balls in small time.In particular,our results indicate that,for symmetric diffusion processes(withα=2)or symmetricα-stable-like processes(withα∈(0,2))on Rd,it holds almost surely that dimH GrX([0,1])=1{α<1}+(2−1/α)1{α≥1,d=1}+(d∧α)1{α≥1,d≥2}.We also systematically prove the corresponding results about the Hausdorff dimension of the range of the processes.

On approximating multifractal traffic burstiness with Markov modulated Poisson processes

We investigate the approximating capability of Markov modulated Poisson processes (MMPP) for modeling multifractal Internet traffic. The choice of MMPP is motivated by its ability to capture the variability and correlation in moderate time scales while being analytically tractable. Important statistics of traffic burstiness are described and a customized moment-based fitting procedure of MMPP to traffic traces is presented. Our methodology of doing this is to examine whether the MMPP can be used to predict the performance of a queue to which MMPP sample paths and measured traffic traces are fed for comparison respectively, in addition to the goodness-of-fit test of MMPP. Numerical results and simulations show that the fitted MMPP can approximate multifractal traffic quite well, i.e. accurately predict the queueing performance.

THE CONSTRUCTION OF MULTITYPE CANONICAL MARKOV BRANCHING CHAINS IN RANDOM ENVIRONMENTS

The investigation for branching processes has a long history by their strong physics background, but only a few authors have investigated the branching processes in random environments. First of all, the author introduces the concepts of the multitype canonical Markov branching chain in random environment （CMBCRE） and multitype Markov branching chain in random environment （MBCRE） and proved that CMBCRE must be MBCRE, and any MBCRE must be equivalent to another CMBCRE in distribution. The main results of this article are the construction of CMBCRE and some of its probability properties.

Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes

A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

SMALL PERTURBATION CRAMER METHODS AND MODERATE DEVIATIONS FOR MARKOV PROCESSES

This paper presents a small perturbation Cramer method for obtaining the large deviation principle of a family of measures (β,ε> 0) on a topological vector space. As an application, we obtain the moderate deviation estimations for uniformly ergodic Markov processes.

MOMENTS AND LARGE DEVIATIONS FOR SUPERCRITICAL BRANCHING PROCESSES WITH IMMIGRATION IN RANDOM ENVIRONMENTS

Let(Z_(n))be a branching process with immigration in a random environmentξ,whereξis an independent and identically distributed sequence of random variables.We show asymptotic properties for all the moments of Z_(n) and describe the decay rates of the n-step transition probabilities.As applications,a large deviation principle for the sequence log Z_(n) is established,and related large deviations are also studied.

MOMENTS OF CONTINUOUS-STATE BRANCHING PROCESSES IN LéVY RANDOM ENVIRONMENTS

For continuous-state branching processes in Lévy random environments, the recursion of n-moments and the equivalent condition for the existence of general f-moments are established, where f is a positive continuous function satisfying some standard conditions.

THE EQUILIBRIUM PROBLEM AND CAPACITY FOR JUMP MARKOV PROCESSES

Let X=(Omega,F,F-t,X(t),theta(t),P-x) be a jump Markov process with q-pair q(x)-q(x, A). In this paper, the equilibrium principle is established and equilibrium functions, energy, capacity and related problems is investigated in terms of the q-pair q(x)-q(x, A).

The Laplace Functional and Moments for Markov Branching Chains in Random Environments

The concepts of random Markov matrix, Markov branching chain in randomenvironment (MBCRE) and Laplace functional of Markov branching chain in random environment (LFMBCRE)are introduced. The properties of LFMBCRE and the explicit formulas of momentsof MBCRE are given.

Convergence of Invariant Measures of Truncation Approximations to Markov Processes

Let Q be the Q-matrix of an irreducible, positive recurrent Markov process on a countable state space. We show that, under a number of conditions, the stationary distributions of the n × n north-west corner augmentations of Q converge in total variation to the stationary distribution of the process. Two conditions guaranteeing such convergence include exponential ergodicity and stochastic monotonicity of the process. The same also holds for processes dominated by a stochastically monotone Markov process. In addition, we shall show that finite perturbations of stochastically monotone processes may be viewed as being dominated by a stochastically monotone process, thus extending the scope of these results to a larger class of processes. Consequently, the augmentation method provides an attractive, intuitive method for approximating the stationary distributions of a large class of Markov processes on countably infinite state spaces from a finite amount of known information.

A note on asymptotic behavior of Galton-Watson branching processes in random environments

In this paper, we investigate Galton-Watson branching processes in random environments. In the case where the environmental process is a Markov chain which is positive recurrent or has a transition matrix Q (θ,α) such that sup_θ Q (θ,α)> 0 for some α, we prove that the model has the asymptotic behavior being similar to that of Galton-Watson branching processes. In other case where the environments are non-stationary independent, the sufficient conditions are obtained for certain extinction and uncertain extinction for the model.

THE ASYMPTOTIC PROPERTIES OF SUPERCRITICAL BISEXUAL GALTON-WATSON BRANCHING PROCESSES WITH IMMIGRATION OF MATING UNITS

In this article the supercritical bisexual Galton-Watson branching processes with the immigration of mating units is considered. A necessary condition for the almost sure convergence, and a sufficient condition for the L^1 convergence are given for the process with the suitably normed condition.

ON THE HITTING PROBABILITY AND POLARITY FOR A CLASS OF SELF-SIMILAR MARKOV PROCESSES

The anthem investigate the hitting probability, polarity and the relationship between the polarity and Hausdorff dimension for self-similar Markov processes with state space (0, infinity) and increasing path.

ON THE BASIC REPRODUCTION NUMBER OF GENERAL BRANCHING PROCESSES

Under a very general condition （TNC condition） we show that the spectral radius of the kernel of a general branching process is a threshold parameter and hence plays a role as the basic reproduction number in usual CMJ processes. We discuss also some properties of the extinction probability and the generating operator of general branching processes. As an application in epidemics, in the final section we suggest a generalization of SIR model which can describe infectious diseases transmission in an inhomogeneous population.

Variance minimization for continuous-time Markov decision processes: two approaches

This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.

Wind power time series simulation model based on typical daily output processes and Markov algorithm

The simulation of wind power time series is a key process in renewable power allocation planning,operation mode calculation,and safety assessment.Traditional single-point modeling methods discretely generate wind power at each moment;however,they ignore the daily output characteristics and are unable to consider both modeling accuracy and efficiency.To resolve this problem,a wind power time series simulation model based on typical daily output processes and Markov algorithm is proposed.First,a typical daily output process classification method based on time series similarity and modified K-means clustering algorithm is presented.Second,considering the typical daily output processes as status variables,a wind power time series simulation model based on Markov algorithm is constructed.Finally,a case is analyzed based on the measured data of a wind farm in China.The proposed model is then compared with traditional methods to verify its effectiveness and applicability.The comparison results indicate that the statistical characteristics,probability distributions,and autocorrelation characteristics of the wind power time series generated by the proposed model are better than those of the traditional methods.Moreover,modeling efficiency considerably improves.

Robust analysis of discounted Markov decision processes with uncertain transition probabilities

Optimal policies in Markov decision problems may be quite sensitive with regard to transition probabilities.In practice,some transition probabilities may be uncertain.The goals of the present study are to find the robust range for a certain optimal policy and to obtain value intervals of exact transition probabilities.Our research yields powerful contributions for Markov decision processes(MDPs)with uncertain transition probabilities.We first propose a method for estimating unknown transition probabilities based on maximum likelihood.Since the estimation may be far from accurate,and the highest expected total reward of the MDP may be sensitive to these transition probabilities,we analyze the robustness of an optimal policy and propose an approach for robust analysis.After giving the definition of a robust optimal policy with uncertain transition probabilities represented as sets of numbers,we formulate a model to obtain the optimal policy.Finally,we define the value intervals of the exact transition probabilities and construct models to determine the lower and upper bounds.Numerical examples are given to show the practicability of our methods.

CONTINUOUS TIME MIXED STATE BRANCHING PROCESSES AND STOCHASTIC EQUATIONS

A continuous time and mixed state branching process is constructed by a scaling limit theorem of two-type Galton-Watson processes.The process can also be obtained by the pathwise unique solution to a stochastic equation system.From the stochastic equation system we derive the distribution of local jumps and give the exponential ergodicity in Wasserstein-type distances of the transition semigroup.Meanwhile,we study immigration structures associated with the process and prove the existence of the stationary distribution of the process with immigration.

Stability Estimation for Markov Control Processes with Discounted Cost

This article explores controllable Borel spaces, stationary, homogeneous Markov processes, discrete time with infinite horizon, with bounded cost functions and using the expected total discounted cost criterion. The problem of the estimation of stability for this type of process is set. The central objective is to obtain a bounded stability index expressed in terms of the Lévy-Prokhorov metric;likewise, sufficient conditions are provided for the existence of such inequalities.

MOMENTS OF PASSAGE TIMES AND ASYMPTOTIC BEHAVIOR OF INCREASING SELF-SIMILAR MARKOV PROCESSES

By using Lamperti＇s bijection between self-similar Markov processes and L@vy processes~ we prove finiteness of moments and asymptotic behavior of passage times for increasing self-similar Markov processes valued in （0, ~）. We Mso investigate the behavior of the process when it crosses a level. A limit theorem concerning the distribution of the process immediately before it crosses some level is proved. Some useful examples are given.