Air Quality Estimation Using Nonhomogeneous Markov Chains: A Case Study Comparing Two Rules Applied to Mexico City Data

A nonhomogeneous Markov chain is applied to the study of the air quality classification in Mexico City when the so-called criterion pollutants are used. We consider the indices associated with air quality using two regulations where different ways of classification are taken into account. Parameters of the model are the initial and transition probabilities of the chain. They are estimated under the Bayesian point of view through samples generated directly from the corresponding posterior distributions. Using the estimated parameters, the probability of having an air quality index in a given hour of the day is obtained.

Markov Chains Based on Random Generalized 1-Flipper Operations for Connected Regular Multi-digraphs

The properties of generalized flip Markov chains on connected regular digraphs are discussed.The 1-Flipper operation on Markov chains for undirected graphs is generalized to that for multi-digraphs.The generalized 1-Flipper operation preserves the regularity and weak connectivity of multi-digraphs.The generalized 1-Flipper operation is proved to be symmetric.Moreover,it is presented that a series of random generalized 1-Flipper operations eventually lead to a uniform probability distribution over all connected d-regular multi-digraphs without loops.

Weighted Markov chains for forecasting and analysis in Incidence of infectious diseases in jiangsu Province,China

This paper first applies the sequential cluster method to set up the classification standard of infectious disease incidence state based on the fact that there are many uncertainty characteristics in the incidence course.Then the paper presents a weighted Markov chain,a method which is used to predict the future incidence state.This method assumes the standardized self-coefficients as weights based on the special characteristics of infectious disease incidence being a dependent stochastic variable.It also analyzes the characteristics of infectious diseases incidence via the Markov chain Monte Carlo method to make the long-term benefit of decision optimal.Our method is successfully validated using existing incidents data of infectious diseases in Jiangsu Province.In summation,this paper proposes ways to improve the accuracy of the weighted Markov chain,specifically in the field of infection epidemiology.

STRONG LAW OF LARGE NUMBERS AND SHANNON-MCMILLAN THEOREM FOR MARKOV CHAINS FIELD ON CAYLEY TREE

This paper studies the strong law of large numbers and the Shannom-McMillan theorem for Markov chains field on Cayley tree. The authors first prove the strong law of large number on the frequencies of states and orderd couples of states for Markov chains field on Cayley tree. Then they prove the Shannon-McMillan theorem with a.e. convergence for Markov chains field on Cayley tree. In the proof, a new technique in the study the strong limit theorem in probability theory is applied.

Model of Markov Chains in Space-Time Random Environments

A general framework of stochastic model for a Markov chain in a space-time random environment is introduced, here the environment ξ^＊：={ξ1,x∈N,x∈ X}is a random field. We study the dependence relations between the environment and the original chain, especially the ＂feedback＂. Some equivalence theorems and law of large numbers are obtained.

Fully Polarimetric Land Cover Classification Based on Markov Chains

A novel land cover classification procedure is presented utilizing the information content of fully polarimetric SAR images. The Cameron coherent target decomposition (CTD) is employed to characterize land cover pixel by pixel. Cameron’s CTD is employed since it provides a complete set of elementary scattering mechanisms to describe the physical properties of the scatterer. The novelty of the proposed land classification approach lies on the fact that the features used for classification are not the types of the elementary scatterers themselves, but the way these types of scatterers alternate from pixel to pixel on the SAR image. Thus, transition matrices that represent local Markov models are used as classification features for land cover classification. The classification rule employs only the most important transitions for decision making. The Frobenius inner product is employed as similarity measure. Ten different types of land cover are used for testing the proposed method. In this aspect, the classification performance is significantly high.

THE CONVERGENCE OF NONHOMOGENEOUS MARKOV CHAINS IN GENERAL STATE SPACES BY THE COUPLING METHOD

We investigate the convergence of nonhomogeneous Markov chains in general state space by using the f norm and the coupling method,and thus,a sufficient condition for the convergence of nonhomogeneous Markov chains in general state space is obtained.

Using Markov chains to predict the natural progression of diabetic retinopathy

AIM: To study the natural progression of diabetic retinopathy in patients with type 2 diabetes.METHODS: This was an observational study of 153 cases with type 2 diabetes from 2010 to 2013. The state of patient was noted at end of each year and transition matrices were developed to model movement between years. Patients who progressed to severe non-proliferative diabetic retinopathy(NPDR) were treated.Markov Chains and Chi-square test were used for statistical analysis.RESULTS: We modelled the transition of 153 patients from NPDR to blindness on an annual basis. At the end of year 3, we compared results from the Markov model versus actual data. The results from Chi-square test confirmed that there was statistically no significant difference(P =0.70) which provided assurance that the model was robust to estimate mean sojourn times. The key finding was that a patient entering the system in mild NPDR state is expected to stay in that state for 5y followed by 1.07 y in moderate NPDR, be in the severe NPDR state for 1.33 y before moving into PDR for roughly8 y. It is therefore expected that such a patient entering the model in a state of mild NPDR will enter blindness after 15.29 y.CONCLUSION: Patients stay for long time periods in mild NPDR before transitioning into moderate NPDR.However, they move rapidly from moderate NPDR to proliferative diabetic retinopathy(PDR) and stay in that state for long periods before transitioning into blindness.

POISSON LIMIT THEOREM FOR COUNTABLE MARKOV CHAINS IN MARKOVIAN ENVIRONMENTS

A countable Markov chain in a Markovian environment is considered.A Poisson limit theorem for the chain recurring to small cylindrical sets is mainly achieved.In order to prove this theorem,the entropy function h is introduced and the Shannon-McMillan-Breiman theorem for the Markov chain in a Markovian environment is shown. It's well-known that a Markov process in a Markovian environment is generally not a standard Markov chain,so an example of Poisson approximation for a process which is not a Markov process is given.On the other hand,when the environmental process degenerates to a constant sequence,a Poisson limit theorem for countable Markov chains,which is the generalization of Pitskel's result for finite Markov chains is obtained.

A Note on Markov Chains in Stationary Random Environments

We consider Markov chains in stationary random environments. The conservative set C of the corresponding skew Markov chain of this process can be thought of as a recurrent set of a standard Markov chain. In some simpler cases, we give some sufficient conditions under which the conservative set C can be decomposed into at most countable minimal closed sets.

Evaluation of Growth Area Foliar Red Tomato Crop Using Markov Chains

The importance of evaluating the leaf area in red tomato plants aims to determine the growth and development of crops established two production cycles, spring-summer and autumn-winter to compare the influence of temperature on the growth of leaf area. Repeated, weekly samples were taken by identifying the week and determining the growth and leaf area development using Markov chains, using an array of transition to describe and represent in a flowchart the finite number of physiological States. With the analysis in the steady state process and applying the equations of odds, we get that leaf area growth is established from the seventh week shown in the first cycle (C1) with the chance of 0.266, 0.264 and 0.263, in the last two weeks. It was observed an increase of 6% in the cycle autumn-winter cycle compared spring-summer.

The Relations Among Various Markov Chains

Some basic equations and the relations among various Markov chains are established. These works are the bases in the investigation of the theory of Markov chain in random environment.

Intrusion detection based on system calls and homogeneous Markov chains

A novel method for detecting anomalous program behavior is presented, which is applicable to hostbased intrusion detection systems that monitor system call activities. The method constructs a homogeneous Markov chain model to characterize the normal behavior of a privileged program, and associates the states of the Markov chain with the unique system calls in the training data. At the detection stage, the probabilities that the Markov chain model supports the system call sequences generated by the program are computed. A low probability indicates an anomalous sequence that may result from intrusive activities. Then a decision rule based on the number of anomalous sequences in a locality frame is adopted to classify the program＇s behavior. The method gives attention to both computational efficiency and detection accuracy, and is especially suitable for on-line detection. It has been applied to practical host-based intrusion detection systems.

ON MARKOV CHAINS IN SPACE-TIME RANDOM ENVIRONMENTS

In Section 1, the authors establish the models of two kinds of Markov chains in space-time random environments (MCSTRE and MCSTRE(+)) with abstract state space. In Section 2, the authors construct a MCSTRE and a MCSTRE(+) by an initial distribution Φ and a random Markov kernel (RMK) p(γ). In Section 3, the authors es-tablish several equivalence theorems on MCSTRE and MCSTRE(+). Finally, the authors give two very important examples of MCMSTRE, the random walk in spce-time random environment and the Markov br...

Waiting times and stopping probabilities for patterns in Markov chains

Suppose that C is a finite collection of patterns. Observe a Markov chain until one of the patterns in C occurs as a run. This time is denoted by τ. In this paper, we aim to give an easy way to calculate the mean waiting time E(τ) and the stopping probabilities P(τ = τA)with A ∈ C, where τA is the waiting time until the pattern A appears as a run.

The Prognosis of Delayed Reactions in Rats Using Markov Chains Method

The original modified method of the direct delayed reaction has been used for the evaluation of food-obtaining strategy across spatial learning tasks in T-maze alternation. The optimal behavioral algorithms for each experimental day have been identified so that the animals obtain maximum possible food amount with minimal number of mistakes. Markov chain method has been used for the prognosis of rat’s behavioral strategy during the spatial learning task. The learning and decision-making represent the probabilistic transition process where the animal choice at each step (state) depends on the learning experience from previous step (state).

On Markov Chains Induced by Partitioned Transition Probability Matrices

Let S be a denumerable state space and let P be a transition probability matrix on S. If a denumerable set M of nonnegative matrices is such that the sum of the matrices is equal to P, then we call M a partition of P. Let K denote the set of probability vectors on S. With every partition M of P we can associate a transition probability function PM on K defined in such a way that if p ∈ K and M ∈M are such that ｜｜pM｜｜ 〉 0, then, with probability ｜｜pM｜｜ the vector p is transferred to the vector pM/｜｜pM｜｜. Here ｜｜·｜｜ denotes the/1-norm. In this paper we investigate the convergence in distribution for Markov chains generated by transition probability functions induced by partitions of transition probability matrices. The main motivation for this investigation is the application of the convergence results obtained to filtering processes of partially observed Markov chains with denumerable state space.

Contraction Integrated Semigroups and Their Applicationto Continuous-time Markov Chains

We introduce the notion of the contraction integrated semigroups and give the Lumber-Phillips characterization of the generator, and also the charaterazied generators of isometric integrated semigroups. For their application, a necessary and sufficient condition for q-matrices Q generating a contraction integrated semigroup is given, and a necessary and sufficient condition for a transition function to be a Feller-Reuter-Riley transition function is also given in terms of its q-matrix.

Estimation of Spectral Gap for Markov Chains

The study of the convergent rate (spectra gap) in the L^2-sense is motivated from several different fields: probability statistics, mathematical physics, computer science and so on and it is now an artive research topic. Based on a new approach (the coupling technique) introduced in [7] for the estimate of the convergent rate and as a continuation of [4],[5],[7-9],[23] and [24], this paper studies the estimate of the rate for time-continuous Markov chains. Two variational formulas for the rate are presented here for the first time for birth-death processes. For diffusions, similar results are presented in an accompany paper [10]. The new formulas enable us to recover or improve the main known results. The connection between the sharp estimate and the corresponding eigenfunction is explored and illustrated by various examples. A previous result on optimal Markovian couplings is also extended in the paper.

Deviation matrix and asymptotic variance for GI/M/1-type Markov chains

We investigate deviation matrix for discrete-time GI/M/1-type Markov chains in terms of the matrix-analytic method, and revisit the link between deviation matrix and the asymptotic variance. Parallel results are obtained for continuous-time GI/M/1-type Markov chains based on the technique of uniformization. We conclude with A. B. Clarke＇s tandem queue as an illustrative example, and compute the asymptotic variance for the queue length for this model.