Ergodic degrees for continuous-time Markov chains

This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.

Integral-type functionals of first hitting times for continuous-time Markov chains

We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.

Contraction Integrated Semigroups and Their Applicationto Continuous-time Markov Chains

We introduce the notion of the contraction integrated semigroups and give the Lumber-Phillips characterization of the generator, and also the charaterazied generators of isometric integrated semigroups. For their application, a necessary and sufficient condition for q-matrices Q generating a contraction integrated semigroup is given, and a necessary and sufficient condition for a transition function to be a Feller-Reuter-Riley transition function is also given in terms of its q-matrix.

Perturbation analysis for continuous-time Markov chains

We investigate perturbation for continuous-time Markov chains(CTMCs) on a countable state space. Explicit bounds on ?D and D are derived in terms of a drift condition, where ? and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu(2012) for uniformly bounded CTMCs to general(possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.

Markov Chains Based on Random Generalized 1-Flipper Operations for Connected Regular Multi-digraphs

The properties of generalized flip Markov chains on connected regular digraphs are discussed.The 1-Flipper operation on Markov chains for undirected graphs is generalized to that for multi-digraphs.The generalized 1-Flipper operation preserves the regularity and weak connectivity of multi-digraphs.The generalized 1-Flipper operation is proved to be symmetric.Moreover,it is presented that a series of random generalized 1-Flipper operations eventually lead to a uniform probability distribution over all connected d-regular multi-digraphs without loops.

Air Quality Estimation Using Nonhomogeneous Markov Chains: A Case Study Comparing Two Rules Applied to Mexico City Data

A nonhomogeneous Markov chain is applied to the study of the air quality classification in Mexico City when the so-called criterion pollutants are used. We consider the indices associated with air quality using two regulations where different ways of classification are taken into account. Parameters of the model are the initial and transition probabilities of the chain. They are estimated under the Bayesian point of view through samples generated directly from the corresponding posterior distributions. Using the estimated parameters, the probability of having an air quality index in a given hour of the day is obtained.

D-S理论和Markov链组合的桥梁性能退化预测研究

为准确预测桥梁性能退化,考虑到数据随机性和微小扰动发生状态跳跃,提出了一种D-S(Dempster-Shafer)证据理论和Markov链组合的桥梁性能退化组合预测模型和性能退化率的概念.该模型基于指数平滑(exponential smoothing,ES)方法获得新的预测数据序列,并利用Markov链和D-S理论不断进行优化,从而实现桥梁性能退化的组合预测.实际工程的应用结果表明:性能退化率可以直观地表征在梁性能退化的速度.其次,该模型的平均相对误差为1.54%,较于回归、灰色和模糊加权Markov链模型,精度分别提高了1.11%,0.88%和2.8%,而后验差比值为0.242,小于0.35;模型的标准差为9.021,相比其他模型分别减小了3.978,3.405和7.500,而变异系数为0.109,均小于其他模型,验证了组合预测模型在精度和稳定性方面的优越性,可为在役桥梁结构性能退化预测与维护提供理论基础.

The Relations Among Various Markov Chains

Some basic equations and the relations among various Markov chains are established. These works are the bases in the investigation of the theory of Markov chain in random environment.

Weighted Markov chains for forecasting and analysis in Incidence of infectious diseases in jiangsu Province,China

This paper first applies the sequential cluster method to set up the classification standard of infectious disease incidence state based on the fact that there are many uncertainty characteristics in the incidence course.Then the paper presents a weighted Markov chain,a method which is used to predict the future incidence state.This method assumes the standardized self-coefficients as weights based on the special characteristics of infectious disease incidence being a dependent stochastic variable.It also analyzes the characteristics of infectious diseases incidence via the Markov chain Monte Carlo method to make the long-term benefit of decision optimal.Our method is successfully validated using existing incidents data of infectious diseases in Jiangsu Province.In summation,this paper proposes ways to improve the accuracy of the weighted Markov chain,specifically in the field of infection epidemiology.

Intrusion detection based on system calls and homogeneous Markov chains

A novel method for detecting anomalous program behavior is presented, which is applicable to hostbased intrusion detection systems that monitor system call activities. The method constructs a homogeneous Markov chain model to characterize the normal behavior of a privileged program, and associates the states of the Markov chain with the unique system calls in the training data. At the detection stage, the probabilities that the Markov chain model supports the system call sequences generated by the program are computed. A low probability indicates an anomalous sequence that may result from intrusive activities. Then a decision rule based on the number of anomalous sequences in a locality frame is adopted to classify the program＇s behavior. The method gives attention to both computational efficiency and detection accuracy, and is especially suitable for on-line detection. It has been applied to practical host-based intrusion detection systems.

ON MARKOV CHAINS IN SPACE-TIME RANDOM ENVIRONMENTS

In Section 1, the authors establish the models of two kinds of Markov chains in space-time random environments (MCSTRE and MCSTRE(+)) with abstract state space. In Section 2, the authors construct a MCSTRE and a MCSTRE(+) by an initial distribution Φ and a random Markov kernel (RMK) p(γ). In Section 3, the authors es-tablish several equivalence theorems on MCSTRE and MCSTRE(+). Finally, the authors give two very important examples of MCMSTRE, the random walk in spce-time random environment and the Markov br...

THE CONSTRUCTION OF MULTITYPE CANONICAL MARKOV BRANCHING CHAINS IN RANDOM ENVIRONMENTS

The investigation for branching processes has a long history by their strong physics background, but only a few authors have investigated the branching processes in random environments. First of all, the author introduces the concepts of the multitype canonical Markov branching chain in random environment （CMBCRE） and multitype Markov branching chain in random environment （MBCRE） and proved that CMBCRE must be MBCRE, and any MBCRE must be equivalent to another CMBCRE in distribution. The main results of this article are the construction of CMBCRE and some of its probability properties.

STRONG LAW OF LARGE NUMBERS AND SHANNON-MCMILLAN THEOREM FOR MARKOV CHAINS FIELD ON CAYLEY TREE

This paper studies the strong law of large numbers and the Shannom-McMillan theorem for Markov chains field on Cayley tree. The authors first prove the strong law of large number on the frequencies of states and orderd couples of states for Markov chains field on Cayley tree. Then they prove the Shannon-McMillan theorem with a.e. convergence for Markov chains field on Cayley tree. In the proof, a new technique in the study the strong limit theorem in probability theory is applied.

Model of Markov Chains in Space-Time Random Environments

A general framework of stochastic model for a Markov chain in a space-time random environment is introduced, here the environment ξ^＊：={ξ1,x∈N,x∈ X}is a random field. We study the dependence relations between the environment and the original chain, especially the ＂feedback＂. Some equivalence theorems and law of large numbers are obtained.

The Laplace Functional and Moments for Markov Branching Chains in Random Environments

The concepts of random Markov matrix, Markov branching chain in randomenvironment (MBCRE) and Laplace functional of Markov branching chain in random environment (LFMBCRE)are introduced. The properties of LFMBCRE and the explicit formulas of momentsof MBCRE are given.

Modeling urban land use dynamics using Markov-chain and cellular automata in Gondar City,Northwest Ethiopia

Modeling urban land-use dynamics is critical for urban experts’and infrastructure managers’planning.This study attempts to explore the land-use/land-cover(LULC)dynamics of Gondar using satellite images from 1984 to 2020.Markov-Chain and Cellular Automata(MC-CA)models have been recognized as performing well in predicting urban land-use change.However,only a few models work in Ethiopia in general,and no study in Gondar has applied this approach to study urban land-use patterns.Therefore,Gondar land-use/land cover changes of Gondar were predicted using the MC-CA model in IDRISI.The built-up area in Gondar city covered 1413 ha(3%of the total area)in 1984 and increased to 2380 ha(5%)in 1994;21153 ha(45.5%)in 2004;22622 ha(48.7%)in 2014;and 23427 ha(50.5%)in 2020.The area has been predicted to reach 57.5%in the 2050s,showing a faster increase that will cause a very vast loss of farmland.This will increase urban sprawl challenges as well as overall environmental disequilibrium in the preceding decade.Thus,innovative and careful structures and systems in urban planning are required to secure a sustainable urban future and to make our cities livable and competitive in the paradigm of sustainable cities.

Variance minimization for continuous-time Markov decision processes: two approaches

This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.

Fully Polarimetric Land Cover Classification Based on Markov Chains

A novel land cover classification procedure is presented utilizing the information content of fully polarimetric SAR images. The Cameron coherent target decomposition (CTD) is employed to characterize land cover pixel by pixel. Cameron’s CTD is employed since it provides a complete set of elementary scattering mechanisms to describe the physical properties of the scatterer. The novelty of the proposed land classification approach lies on the fact that the features used for classification are not the types of the elementary scatterers themselves, but the way these types of scatterers alternate from pixel to pixel on the SAR image. Thus, transition matrices that represent local Markov models are used as classification features for land cover classification. The classification rule employs only the most important transitions for decision making. The Frobenius inner product is employed as similarity measure. Ten different types of land cover are used for testing the proposed method. In this aspect, the classification performance is significantly high.

Ecological Classroom Model for Vocabulary Teaching Based on Markov Chain

This paper explores the current English vocabulary teaching classroom based on an eco-linguistic perspective and the Markov chain,points out the ecological imbalance in the English vocabulary classroom,and proposes measures to solve the problem and strategies to build an ecological English vocabulary learning classroom,in order to provide some reference for future English vocabulary curriculum reform and curriculum design.In this paper,a random sample of 20 students from each class was taken from an advanced arts class and an advanced science class.The results show that after a semester of teaching process,the overall English scores of the arts and science classes improved,and since,the arts classes improved more and the teachers’teaching quality was better.

Oil-gas reservoir lithofacies stochastic modeling based on one- to three-dimensional Markov chains

随机建模方法已经被广泛地应用于油气储层岩相的模拟中。然而,基于马尔科夫链的模拟技术仍然处于发展和完善中。这主要是由于在合理定义多维马尔科夫链的条件概率和水平方向上的转移概率时存在困难。本文着力解决这些问题。首先,基于完全独立假设和条件独立假设推导出了多维马尔科夫链条件概率的计算公式,并指出因为基于条件独立假设的多维马尔科夫链解决了小类过度小估计问题,所以更为合理。然后,给出了计算转移概率的方法:垂直方向上的转移概率可以通过统计井数据中的转移频数获取,水平方向上的转移概率则基于井数据和延伸率运用Walther定律估算得到。最后,运用提出的模型对中国塔河油田储层岩相的分布进行随机模拟。结果表明:与完全独立假设相比,基于条件独立假设的随机模拟结果更接近真实的岩相比例并能更好地重现岩相的空间特征。