Integral-type functionals of first hitting times for continuous-time Markov chains

We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.

Ergodic degrees for continuous-time Markov chains

This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.

Contraction Integrated Semigroups and Their Applicationto Continuous-time Markov Chains

We introduce the notion of the contraction integrated semigroups and give the Lumber-Phillips characterization of the generator, and also the charaterazied generators of isometric integrated semigroups. For their application, a necessary and sufficient condition for q-matrices Q generating a contraction integrated semigroup is given, and a necessary and sufficient condition for a transition function to be a Feller-Reuter-Riley transition function is also given in terms of its q-matrix.

Perturbation analysis for continuous-time Markov chains

We investigate perturbation for continuous-time Markov chains(CTMCs) on a countable state space. Explicit bounds on ?D and D are derived in terms of a drift condition, where ? and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu(2012) for uniformly bounded CTMCs to general(possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.

Mathematical Modeling of Possibility Markov Chains by Possibility Theory

Statistical regression models are input-oriented estimation models that account for observation errors. On the other hand, an output-oriented possibility regression model that accounts for system fluctuations is proposed. Furthermore, the possibility Markov chain is proposed, which has a disidentifiable state (posterior) and a nondiscriminable state (prior). In this paper, we first take up the entity efficiency evaluation problem as a case study of the posterior non-discriminable production possibility region and mention Fuzzy DEA with fuzzy constraints. Next, the case study of the ex-ante non-discriminable event setting is discussed. Finally, we introduce the measure of the fuzzy number and the equality relation and attempt to model the possibility Markov chain mathematically. Furthermore, we show that under ergodic conditions, the direct sum state can be decomposed and reintegrated using fuzzy OR logic. We had already constructed the Possibility Markov process based on the indifferent state of this world. In this paper, we try to extend it to the indifferent event in another world. It should be noted that we can obtain the possibility transfer matrix by full use of possibility theory.

Variance minimization for continuous-time Markov decision processes: two approaches

This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.

Development of Optimal Maintenance Policies for Offshore Wind Turbine Gearboxes Based on the Non-homogeneous Continuous-Time Markov Process

Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.

Variance Optimization for Continuous-Time Markov Decision Processes

This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.

多重填补法Markov Chain Monte Carlo模型在有缺失值的妇幼卫生纵向数据中的应用

目的 针对妇幼卫生纵向数据的任意缺失模式,采用多重填补方法进行填补,探求最佳填补结果,以便对数据作进一步分析与研究。方法 运用SAS9.0 ,采用多重填补方法Markov China Monte Carlo(MCMC)模型对缺失数据进行多次填补并综合分析。结果 填补5次所得结果最优。结论 多重填补方法可以处理有缺失数据资料中的许多普遍问题,可提高统计效率,尤其是MCMC模型在处理复杂的缺失数据上,优势明显。

基于Markov Chain的协议异常检测模型

本文介绍了基于Markov链的协议异常检测模型,此外,通过对MIT Lincoln实验室1999评估数据的分析,证明此模型的正确性和有效性。

基于MLP-ANN与Markov Chain的土地利用变化预测方法——以锡林郭勒盟为例

以北方草地典型地区—内蒙古锡林郭勒盟为案例区,在1995年到2000年的土地利用变化与驱动力分析的基础上,利用土地利用转换类型和驱动力模型,采用多层感知人工神经网络模型分析了各种土地利用类型未来的转换潜力;利用马尔可夫链模型,预测了2005和2010年土地利用格局。预测结果显示:高覆盖度草地减少幅度最大,中覆盖度草地减少相对和缓,高、中覆盖度草地的减少造成了未利用地和低覆盖度草地的增加,尤其是前者增加的幅度最大;从空间分布看,高覆盖度草地的减少集中在西北部地区,主要转变为中低覆盖度草地,中覆盖度草地的减少主要集中在西南部地区,其流向主要是以沙化土地为主的未利用地;案例研究表明,多层感知人工神经网络模型与马尔可夫链模型的结合与应用能够在很大程度上预测稳定驱动力作用下的土地利用变化趋势,从而为生态干预提供指导。

高等院校教师人才流动的Markov-chain预测模型

介绍并解析随机过程理论中的马尔科夫过程与马尔科夫链;针对高等院校教师人才流动变化过程,利用马尔科夫过程分析方法建立描述人员流动变化趋势的Markov-chain预测模型,详细阐述了模型的算法步骤.以某所高等院校教师人才流动的状态转移数据作为算例,运用新建立的预测模型,对该院校教师人才的流动趋势做出了预测分析.最后,将教师进修状态纳入分析范围,进行了教师职业生涯和职务发展趋势预测的深入分析.应用模型对实际算例的求解结果表明:Markov-chain预测模型及算法,叙述简洁、运算方便,为高等院校教师人才流动,乃至其他行业人才流动的预测提供了一种新的、有效的思路和方法.

基于Markov chain理论的科技成果转化的市场预测

科技成果成功转化的主要标志是其产品的市场畅销度。产品的畅销度由于受各种内外部不确定因素影响,具有随机不确定性。基于这种随机不确定性,运用Markovchain对其产品进行市场预测,并通过实证分析说明其有效性,为科技成果的转化提供具体可行的定量依据。

基于RS-SPA-Markov chain的生物质气化站安全评价与预测模型研究

本文在资料收集、实地考察、专家评估的基础上,建立了相对通用的生物质气化站安全评价指标体系,引入了RS-SPA-Markov chain安全评价与预测模型。通过实例应用,证明了该模型应用于生物质气化站安全评价与预测的可行性和科学性,为生物质气化站安全评价与预测提供了一种新的途径。

Auto-selection order of Markov chain for background sequences with chi-square test

Modeling non coding background sequences appropriately is important for the detection of regulatory elements from DNA sequences. Based on the chi square statistic test, some explanations about why to choose higher order Markov chain model and how to automatically select the proper order are given in this paper. The chi square test is first run on synthetic data sets to show that it can efficiently find the proper order of Markov chain. Using chi square test, distinct higher order context dependences inherent in ten sets of sequences of yeast S.cerevisiae from other literature have been found. So the Markov chain with higher order would be more suitable for modeling the non coding background sequences than an independent model.

Applicability of Markov chain-based stochastic model for bubbling fluidized beds

A Markov chain-based stochastic model （MCM） is developed to simulate the movement of particles in a 2D bubbling fluidized bed （BFB）. The state spaces are determined by the discretized physical cells of the bed, and the transition probability matrix is directly calculated by the results of a discrete element method （DEM） simulation. The Markov property of the BFB is discussed by the comparison results calculated from both static and dynamic transition probability matrices. The static matrix is calculated based on the Markov chain while the dynamic matrix is calculated based on the memory property of the particle movement. Results show that the difference in the trends of particle movement between the static and dynamic matrix calculation is very small. Besides, the particle mixing curves of the MCM and DEM have the same trend and similar numerical values, and the details show the time averaged characteristic of the MCM and also expose its shortcoming in describing the instantaneous particle dynamics in the BFB.

基于Markov Chain Monte Carlo模型对医院调查资料中缺失数据的多重估算

目的探讨基于Markov Chain Monte Carlo(MCMC)模型的多重估算法在处理医院调查资料缺失数据中的应用。方法运用SAS9.2编写程序,在分析数据的分布类型和缺失机制的基础上,采用MCMC法对缺失数据进行多次填补和联合统计推断,分析多重估算法的优势。结果数据服从多元正态分布与随机缺失,采用MCMC法填补10次所得的结果最佳。结论多重估算既可反映缺失数据的不确定性,又可充分利用现有资料的信息、提高统计效率、对模型的估计结果更加可信,是处理缺失数据的有效方法。

D-S理论和Markov链组合的桥梁性能退化预测研究

为准确预测桥梁性能退化,考虑到数据随机性和微小扰动发生状态跳跃,提出了一种D-S(Dempster-Shafer)证据理论和Markov链组合的桥梁性能退化组合预测模型和性能退化率的概念.该模型基于指数平滑(exponential smoothing,ES)方法获得新的预测数据序列,并利用Markov链和D-S理论不断进行优化,从而实现桥梁性能退化的组合预测.实际工程的应用结果表明:性能退化率可以直观地表征在梁性能退化的速度.其次,该模型的平均相对误差为1.54%,较于回归、灰色和模糊加权Markov链模型,精度分别提高了1.11%,0.88%和2.8%,而后验差比值为0.242,小于0.35;模型的标准差为9.021,相比其他模型分别减小了3.978,3.405和7.500,而变异系数为0.109,均小于其他模型,验证了组合预测模型在精度和稳定性方面的优越性,可为在役桥梁结构性能退化预测与维护提供理论基础.

Basic Lithofacies-Succession Model for the Wumishan Cyclothems: Their Markov Chain Analysis and Regularly Vertical Stacking Patterns in the Third-Order Sequences

The Mesoproterozoic Wumishan Formation in the Jixian section of Tianjin is a succession of 3300-m-thick carbonate strata formed in a period of about 100 Ma (1310±20 Ma-1207±10 Ma). In this succession of strata, the carbonate metre-scale cyclic sequences belonging to peritidal type with an approximately symmetrical lithofacies-succession are best developed. The wide development of 1:4 stacking patterns shows that these metre-scale cyclic sequences are genetically related to the short-eccentricity cycles, which are called the Wumishan cyclothems that could truly represent sedimentary cycles. Generally, massive and thick-bedded calcareous dolomites and dolomitic limestones of stromatolite biostromes and thrombolite bioherms constitute the central part of the Wumishan cyclothems. The lower and upper parts consist of tidal flat dolostones, sandy-muddy dolostone and the top part is composed of lagoonal facies dolomitic shales with a paleosol cap. Therefore, an approximately symmetrical lithofacies-succession is formed. Many features such as the clear deepening and shoaling vectors of cyclothems, and all kinds of marks of fresh-water diagenesis indicate that the Wumishan cyclothems are the product of autocyclic sedimentation governed by allocyclic high-frequency sea-level changes. The results of a Markov chain analysis reaffirm the existence of the lithofacies-succession model of the Wumishan cyclothems. The boundaries of the Wumishan cyclothems are marked by the instantaneous exposed punctuated surface, which leads to the discrepancy between the cyclothems and the parasequences of the sequence stratigraphy terminology system. It is difficult to form a judgment that the time span of the Milankovitch cycles in the Precambrian is certainly equal to that of the Phanerozoic, but the regularly vertical stacking patterns of the seventh-order rhythms, sixth-order cyclothems and fifth-order parasequence sets still indicate their consistency with the duration of the Milankovitch cycles in the Phanerozoic.

The Relations Among Various Markov Chains

Some basic equations and the relations among various Markov chains are established. These works are the bases in the investigation of the theory of Markov chain in random environment.