Integral-type functionals of first hitting times for continuous-time Markov chains

We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.

Ergodic degrees for continuous-time Markov chains

This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.

Contraction Integrated Semigroups and Their Applicationto Continuous-time Markov Chains

We introduce the notion of the contraction integrated semigroups and give the Lumber-Phillips characterization of the generator, and also the charaterazied generators of isometric integrated semigroups. For their application, a necessary and sufficient condition for q-matrices Q generating a contraction integrated semigroup is given, and a necessary and sufficient condition for a transition function to be a Feller-Reuter-Riley transition function is also given in terms of its q-matrix.

Perturbation analysis for continuous-time Markov chains

We investigate perturbation for continuous-time Markov chains(CTMCs) on a countable state space. Explicit bounds on ?D and D are derived in terms of a drift condition, where ? and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu(2012) for uniformly bounded CTMCs to general(possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.

Mathematical Modeling of Possibility Markov Chains by Possibility Theory

Statistical regression models are input-oriented estimation models that account for observation errors. On the other hand, an output-oriented possibility regression model that accounts for system fluctuations is proposed. Furthermore, the possibility Markov chain is proposed, which has a disidentifiable state (posterior) and a nondiscriminable state (prior). In this paper, we first take up the entity efficiency evaluation problem as a case study of the posterior non-discriminable production possibility region and mention Fuzzy DEA with fuzzy constraints. Next, the case study of the ex-ante non-discriminable event setting is discussed. Finally, we introduce the measure of the fuzzy number and the equality relation and attempt to model the possibility Markov chain mathematically. Furthermore, we show that under ergodic conditions, the direct sum state can be decomposed and reintegrated using fuzzy OR logic. We had already constructed the Possibility Markov process based on the indifferent state of this world. In this paper, we try to extend it to the indifferent event in another world. It should be noted that we can obtain the possibility transfer matrix by full use of possibility theory.

Variance minimization for continuous-time Markov decision processes: two approaches

This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.

Development of Optimal Maintenance Policies for Offshore Wind Turbine Gearboxes Based on the Non-homogeneous Continuous-Time Markov Process

Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.

Application of Cellular Automata and Markov Chain model for urban green infrastructure in Kuala Lumpur,Malaysia

Kuala Lumpur of Malaysia,as a tropical city,has experienced a notable decline in its critical urban green infrastructure(UGI)due to rapid urbanization and haphazard development.The decrease of UGI,especially natural forest and artificial forest,may reduce the diversity of ecosystem services and the ability of Kuala Lumpur to build resilience in the future.This study analyzed land use and land cover(LULC)and UGI changes in Kuala Lumpur based on Landsat satellite images in 1990,2005,and 2021and employed the overall accuracy and Kappa coefficient to assess classification accuracy.LULC was categorized into six main types:natural forest,artificial forest,grassland,water body,bare ground,and built-up area.Satellite images in 1990,2005,and 2021 showed the remarkable overall accuracy values of 91.06%,96.67%,and 98.28%,respectively,along with the significant Kappa coefficient values of 0.8997,0.9626,and 0.9512,respectively.Then,this study utilized Cellular Automata and Markov Chain model to analyze the transition of different LULC types during 1990-2005 and 1990-2021 and predict LULC types in 2050.The results showed that natural forest decreased from 15.22%to 8.20%and artificial forest reduced from 18.51%to 15.16%during 1990-2021.Reductions in natural forest and artificial forest led to alterations in urban surface water dynamics,increasing the risk of urban floods.However,grassland showed a significant increase from 7.80%to 24.30%during 1990-2021.Meanwhile,bare ground increased from 27.16%to 31.56%and built-up area increased from 30.45%to 39.90%during 1990-2005.In 2021,built-up area decreased to 35.10%and bare ground decreased to 13.08%,indicating a consistent dominance of built-up area in the central parts of Kuala Lumpur.This study highlights the importance of integrating past,current,and future LULC changes to improve urban ecosystem services in the city.

Forecast future disasters using hydro-meteorological datasets in the Yamuna river basin,Western Himalaya:Using Markov Chain and LSTM approaches

This research aim to evaluate hydro-meteorological data from the Yamuna River Basin,Uttarakhand,India,utilizing Extreme Value Distribution of Frequency Analysis and the Markov Chain Approach.This method assesses persistence and allows for combinatorial probability estimations such as initial and transitional probabilities.The hydrologic data was generated(in-situ)and received from Uttarakhand Jal Vidut Nigam Limited(UJVNL),and meteorological data was acquired from NASA’s archives MERRA-2 product.A total of sixteen years(2005-2020)of data was used to foresee daily Precipitation from 2020 to 2022.MERRA-2 products are utilized as observed and forecast values for daily Precipitation throughout the monsoon season,which runs from July to September.Markov Chain and Long Short-Term Memory(LSTM)findings for 2020,2021,and 2022 were observed,and anticipated values for daily rainfall during the monsoon season between July and September.According to test findings,the artificial intelligence technique cannot anticipate future regional meteorological formations;the correlation coefficient R^(2) is around 0.12.According to the randomly verified precipitation data findings,the Markov Chain model has a success rate of 79.17 percent.The results suggest that extended return periods should be a warning sign for drought and flood risk in the Himalayan region.This study gives a better knowledge of the water budget,climate change variability,and impact of global warming,ultimately leading to improved water resource management and better emergency planning to the establishment of the Early Warning System(EWS)for extreme occurrences such as cloudbursts,flash floods,landslides hazards in the complex Himalayan region.

Variance Optimization for Continuous-Time Markov Decision Processes

This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.

多重填补法Markov Chain Monte Carlo模型在有缺失值的妇幼卫生纵向数据中的应用

目的 针对妇幼卫生纵向数据的任意缺失模式,采用多重填补方法进行填补,探求最佳填补结果,以便对数据作进一步分析与研究。方法 运用SAS9.0 ,采用多重填补方法Markov China Monte Carlo(MCMC)模型对缺失数据进行多次填补并综合分析。结果 填补5次所得结果最优。结论 多重填补方法可以处理有缺失数据资料中的许多普遍问题,可提高统计效率,尤其是MCMC模型在处理复杂的缺失数据上,优势明显。

基于Markov Chain的协议异常检测模型

本文介绍了基于Markov链的协议异常检测模型,此外,通过对MIT Lincoln实验室1999评估数据的分析,证明此模型的正确性和有效性。

基于MLP-ANN与Markov Chain的土地利用变化预测方法——以锡林郭勒盟为例

以北方草地典型地区—内蒙古锡林郭勒盟为案例区,在1995年到2000年的土地利用变化与驱动力分析的基础上,利用土地利用转换类型和驱动力模型,采用多层感知人工神经网络模型分析了各种土地利用类型未来的转换潜力;利用马尔可夫链模型,预测了2005和2010年土地利用格局。预测结果显示:高覆盖度草地减少幅度最大,中覆盖度草地减少相对和缓,高、中覆盖度草地的减少造成了未利用地和低覆盖度草地的增加,尤其是前者增加的幅度最大;从空间分布看,高覆盖度草地的减少集中在西北部地区,主要转变为中低覆盖度草地,中覆盖度草地的减少主要集中在西南部地区,其流向主要是以沙化土地为主的未利用地;案例研究表明,多层感知人工神经网络模型与马尔可夫链模型的结合与应用能够在很大程度上预测稳定驱动力作用下的土地利用变化趋势,从而为生态干预提供指导。

高等院校教师人才流动的Markov-chain预测模型

介绍并解析随机过程理论中的马尔科夫过程与马尔科夫链;针对高等院校教师人才流动变化过程,利用马尔科夫过程分析方法建立描述人员流动变化趋势的Markov-chain预测模型,详细阐述了模型的算法步骤.以某所高等院校教师人才流动的状态转移数据作为算例,运用新建立的预测模型,对该院校教师人才的流动趋势做出了预测分析.最后,将教师进修状态纳入分析范围,进行了教师职业生涯和职务发展趋势预测的深入分析.应用模型对实际算例的求解结果表明:Markov-chain预测模型及算法,叙述简洁、运算方便,为高等院校教师人才流动,乃至其他行业人才流动的预测提供了一种新的、有效的思路和方法.

基于Markov chain理论的科技成果转化的市场预测

科技成果成功转化的主要标志是其产品的市场畅销度。产品的畅销度由于受各种内外部不确定因素影响,具有随机不确定性。基于这种随机不确定性,运用Markovchain对其产品进行市场预测,并通过实证分析说明其有效性,为科技成果的转化提供具体可行的定量依据。

基于RS-SPA-Markov chain的生物质气化站安全评价与预测模型研究

本文在资料收集、实地考察、专家评估的基础上,建立了相对通用的生物质气化站安全评价指标体系,引入了RS-SPA-Markov chain安全评价与预测模型。通过实例应用,证明了该模型应用于生物质气化站安全评价与预测的可行性和科学性,为生物质气化站安全评价与预测提供了一种新的途径。

Auto-selection order of Markov chain for background sequences with chi-square test

Modeling non coding background sequences appropriately is important for the detection of regulatory elements from DNA sequences. Based on the chi square statistic test, some explanations about why to choose higher order Markov chain model and how to automatically select the proper order are given in this paper. The chi square test is first run on synthetic data sets to show that it can efficiently find the proper order of Markov chain. Using chi square test, distinct higher order context dependences inherent in ten sets of sequences of yeast S.cerevisiae from other literature have been found. So the Markov chain with higher order would be more suitable for modeling the non coding background sequences than an independent model.

Applicability of Markov chain-based stochastic model for bubbling fluidized beds

A Markov chain-based stochastic model （MCM） is developed to simulate the movement of particles in a 2D bubbling fluidized bed （BFB）. The state spaces are determined by the discretized physical cells of the bed, and the transition probability matrix is directly calculated by the results of a discrete element method （DEM） simulation. The Markov property of the BFB is discussed by the comparison results calculated from both static and dynamic transition probability matrices. The static matrix is calculated based on the Markov chain while the dynamic matrix is calculated based on the memory property of the particle movement. Results show that the difference in the trends of particle movement between the static and dynamic matrix calculation is very small. Besides, the particle mixing curves of the MCM and DEM have the same trend and similar numerical values, and the details show the time averaged characteristic of the MCM and also expose its shortcoming in describing the instantaneous particle dynamics in the BFB.

基于Markov Chain Monte Carlo模型对医院调查资料中缺失数据的多重估算

目的探讨基于Markov Chain Monte Carlo(MCMC)模型的多重估算法在处理医院调查资料缺失数据中的应用。方法运用SAS9.2编写程序,在分析数据的分布类型和缺失机制的基础上,采用MCMC法对缺失数据进行多次填补和联合统计推断,分析多重估算法的优势。结果数据服从多元正态分布与随机缺失,采用MCMC法填补10次所得的结果最佳。结论多重估算既可反映缺失数据的不确定性,又可充分利用现有资料的信息、提高统计效率、对模型的估计结果更加可信,是处理缺失数据的有效方法。

D-S理论和Markov链组合的桥梁性能退化预测研究

为准确预测桥梁性能退化,考虑到数据随机性和微小扰动发生状态跳跃,提出了一种D-S(Dempster-Shafer)证据理论和Markov链组合的桥梁性能退化组合预测模型和性能退化率的概念.该模型基于指数平滑(exponential smoothing,ES)方法获得新的预测数据序列,并利用Markov链和D-S理论不断进行优化,从而实现桥梁性能退化的组合预测.实际工程的应用结果表明:性能退化率可以直观地表征在梁性能退化的速度.其次,该模型的平均相对误差为1.54%,较于回归、灰色和模糊加权Markov链模型,精度分别提高了1.11%,0.88%和2.8%,而后验差比值为0.242,小于0.35;模型的标准差为9.021,相比其他模型分别减小了3.978,3.405和7.500,而变异系数为0.109,均小于其他模型,验证了组合预测模型在精度和稳定性方面的优越性,可为在役桥梁结构性能退化预测与维护提供理论基础.