Integral-type functionals of first hitting times for continuous-time Markov chains

We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.

Mathematical Modeling of Possibility Markov Chains by Possibility Theory

Statistical regression models are input-oriented estimation models that account for observation errors. On the other hand, an output-oriented possibility regression model that accounts for system fluctuations is proposed. Furthermore, the possibility Markov chain is proposed, which has a disidentifiable state (posterior) and a nondiscriminable state (prior). In this paper, we first take up the entity efficiency evaluation problem as a case study of the posterior non-discriminable production possibility region and mention Fuzzy DEA with fuzzy constraints. Next, the case study of the ex-ante non-discriminable event setting is discussed. Finally, we introduce the measure of the fuzzy number and the equality relation and attempt to model the possibility Markov chain mathematically. Furthermore, we show that under ergodic conditions, the direct sum state can be decomposed and reintegrated using fuzzy OR logic. We had already constructed the Possibility Markov process based on the indifferent state of this world. In this paper, we try to extend it to the indifferent event in another world. It should be noted that we can obtain the possibility transfer matrix by full use of possibility theory.

Ergodic degrees for continuous-time Markov chains

This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.

Contraction Integrated Semigroups and Their Applicationto Continuous-time Markov Chains

We introduce the notion of the contraction integrated semigroups and give the Lumber-Phillips characterization of the generator, and also the charaterazied generators of isometric integrated semigroups. For their application, a necessary and sufficient condition for q-matrices Q generating a contraction integrated semigroup is given, and a necessary and sufficient condition for a transition function to be a Feller-Reuter-Riley transition function is also given in terms of its q-matrix.

Perturbation analysis for continuous-time Markov chains

We investigate perturbation for continuous-time Markov chains(CTMCs) on a countable state space. Explicit bounds on ?D and D are derived in terms of a drift condition, where ? and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu(2012) for uniformly bounded CTMCs to general(possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.

D-S理论和Markov链组合的桥梁性能退化预测研究

为准确预测桥梁性能退化,考虑到数据随机性和微小扰动发生状态跳跃,提出了一种D-S(Dempster-Shafer)证据理论和Markov链组合的桥梁性能退化组合预测模型和性能退化率的概念.该模型基于指数平滑(exponential smoothing,ES)方法获得新的预测数据序列,并利用Markov链和D-S理论不断进行优化,从而实现桥梁性能退化的组合预测.实际工程的应用结果表明:性能退化率可以直观地表征在梁性能退化的速度.其次,该模型的平均相对误差为1.54%,较于回归、灰色和模糊加权Markov链模型,精度分别提高了1.11%,0.88%和2.8%,而后验差比值为0.242,小于0.35;模型的标准差为9.021,相比其他模型分别减小了3.978,3.405和7.500,而变异系数为0.109,均小于其他模型,验证了组合预测模型在精度和稳定性方面的优越性,可为在役桥梁结构性能退化预测与维护提供理论基础.

Modeling urban land use dynamics using Markov-chain and cellular automata in Gondar City,Northwest Ethiopia

Modeling urban land-use dynamics is critical for urban experts’and infrastructure managers’planning.This study attempts to explore the land-use/land-cover(LULC)dynamics of Gondar using satellite images from 1984 to 2020.Markov-Chain and Cellular Automata(MC-CA)models have been recognized as performing well in predicting urban land-use change.However,only a few models work in Ethiopia in general,and no study in Gondar has applied this approach to study urban land-use patterns.Therefore,Gondar land-use/land cover changes of Gondar were predicted using the MC-CA model in IDRISI.The built-up area in Gondar city covered 1413 ha(3%of the total area)in 1984 and increased to 2380 ha(5%)in 1994;21153 ha(45.5%)in 2004;22622 ha(48.7%)in 2014;and 23427 ha(50.5%)in 2020.The area has been predicted to reach 57.5%in the 2050s,showing a faster increase that will cause a very vast loss of farmland.This will increase urban sprawl challenges as well as overall environmental disequilibrium in the preceding decade.Thus,innovative and careful structures and systems in urban planning are required to secure a sustainable urban future and to make our cities livable and competitive in the paradigm of sustainable cities.

Markov Chains Based on Random Generalized 1-Flipper Operations for Connected Regular Multi-digraphs

The properties of generalized flip Markov chains on connected regular digraphs are discussed.The 1-Flipper operation on Markov chains for undirected graphs is generalized to that for multi-digraphs.The generalized 1-Flipper operation preserves the regularity and weak connectivity of multi-digraphs.The generalized 1-Flipper operation is proved to be symmetric.Moreover,it is presented that a series of random generalized 1-Flipper operations eventually lead to a uniform probability distribution over all connected d-regular multi-digraphs without loops.

Variance minimization for continuous-time Markov decision processes: two approaches

This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.

Development of Optimal Maintenance Policies for Offshore Wind Turbine Gearboxes Based on the Non-homogeneous Continuous-Time Markov Process

Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.

Air Quality Estimation Using Nonhomogeneous Markov Chains: A Case Study Comparing Two Rules Applied to Mexico City Data

A nonhomogeneous Markov chain is applied to the study of the air quality classification in Mexico City when the so-called criterion pollutants are used. We consider the indices associated with air quality using two regulations where different ways of classification are taken into account. Parameters of the model are the initial and transition probabilities of the chain. They are estimated under the Bayesian point of view through samples generated directly from the corresponding posterior distributions. Using the estimated parameters, the probability of having an air quality index in a given hour of the day is obtained.

基于Markov模型的低风速地区风力发电机预防性维护优化策略

在“3060”双碳战略目标下,随着风电产业的迅速发展,预防性维护已然成为提高风力发电机运行可靠性的重要手段。关于复杂环境下风力发电机的运行管理尚存在对退化状态认识不充分、维修策略不可靠等问题。本文以期望成本最小化为目的,考虑风力发电机关键部件退化过程与多阶段预防性维护策略的关系,构建基于Markov状态转移的多阶段预防性维护成本模型。首先,依据设备衰退规律,利用Markov链的转移概率对退化状态、维护策略进行刻画。其次,引入可靠度、故障率、役龄因子计算多阶段预防性维护时长和故障时长;再进一步考虑天气条件对维护成本模型的影响,构建风力发电机在整个维护周期内的期望成本模型。最后,利用数值分析对模型进行求解。结果表明,实施最小、更换维护策略的期望成本占据了总维护成本的80%以上,相比之下,采取预防性维护优化策略的期望成本则低于总维护成本的20%,故该策略可为低风速地区的风电企业提供有效的决策参考,提高风力发电机的运行可靠性。

Ecological Classroom Model for Vocabulary Teaching Based on Markov Chain

This paper explores the current English vocabulary teaching classroom based on an eco-linguistic perspective and the Markov chain,points out the ecological imbalance in the English vocabulary classroom,and proposes measures to solve the problem and strategies to build an ecological English vocabulary learning classroom,in order to provide some reference for future English vocabulary curriculum reform and curriculum design.In this paper,a random sample of 20 students from each class was taken from an advanced arts class and an advanced science class.The results show that after a semester of teaching process,the overall English scores of the arts and science classes improved,and since,the arts classes improved more and the teachers’teaching quality was better.

Service Function Chain Deployment Algorithm Based on Multi-Agent Deep Reinforcement Learning

Aiming at the rapid growth of network services,which leads to the problems of long service request processing time and high deployment cost in the deployment of network function virtualization service function chain(SFC)under 5G networks,this paper proposes a multi-agent deep deterministic policy gradient optimization algorithm for SFC deployment(MADDPG-SD).Initially,an optimization model is devised to enhance the request acceptance rate,minimizing the latency and deploying the cost SFC is constructed for the network resource-constrained case.Subsequently,we model the dynamic problem as a Markov decision process(MDP),facilitating adaptation to the evolving states of network resources.Finally,by allocating SFCs to different agents and adopting a collaborative deployment strategy,each agent aims to maximize the request acceptance rate or minimize latency and costs.These agents learn strategies from historical data of virtual network functions in SFCs to guide server node selection,and achieve approximately optimal SFC deployment strategies through a cooperative framework of centralized training and distributed execution.Experimental simulation results indicate that the proposed method,while simultaneously meeting performance requirements and resource capacity constraints,has effectively increased the acceptance rate of requests compared to the comparative algorithms,reducing the end-to-end latency by 4.942%and the deployment cost by 8.045%.

Variance Optimization for Continuous-Time Markov Decision Processes

This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.

基于互Box-Cox变换和Markov链风速云模型的发电系统充裕度评估

为准确计及风速随机性和自相关性对风电并网系统充裕度的影响,建立基于互Box-Cox变换和Markov链的风速云模型,并将该模型与时序Monte Carlo模拟法结合,提出计及风速随机性和自相关性的风电并网系统充裕度评估方法。仿真结果表明,所提模型产生的仿真风速样本与实测风速样本具备相似的概率分布特性和自相关性,所提方法可较精确地评估风电并网系统充裕度及风电容量可信度。

基于改进Markov智能网联多车型混合流编队策略

为提高智能网联(connected and automated,CA)卡车、小车及人工驾驶卡车、小车的混合流道路通行能力,提出基于排强度和渗透率的CA车辆单独编队和合作编队策略.分别设计两种策略下混合流车辆跟驰模式,推导出基于改进Markov模型,涵盖CA车辆渗透率和排强度的车辆状态转移概率;分析两种策略下CA车辆队列分布,建立各策略下的混合流道路容量模型,并通过理论证明和仿真实验予以验证.结果表明,与不编队策略相比,两种策略下道路容量分别提高1.23%~49.62%和1.47%~60.34%,合作编队策略与单独编队策略相比能将道路容量再提高11%;当CA车辆渗透率大于50%和排强度大于0时,编队策略对道路容量的提升效果更显著,容量能提高13.27%~60.34%;单独编队策略下CA小车和CA卡车最大队列规模分别为8辆和6辆,合作编队下CA车辆最大队列规模为8辆.

新疆旅游经济贡献空间非均衡与动态演化研究——基于Dagum基尼系数与Markov链估计

结合投入产出法和旅游剥离系数法测算了新疆各地州旅游业增加值基础数据,基于Dagum基尼系数和Markov链估计方法,探讨了2001—2019年新疆旅游经济贡献的空间非均衡特征及动态演变。研究表明:新疆旅游经济贡献空间分布表现为“北高-东低-南低”的格局,且主要发生在东北-西南方向上,呈显著的空间非均衡特征;其中,旅游经济贡献高的核心地州有乌鲁木齐、伊犁州直、昌吉州、阿勒泰、吐鲁番等。Dagum基尼系数及分解研究发现新疆旅游经济贡献空间非均衡的状态仍是旅游经济发展的主流状态,组内差距的贡献缩小引致旅游经济贡献趋向平衡,而空间非均衡的主要来源是组间差距;进一步解构新疆旅游经济贡献内部动态演进,发现新疆旅游经济贡献水平存在“两头固化,中间活跃”的演化特征,内部状态流动性较低,反映出区域旅游经济跃迁需要具备发展基础和长期积累的条件。基于此,研究提出了“缩差距、破固化、强投入、先人才”的建议。

Markov切换拓扑下离散多智能体系统滑模一致性控制

针对具有Markov切换拓扑的离散多智能体系统,研究了其在滑模控制下的均方领导跟随一致性问题。通过引入Markov切换拓扑,描述了智能体之间的随机信息交互。采用一致性误差定义了与随机切换拓扑相关的滑模面。根据理想准滑动模态条件得到了等效控制律,并将离散多智能体系统的一致性问题转化为领导跟随误差系统的稳定性问题;利用Lyapunov函数的构造和矩阵理论最终给出了离散滑动模态均方领导跟随一致性的充分条件,并给出了控制增益的设计方法。然后利用趋近律原理给出了一种离散时间滑模控制器,实现了跟随误差能够在有限时间内到达所定义的滑模面区域内。如何处理部分转移概率未知的Markov切换拓扑和设计与随机切换拓扑相关的滑模面是本文的两个难点。最后通过一个数值算例验证了所提方法的有效性。

多重填补法Markov Chain Monte Carlo模型在有缺失值的妇幼卫生纵向数据中的应用

目的 针对妇幼卫生纵向数据的任意缺失模式,采用多重填补方法进行填补,探求最佳填补结果,以便对数据作进一步分析与研究。方法 运用SAS9.0 ,采用多重填补方法Markov China Monte Carlo(MCMC)模型对缺失数据进行多次填补并综合分析。结果 填补5次所得结果最优。结论 多重填补方法可以处理有缺失数据资料中的许多普遍问题,可提高统计效率,尤其是MCMC模型在处理复杂的缺失数据上,优势明显。