This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance mi...This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.展开更多
Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of off...Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.展开更多
This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space...This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.展开更多
We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The co...We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.展开更多
For the continuous time Markov chain with transition function P（t） on Z d ＋ , we give the necessary and sufficient conditions for the existence of its Siegmund dual with transition function P - （t）. If Q, the q-m...For the continuous time Markov chain with transition function P（t） on Z d ＋ , we give the necessary and sufficient conditions for the existence of its Siegmund dual with transition function P - （t）. If Q, the q-matrix of P（t）, is uniformly bounded, we show that the Siegmund dual relation can be expressed directly in terms of q-matrices, and a sufficient condition under which the Q-function is the Siegnmnd dual of some Q-function is also given.展开更多
基金supported by the National Natural Science Foundation of China(10801056)the Natural Science Foundation of Ningbo(2010A610094)
文摘This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.
文摘Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.
文摘This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.
文摘为了实现同一地域范围内的众多用户在有限带宽条件下提出的高QoS要求,本文对基于IEEE 802.16标准的宽带无线接入网中数据包级QoS(Quality of Service)性能进行了研究.具体做法是,首先采用批马尔可夫到达过程(BMAP,Batch Markov Arrival Process)和连续时间马尔科夫链(CTMC,Continuous Time Markov Chain)对到达过程和流量源进行建模,得到更符合实际和更准确的排队模型;然后基于状态空间,对一个无线接入网络系统进行建模,通过对得到的系统模型并结合前面得到的排队模型的深入分析,从而获得该模型下的各项QoS性能指标,如平均队列长度、丢包率、队列吞吐量和平均包时延.仿真实验结果表明,本文提出的算法模型相比于其他典型的算法模型,能够使得各项QoS性能指标有较大的改善和提高.
基金Acknowledgements The authors would like to thank Professor Yong-Hua Mao for useful discussion. This work was supported in part by the National Natural Science Foundation of China （Grant Nos. 11571372, 11501576, 11771452） and the Excellent Young Scientific Research Fund of Hunan Provincial Education Department （Grant No. 15B252）.
文摘We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.
基金Supported by NSFC(Grant Nos.11626245 and 11571043)
文摘For the continuous time Markov chain with transition function P（t） on Z d ＋ , we give the necessary and sufficient conditions for the existence of its Siegmund dual with transition function P - （t）. If Q, the q-matrix of P（t）, is uniformly bounded, we show that the Siegmund dual relation can be expressed directly in terms of q-matrices, and a sufficient condition under which the Q-function is the Siegnmnd dual of some Q-function is also given.
