For critical engineering systems such as aircraft and aerospace vehicles, accurate Remaining Useful Life(RUL) prediction not only means cost saving, but more importantly, is of great significance in ensuring system re...For critical engineering systems such as aircraft and aerospace vehicles, accurate Remaining Useful Life(RUL) prediction not only means cost saving, but more importantly, is of great significance in ensuring system reliability and preventing disaster. RUL is affected not only by a system's intrinsic deterioration, but also by the operational conditions under which the system is operating. This paper proposes an RUL prediction approach to estimate the mean RUL of a continuously degrading system under dynamic operational conditions and subjected to condition monitoring at short equi-distant intervals. The dynamic nature of the operational conditions is described by a discrete-time Markov chain, and their influences on the degradation signal are quantified by degradation rates and signal jumps in the degradation model. The uniqueness of our proposed approach is formulating the RUL prediction problem in a semi-Markov decision process framework, by which the system mean RUL can be obtained through the solution to a limited number of equations. To extend the use of our proposed approach in real applications, different failure standards according to different operational conditions are also considered. The application and effectiveness of this approach are illustrated by a turbofan engine dataset and a comparison with existing results for the same dataset.展开更多
This paper is the first attempt to investigate the risk probability criterion in semi-Markov decision processes with loss rates. The goal is to find an optimal policy with the minimum risk probability that the total l...This paper is the first attempt to investigate the risk probability criterion in semi-Markov decision processes with loss rates. The goal is to find an optimal policy with the minimum risk probability that the total loss incurred during a first passage time to some target set exceeds a loss level. First, we establish the optimality equation via a successive approximation technique, and show that the value function is the unique solution to the optimality equation. Second, we give suitable conditions, under which we prove the existence of optimal policies and develop an algorithm for computing ?-optimal policies. Finally, we apply our main results to a business system.展开更多
In intelligent transportation system(ITS), the interworking of vehicular networks(VN) and cellular networks(CN) is proposed to provide high-data-rate services to vehicles. As the network access quality for CN and VN i...In intelligent transportation system(ITS), the interworking of vehicular networks(VN) and cellular networks(CN) is proposed to provide high-data-rate services to vehicles. As the network access quality for CN and VN is location related, mobile data offloading(MDO), which dynamically selects access networks for vehicles, should be considered with vehicle route planning to further improve the wireless data throughput of individual vehicles and to enhance the performance of the entire ITS. In this paper, we investigate joint MDO and route selection for an individual vehicle in a metropolitan scenario. We aim to improve the throughput of the target vehicle while guaranteeing its transportation efficiency requirements in terms of traveling time and distance. To achieve this objective, we first formulate the joint route and access network selection problem as a semi-Markov decision process(SMDP). Then we propose an optimal algorithm to calculate its optimal policy. To further reduce the computation complexity, we derive a suboptimal algorithm which reduces the action space. Simulation results demonstrate that the proposed optimal algorithm significantly outperforms the existing work in total throughput and the late arrival ratio.Moreover, the heuristic algorithm is able to substantially reduce the computation time with only slight performance degradation.展开更多
This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs. The criterion to be optimized is the expected discounted cost incurred during a f...This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs. The criterion to be optimized is the expected discounted cost incurred during a first passage time to a given target set. We first construct a semi-Markov decision process under a given semi-Markov decision kernel and a policy. Then, we prove that the value function satisfies the optimality equation and there exists an optimal （or ε-optimal） stationary policy under suitable conditions by using a minimum nonnegative solution approach. Further we give some properties of optimal policies. In addition, a value iteration algorithm for computing the value function and optimal policies is developed and an example is given. Finally, it is showed that our model is an extension of the first passage models for both discrete-time and continuous-time Markov decision processes.展开更多
This paper investigates the Borel state space semi-Markov decision process (SMDP) with the criterion of expected total rewards in a semi-Markov environment. It describes a system which behaves like a SMDP except that ...This paper investigates the Borel state space semi-Markov decision process (SMDP) with the criterion of expected total rewards in a semi-Markov environment. It describes a system which behaves like a SMDP except that the system is influenced by its environment modeled by a semi-Markov process. We transform the SMDP in a semiMarkov environment into an equivalent discrete time Markov decision process under the condition that rewards are all positive or all negative, and obtain the optimality equation and some properties for it.展开更多
车联网是物联网(Internet of Things,IOT)技术在智能交通领域的典型应用,研究车联网关键技术,可以高效促进我国交通系统建设。车载云计算(Vehicular Cloud Computing,VCC)作为实现智能交通的关键技术之一,在降低功率和时间的消耗,提高...车联网是物联网(Internet of Things,IOT)技术在智能交通领域的典型应用,研究车联网关键技术,可以高效促进我国交通系统建设。车载云计算(Vehicular Cloud Computing,VCC)作为实现智能交通的关键技术之一,在降低功率和时间的消耗,提高车辆总体资源利用率和系统长期收益等方面具有至关重要的作用。针对车辆自身资源受限以及将任务卸载到中心云将导致较高通信成本的情况,提出在车载云之间引入服务迁移的机制,同时将路边单元(Road Side Unit,RSU)和车辆异构性考虑进VCC系统中,基于半马尔科夫决策过程(Semi-Markov Decision Processes,SMDP)建立了VCC系统模型,最后应用值迭代算法求解,来寻找VCC资源分配的最优策略。仿真结果展示了车辆异构性对资源分配的影响,同时表明了SMDP资源管理方案的优越性,SMDP相比于贪婪算法(Greedy Algorithm,GA)和模拟退火算法(Simulated Annealing,SA)这两个传统算法,系统长期收益分别提高了10%和3%左右。展开更多
Testing is the premise and foundation of realizing equipment health management （EHM）. To address the problem that the static periodic test strategy may cause deficient test or excessive test, a dynamic sequential te...Testing is the premise and foundation of realizing equipment health management （EHM）. To address the problem that the static periodic test strategy may cause deficient test or excessive test, a dynamic sequential test strategy （DSTS） for EHM is presented. Considering the situation that equipment health state is not completely observable in reality, a DSTS optimization method based on partially observable semi-Markov decision pro- cess （POSMDP） is proposed. Firstly, an equipment health state degradation model is constructed by Markov process, and the control limit maintenance policy is also introduced. Secondly, POSMDP is formulated in great detail. And then, POSMDP is converted to completely observable belief semi-Markov decision process （BSMDP） through belief state. The optimal equation and the corresponding optimal DSTS, which minimize the long-run ex- pected average cost per unit time, are obtained with BSMDP. The results of application in complex equipment show that the proposed DSTS is feasible and effective.展开更多
基金the National Natural science Foundation of China (No. 71701008) for supporting this research
文摘For critical engineering systems such as aircraft and aerospace vehicles, accurate Remaining Useful Life(RUL) prediction not only means cost saving, but more importantly, is of great significance in ensuring system reliability and preventing disaster. RUL is affected not only by a system's intrinsic deterioration, but also by the operational conditions under which the system is operating. This paper proposes an RUL prediction approach to estimate the mean RUL of a continuously degrading system under dynamic operational conditions and subjected to condition monitoring at short equi-distant intervals. The dynamic nature of the operational conditions is described by a discrete-time Markov chain, and their influences on the degradation signal are quantified by degradation rates and signal jumps in the degradation model. The uniqueness of our proposed approach is formulating the RUL prediction problem in a semi-Markov decision process framework, by which the system mean RUL can be obtained through the solution to a limited number of equations. To extend the use of our proposed approach in real applications, different failure standards according to different operational conditions are also considered. The application and effectiveness of this approach are illustrated by a turbofan engine dataset and a comparison with existing results for the same dataset.
基金supported by National Natural Science Foundation of China(Grant Nos.61374067 and 11471341)
文摘This paper is the first attempt to investigate the risk probability criterion in semi-Markov decision processes with loss rates. The goal is to find an optimal policy with the minimum risk probability that the total loss incurred during a first passage time to some target set exceeds a loss level. First, we establish the optimality equation via a successive approximation technique, and show that the value function is the unique solution to the optimality equation. Second, we give suitable conditions, under which we prove the existence of optimal policies and develop an algorithm for computing ?-optimal policies. Finally, we apply our main results to a business system.
基金the National Natural Science Foundation of China under Grants 61631005 and U1801261the National Key R&D Program of China under Grant 2018YFB1801105+3 种基金the Central Universities under Grant ZYGX2019Z022the Key Areas of Research and Development Program of Guangdong Province, China, under Grant 2018B010114001the 111 Project under Grant B20064the China Postdoctoral Science Foundation under Grant No. 2018M631075
文摘In intelligent transportation system(ITS), the interworking of vehicular networks(VN) and cellular networks(CN) is proposed to provide high-data-rate services to vehicles. As the network access quality for CN and VN is location related, mobile data offloading(MDO), which dynamically selects access networks for vehicles, should be considered with vehicle route planning to further improve the wireless data throughput of individual vehicles and to enhance the performance of the entire ITS. In this paper, we investigate joint MDO and route selection for an individual vehicle in a metropolitan scenario. We aim to improve the throughput of the target vehicle while guaranteeing its transportation efficiency requirements in terms of traveling time and distance. To achieve this objective, we first formulate the joint route and access network selection problem as a semi-Markov decision process(SMDP). Then we propose an optimal algorithm to calculate its optimal policy. To further reduce the computation complexity, we derive a suboptimal algorithm which reduces the action space. Simulation results demonstrate that the proposed optimal algorithm significantly outperforms the existing work in total throughput and the late arrival ratio.Moreover, the heuristic algorithm is able to substantially reduce the computation time with only slight performance degradation.
基金Supported by the Natural Science Foundation of China(No.60874004,60736028)Guangdong Province Universities and Colleges Pearl River Scholar Funded Scheme(2010)
文摘This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs. The criterion to be optimized is the expected discounted cost incurred during a first passage time to a given target set. We first construct a semi-Markov decision process under a given semi-Markov decision kernel and a policy. Then, we prove that the value function satisfies the optimality equation and there exists an optimal （or ε-optimal） stationary policy under suitable conditions by using a minimum nonnegative solution approach. Further we give some properties of optimal policies. In addition, a value iteration algorithm for computing the value function and optimal policies is developed and an example is given. Finally, it is showed that our model is an extension of the first passage models for both discrete-time and continuous-time Markov decision processes.
文摘This paper investigates the Borel state space semi-Markov decision process (SMDP) with the criterion of expected total rewards in a semi-Markov environment. It describes a system which behaves like a SMDP except that the system is influenced by its environment modeled by a semi-Markov process. We transform the SMDP in a semiMarkov environment into an equivalent discrete time Markov decision process under the condition that rewards are all positive or all negative, and obtain the optimality equation and some properties for it.
基金supported by the National Natural Science Foundation of China (51175502)
文摘Testing is the premise and foundation of realizing equipment health management （EHM）. To address the problem that the static periodic test strategy may cause deficient test or excessive test, a dynamic sequential test strategy （DSTS） for EHM is presented. Considering the situation that equipment health state is not completely observable in reality, a DSTS optimization method based on partially observable semi-Markov decision pro- cess （POSMDP） is proposed. Firstly, an equipment health state degradation model is constructed by Markov process, and the control limit maintenance policy is also introduced. Secondly, POSMDP is formulated in great detail. And then, POSMDP is converted to completely observable belief semi-Markov decision process （BSMDP） through belief state. The optimal equation and the corresponding optimal DSTS, which minimize the long-run ex- pected average cost per unit time, are obtained with BSMDP. The results of application in complex equipment show that the proposed DSTS is feasible and effective.
