Asymptotic Evaluations of the Stability Index for a Markov Control Process with the Expected Total Discounted Reward Criterion

In this work, for a control consumption-investment process with the discounted reward optimization criteria, a numerical estimate of the stability index is made. Using explicit formulas for the optimal stationary policies and for the value functions, the stability index is explicitly calculated and through statistical techniques its asymptotic behavior is investigated (using numerical experiments) when the discount coefficient approaches 1. The results obtained define the conditions under which an approximate optimal stationary policy can be used to control the original process.

Evolution of Damage on Historical Heritage Buildings in Presence of Catastrophic Events and Aggressive Natural Phenomena

The feasibility analysis of projects for the preservation of the historical heritage buildings is an important problem concerning the evaluation of ＂the total cost of intervention＂, which includes all the future damage costs. The total cost of intervention represents a suitable measure of the expected deterioration risk and its evolution obviously depends on the damage process which buildings are subjected to. That damage phenomena affecting masonry buildings pleased into an aggressive environment are suitably modelled by renewal processes： this happens both in the case of catastrophic events, or in the case of the so-called ＂natural aging＂, in which damage comes off gradually in time. In the hypothesis ofa Markovian renewal process （Mrp） describing the damage process, the total cost of all the future damage is evaluated taking into account both the damage aspects： damages due to catastrophic aspects and damages due to aggressive environment, supposing different maintenance and/or rehabilitation scenarios. A semi-Markov process （s-Mp） is defined to model the damage rehabilitation history of buildings in presence of seismic events, natural ageing and rehabilitation strategies. The expected rewards connected to the process are defined; they represent a significant measure of the risk.

CONSTRAINED DENUMERABLE STATE NON-STATIONARY MDPs WITH EXPECTED TOTAL REWARD CRITERION

In this paper, we consider constrained denumerable state non-stationary Markov decision processes (MDPs, for short) with expected total reward criterion. By the mechanics of intro- ducing Lagrange multiplier and using the methods of probability and analytics, we prove the existence of constrained optimal policies. Moreover, we prove that a constrained optimal policy may be a Markov policy, or be a randomized Markov policy that randomizes between two Markov policies, that differ in only one state.

STRONG N-DISCOUNT AND FINITE-HORIZON OPTIMALITY FOR CONTINUOUS-TIME MARKOV DECISION PROCESSES

This paper studies the strong n(n =—1,0)-discount and finite horizon criteria for continuoustime Markov decision processes in Polish spaces.The corresponding transition rates are allowed to be unbounded,and the reward rates may have neither upper nor lower bounds.Under mild conditions,the authors prove the existence of strong n(n =—1,0)-discount optimal stationary policies by developing two equivalence relations:One is between the standard expected average reward and strong—1-discount optimality,and the other is between the bias and strong 0-discount optimality.The authors also prove the existence of an optimal policy for a finite horizon control problem by developing an interesting characterization of a canonical triplet.