Ergodic switching control for diffusion-type processes

We consider the control of the diserete component n_(t)of a owitching Markov proceaa x_(t)=(z_(t),n_(t))when there ia a running cost and an immediate coat c(i,j)for owitching n_(t)from i to j.We satudy the minimization of the ergodic(or long-term average)total coat.Eooentially,this paper trento the cnce where,for n_(t)=n fixed,z_(t)ia a reflected diffusion or a reflected diffusion with jumps,nt being,for fixed z,a continuous-time Markov chain.Using the vanishing discount appronch,we exctend existing reoulta dealing with the situation where nt evolvea only by the switching control action and the diffusion is non-degenerate.Moreover,we solve the ergodic problem for a claso of diffusiono which can be degenerate and for an example with aboorbing atate.

SIMPLE COMPUTING OF THE CUSTOMER LIFETIME VALUE:A FIXED LOCAL-OPTIMAL POLICY APPROACH

In this paper,we present a new method for finding a fixed local-optimal policy for computing the customer lifetime value.The method is developed for a class of ergodic controllable finite Markov chains.We propose an approach based on a non-converging state-value function that fluctuates（increases and decreases） between states of the dynamic process.We prove that it is possible to represent that function in a recursive format using a one-step-ahead fixed-optimal policy.Then,we provide an analytical formula for the numerical realization of the fixed local-optimal strategy.We also present a second approach based on linear programming,to solve the same problem,that implement the c-variable method for making the problem computationally tractable.At the end,we show that these two approaches are related：after a finite number of iterations our proposed approach converges to same result as the linear programming method.We also present a non-traditional approach for ergodicity verification.The validity of the proposed methods is successfully demonstrated theoretically and,by simulated credit-card marketing experiments computing the customer lifetime value for both an optimization and a game theory approach.