The state equations of stochastic control problems,which are controlled stochastic differential equations,are proposed to be discretized by the weak midpoint rule and predictor-corrector methods for the Markov chain a...The state equations of stochastic control problems,which are controlled stochastic differential equations,are proposed to be discretized by the weak midpoint rule and predictor-corrector methods for the Markov chain approximation approach. Local consistency of the methods are proved.Numerical tests on a simplified Merton's portfolio model show better simulation to feedback control rules by these two methods, as compared with the weak Euler-Maruyama discretisation used by Krawczyk.This suggests a new approach of improving accuracy of approximating Markov chains for stochastic control problems.展开更多
In this paper, we study the count of head runs up to a fixed time in a two-state stationary Markov chain. We prove that in total variance distance, the negative binomial, Poisson and binomial distributions are appropr...In this paper, we study the count of head runs up to a fixed time in a two-state stationary Markov chain. We prove that in total variance distance, the negative binomial, Poisson and binomial distributions are appropriate approximations according to the relation of the variance and mean of the count, generalizing earlier results in previous literatures. The proof is based on Stein's method and coupling.展开更多
This work is concerned with rates of convergence of numerical methods using Markov chainapproximation for controlled diffusions with stopping (the first exit time from a bounded region).In lieuof considering the assoc...This work is concerned with rates of convergence of numerical methods using Markov chainapproximation for controlled diffusions with stopping (the first exit time from a bounded region).In lieuof considering the associated finite difference schemes for Hamilton-Jacobi-Bellman (HJB) equations,a purely probabilistic approach is used.There is an added difficulty due to the boundary condition,which requires the continuity of the first exit time with respect to the discrete parameter.To prove theconvergence of the algorithm by Markov chain approximation method,a tangency problem might arise.A common approach uses certain conditions to avoid the tangency problem.Here,by modifying thevalue function,it is demonstrated that the tangency problem will not arise in the sense of convergencein probability and in L^1.In addition,controlled diffusions with a discount factor is also treated.展开更多
Two stochastic models are derived for a susceptible-infectious-susceptible epidemic spreading through a metapopulation： a continuous time Markov chain （CTMC） model and an It6 stochastic differential equation （SDE...Two stochastic models are derived for a susceptible-infectious-susceptible epidemic spreading through a metapopulation： a continuous time Markov chain （CTMC） model and an It6 stochastic differential equation （SDE） model. The stochastic models are numerically compared. Close agreement suggests that computationally intense CTMC simulations can be approximated by simpler SDE simulations. Differential equations for the moments of the SDE probability distribution are also derived, the steady states are solved numerically using a moment closure technique, and these results are compared to simulations. The moment closure technique only coarsely approximates simulation results. The effect of model parameters on stability of the disease-free equilibrium is also numerically investigated.展开更多
基金supported by the China Postdoctoral Science Foundation (No.20080430402).
文摘The state equations of stochastic control problems,which are controlled stochastic differential equations,are proposed to be discretized by the weak midpoint rule and predictor-corrector methods for the Markov chain approximation approach. Local consistency of the methods are proved.Numerical tests on a simplified Merton's portfolio model show better simulation to feedback control rules by these two methods, as compared with the weak Euler-Maruyama discretisation used by Krawczyk.This suggests a new approach of improving accuracy of approximating Markov chains for stochastic control problems.
基金supported by National Natural Science Foundation of China (Grant No. 11071021)
文摘In this paper, we study the count of head runs up to a fixed time in a two-state stationary Markov chain. We prove that in total variance distance, the negative binomial, Poisson and binomial distributions are appropriate approximations according to the relation of the variance and mean of the count, generalizing earlier results in previous literatures. The proof is based on Stein's method and coupling.
基金supported in part by the National Science Foundation under Grant Nos. DMS-0624849 and DMS-0907753in part by the Natural Science Foundation of China under Grant No. #70871055
文摘This work is concerned with rates of convergence of numerical methods using Markov chainapproximation for controlled diffusions with stopping (the first exit time from a bounded region).In lieuof considering the associated finite difference schemes for Hamilton-Jacobi-Bellman (HJB) equations,a purely probabilistic approach is used.There is an added difficulty due to the boundary condition,which requires the continuity of the first exit time with respect to the discrete parameter.To prove theconvergence of the algorithm by Markov chain approximation method,a tangency problem might arise.A common approach uses certain conditions to avoid the tangency problem.Here,by modifying thevalue function,it is demonstrated that the tangency problem will not arise in the sense of convergencein probability and in L^1.In addition,controlled diffusions with a discount factor is also treated.
文摘Two stochastic models are derived for a susceptible-infectious-susceptible epidemic spreading through a metapopulation： a continuous time Markov chain （CTMC） model and an It6 stochastic differential equation （SDE） model. The stochastic models are numerically compared. Close agreement suggests that computationally intense CTMC simulations can be approximated by simpler SDE simulations. Differential equations for the moments of the SDE probability distribution are also derived, the steady states are solved numerically using a moment closure technique, and these results are compared to simulations. The moment closure technique only coarsely approximates simulation results. The effect of model parameters on stability of the disease-free equilibrium is also numerically investigated.
