This paper focuses on the constrained optimality problem （COP） of first passage discrete-time Markov decision processes （DTMDPs） in denumerable state and compact Borel action spaces with multi-constraints, state-d...This paper focuses on the constrained optimality problem （COP） of first passage discrete-time Markov decision processes （DTMDPs） in denumerable state and compact Borel action spaces with multi-constraints, state-dependent discount factors, and possibly unbounded costs. By means of the properties of a so-called occupation measure of a policy, we show that the constrained optimality problem is equivalent to an （infinite-dimensional） linear programming on the set of occupation measures with some constraints, and thus prove the existence of an optimal policy under suitable conditions. Furthermore, using the equivalence between the constrained optimality problem and the linear programming, we obtain an exact form of an optimal policy for the case of finite states and actions. Finally, as an example, a controlled queueing system is given to illustrate our results.展开更多
A robust and reliable parameter estimation is a critical issue for modeling in immunology. We developed a computational methodology for analysis of the best-fit parameter estimates and the information-theoretic assess...A robust and reliable parameter estimation is a critical issue for modeling in immunology. We developed a computational methodology for analysis of the best-fit parameter estimates and the information-theoretic assessment of the mathematical models formulated with ODEs. The core element of the methodology is a robust evaluation of the first and second derivatives of the model solution with respect to the model parameter values. The critical issue of the reliable estimation of the derivatives was addressed in the context of inverse problems arising in mathematical immunology. To evaluate the first and second derivatives of the ODE solution with respect to parameters, we implemented the variational equations-, automatic differentiation and complex-step derivative approximation methods. A comprehensive analysis of these approaches to the derivative approximations is presented to understand their advantages and limitations.展开更多
基金This work was supported in part by the National Natural Science Foundation of China （Grant Nos. 61374067, 41271076）.
文摘This paper focuses on the constrained optimality problem （COP） of first passage discrete-time Markov decision processes （DTMDPs） in denumerable state and compact Borel action spaces with multi-constraints, state-dependent discount factors, and possibly unbounded costs. By means of the properties of a so-called occupation measure of a policy, we show that the constrained optimality problem is equivalent to an （infinite-dimensional） linear programming on the set of occupation measures with some constraints, and thus prove the existence of an optimal policy under suitable conditions. Furthermore, using the equivalence between the constrained optimality problem and the linear programming, we obtain an exact form of an optimal policy for the case of finite states and actions. Finally, as an example, a controlled queueing system is given to illustrate our results.
文摘A robust and reliable parameter estimation is a critical issue for modeling in immunology. We developed a computational methodology for analysis of the best-fit parameter estimates and the information-theoretic assessment of the mathematical models formulated with ODEs. The core element of the methodology is a robust evaluation of the first and second derivatives of the model solution with respect to the model parameter values. The critical issue of the reliable estimation of the derivatives was addressed in the context of inverse problems arising in mathematical immunology. To evaluate the first and second derivatives of the ODE solution with respect to parameters, we implemented the variational equations-, automatic differentiation and complex-step derivative approximation methods. A comprehensive analysis of these approaches to the derivative approximations is presented to understand their advantages and limitations.
