An algorithm for numerical solution of discrete Hamilton-Jacobi-Bellman equations is proposed. The method begins with a suitable initial guess value of the solution,then finds a suitable matrix to linearize the system...An algorithm for numerical solution of discrete Hamilton-Jacobi-Bellman equations is proposed. The method begins with a suitable initial guess value of the solution,then finds a suitable matrix to linearize the system and constructs an iteration algorithm to generate the monotone sequence. The convergence of the algorithm for nonlinear discrete Hamilton-Jacobi-Bellman equations is proved. Some numerical examples are presented to confirm the effciency of this algorithm.展开更多
This paper presents a modified domain decomposition method for the numerical solution of discrete Hamilton-Jacobi-Bellman equations arising from a class of optimal control problems using diffusion models. A convergenc...This paper presents a modified domain decomposition method for the numerical solution of discrete Hamilton-Jacobi-Bellman equations arising from a class of optimal control problems using diffusion models. A convergence theorem is established. Numerical results indicate the effectiveness and accuracy of the method.展开更多
In the framework of stochastic processes, the connection between the dynamic programming scheme given by the Hamilton-Jacobi-Bellman equation and a recently proposed control approach based on the Fokker-Planck equatio...In the framework of stochastic processes, the connection between the dynamic programming scheme given by the Hamilton-Jacobi-Bellman equation and a recently proposed control approach based on the Fokker-Planck equation is discussed. Under appropriate assumptions it is shown that the two strategies are equivalent in the case of expected cost functionals, while the Fokker-Planck formalism allows considering a larger classof objectives. To illustratethe connection between the two control strategies, the cases of an Itō stochastic process and of a piecewise-deterministic process are considered.展开更多
In this paper, we propose an iterative relaxation method for solving the Hamilton-Jacobi-Bellman-Isaacs equation(HJBIE) arising in deterministic optimal control of affine nonlinear systems. Local convergence of the me...In this paper, we propose an iterative relaxation method for solving the Hamilton-Jacobi-Bellman-Isaacs equation(HJBIE) arising in deterministic optimal control of affine nonlinear systems. Local convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the method. An extension of the approach to Lyapunov equations is also discussed. The preliminary results presented are promising, and it is hoped that the approach will ultimately develop into an efficient computational tool for solving the HJBIEs.展开更多
In this paper, iterative or successive approximation methods for the Hamilton-Jacobi-Bellman-lsaacs equations （HJBIEs） arising in both deterministic and stochastic optimal control for affine nonlinear systems are de...In this paper, iterative or successive approximation methods for the Hamilton-Jacobi-Bellman-lsaacs equations （HJBIEs） arising in both deterministic and stochastic optimal control for affine nonlinear systems are developed. Convergence of the methods are established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the methods. However, the results presented in the paper are preliminary, and do not yet imply in anyway that the solutions computed will be stabilizing. More improvements and experimentation will be required before a satisfactory algorithm is developed.展开更多
This paper is concerned with a novel integrated multi-step heuristic dynamic programming(MsHDP)algorithm for solving optimal control problems.It is shown that,initialized by the zero cost function,MsHDP can converge t...This paper is concerned with a novel integrated multi-step heuristic dynamic programming(MsHDP)algorithm for solving optimal control problems.It is shown that,initialized by the zero cost function,MsHDP can converge to the optimal solution of the Hamilton-Jacobi-Bellman(HJB)equation.Then,the stability of the system is analyzed using control policies generated by MsHDP.Also,a general stability criterion is designed to determine the admissibility of the current control policy.That is,the criterion is applicable not only to traditional value iteration and policy iteration but also to MsHDP.Further,based on the convergence and the stability criterion,the integrated MsHDP algorithm using immature control policies is developed to accelerate learning efficiency greatly.Besides,actor-critic is utilized to implement the integrated MsHDP scheme,where neural networks are used to evaluate and improve the iterative policy as the parameter architecture.Finally,two simulation examples are given to demonstrate that the learning effectiveness of the integrated MsHDP scheme surpasses those of other fixed or integrated methods.展开更多
In this paper, we consider the optimal risk sharing problem between two parties in the insurance business: the insurer and the insured. The risk is allocated between the insurer and the insured by setting a deductible...In this paper, we consider the optimal risk sharing problem between two parties in the insurance business: the insurer and the insured. The risk is allocated between the insurer and the insured by setting a deductible and coverage in the insurance contract. We obtain the optimal deductible and coverage by considering the expected product of the two parties' utilities of terminal wealth according to stochastic optimal control theory. An equilibrium policy is also derived for when there are both a deductible and coverage;this is done by modelling the problem as a stochastic game in a continuous-time framework. A numerical example is provided to illustrate the results of the paper.展开更多
文摘An algorithm for numerical solution of discrete Hamilton-Jacobi-Bellman equations is proposed. The method begins with a suitable initial guess value of the solution,then finds a suitable matrix to linearize the system and constructs an iteration algorithm to generate the monotone sequence. The convergence of the algorithm for nonlinear discrete Hamilton-Jacobi-Bellman equations is proved. Some numerical examples are presented to confirm the effciency of this algorithm.
文摘This paper presents a modified domain decomposition method for the numerical solution of discrete Hamilton-Jacobi-Bellman equations arising from a class of optimal control problems using diffusion models. A convergence theorem is established. Numerical results indicate the effectiveness and accuracy of the method.
基金the support by the European Science Foundation Exchange OPTPDE Grantthe support of CADMOS(Center for Advances Modeling and Science)Supported in part by the European Union under Grant Agreement“Multi-ITN STRIKE-Novel Methods in Computational Finance”.Fund Project No.304617 Marie Curie Research Training Network.
文摘In the framework of stochastic processes, the connection between the dynamic programming scheme given by the Hamilton-Jacobi-Bellman equation and a recently proposed control approach based on the Fokker-Planck equation is discussed. Under appropriate assumptions it is shown that the two strategies are equivalent in the case of expected cost functionals, while the Fokker-Planck formalism allows considering a larger classof objectives. To illustratethe connection between the two control strategies, the cases of an Itō stochastic process and of a piecewise-deterministic process are considered.
文摘In this paper, we propose an iterative relaxation method for solving the Hamilton-Jacobi-Bellman-Isaacs equation(HJBIE) arising in deterministic optimal control of affine nonlinear systems. Local convergence of the method is established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the method. An extension of the approach to Lyapunov equations is also discussed. The preliminary results presented are promising, and it is hoped that the approach will ultimately develop into an efficient computational tool for solving the HJBIEs.
文摘In this paper, iterative or successive approximation methods for the Hamilton-Jacobi-Bellman-lsaacs equations （HJBIEs） arising in both deterministic and stochastic optimal control for affine nonlinear systems are developed. Convergence of the methods are established under fairly mild assumptions, and examples are solved to demonstrate the effectiveness of the methods. However, the results presented in the paper are preliminary, and do not yet imply in anyway that the solutions computed will be stabilizing. More improvements and experimentation will be required before a satisfactory algorithm is developed.
基金the National Key Research and Development Program of China(2021ZD0112302)the National Natural Science Foundation of China(62222301,61890930-5,62021003)the Beijing Natural Science Foundation(JQ19013).
文摘This paper is concerned with a novel integrated multi-step heuristic dynamic programming(MsHDP)algorithm for solving optimal control problems.It is shown that,initialized by the zero cost function,MsHDP can converge to the optimal solution of the Hamilton-Jacobi-Bellman(HJB)equation.Then,the stability of the system is analyzed using control policies generated by MsHDP.Also,a general stability criterion is designed to determine the admissibility of the current control policy.That is,the criterion is applicable not only to traditional value iteration and policy iteration but also to MsHDP.Further,based on the convergence and the stability criterion,the integrated MsHDP algorithm using immature control policies is developed to accelerate learning efficiency greatly.Besides,actor-critic is utilized to implement the integrated MsHDP scheme,where neural networks are used to evaluate and improve the iterative policy as the parameter architecture.Finally,two simulation examples are given to demonstrate that the learning effectiveness of the integrated MsHDP scheme surpasses those of other fixed or integrated methods.
基金supported by the NSF of China(11931018, 12271274)the Tianjin Natural Science Foundation (19JCYBJC30400)。
文摘In this paper, we consider the optimal risk sharing problem between two parties in the insurance business: the insurer and the insured. The risk is allocated between the insurer and the insured by setting a deductible and coverage in the insurance contract. We obtain the optimal deductible and coverage by considering the expected product of the two parties' utilities of terminal wealth according to stochastic optimal control theory. An equilibrium policy is also derived for when there are both a deductible and coverage;this is done by modelling the problem as a stochastic game in a continuous-time framework. A numerical example is provided to illustrate the results of the paper.
