In this paper, we introduce a novel reinforcement learning(RL) scheme for linear continuous-time dynamical systems. Different from traditional batch learning algorithms,an incremental learning approach is developed, w...In this paper, we introduce a novel reinforcement learning(RL) scheme for linear continuous-time dynamical systems. Different from traditional batch learning algorithms,an incremental learning approach is developed, which provides a more efficient way to tackle the on-line learning problem in realworld applications. We provide concrete convergence and robust analysis on this incremental-learning algorithm. An extension to solving robust optimal control problems is also given. Two simulation examples are also given to illustrate the effectiveness of our theoretical result.展开更多
This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discretetime systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-vary...This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discretetime systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-varying discrete-time systems is presented. Its connections with existing in finite-horizon PI methods are discussed. Then, both data-drive n off-policy PI and Value Iteration (VI) algorithms are derived to find approximate optimal controllers when the system dynamics is completely unknown. Under mild conditions, the proposed data-driven off-policy algorithms converge to the optimal solution. Finally, the effectiveness and feasibility of the developed methods are validated by a practical example of spacecraft attitude control.展开更多
The classic polynomial chaos method（PCM）, characterized as an intrusive methodology,has been applied to uncertainty propagation（UP） in many dynamic systems. However, the intrusive polynomial chaos method（IPCM） r...The classic polynomial chaos method（PCM）, characterized as an intrusive methodology,has been applied to uncertainty propagation（UP） in many dynamic systems. However, the intrusive polynomial chaos method（IPCM） requires tedious modification of the governing equations, which might introduce errors and can be impractical. Alternative to IPCM, the non-intrusive polynomial chaos method（NIPCM） that avoids such modifications has been developed. In spite of the frequent application to dynamic problems, almost all the existing works about NIPCM for dynamic UP fail to elaborate the implementation process in a straightforward way, which is important to readers who are unfamiliar with the mathematics of the polynomial chaos theory. Meanwhile, very few works have compared NIPCM to IPCM in terms of their merits and applicability. Therefore, the mathematic procedure of dynamic UP via both methods considering parametric and initial condition uncertainties are comparatively discussed and studied in the present paper. Comparison of accuracy and efficiency in statistic moment estimation is made by applying the two methods to several dynamic UP problems. The relative merits of both approaches are discussed and summarized. The detailed description and insights gained with the two methods through this work are expected to be helpful to engineering designers in solving dynamic UP problems.展开更多
基金supported partially by the National Science Foundation(ECCS-1230040 and ECCS-1501044)
文摘In this paper, we introduce a novel reinforcement learning(RL) scheme for linear continuous-time dynamical systems. Different from traditional batch learning algorithms,an incremental learning approach is developed, which provides a more efficient way to tackle the on-line learning problem in realworld applications. We provide concrete convergence and robust analysis on this incremental-learning algorithm. An extension to solving robust optimal control problems is also given. Two simulation examples are also given to illustrate the effectiveness of our theoretical result.
基金The work of B. Pang and Z.-P. Jiang has been supported in part by the National Science Foundation (No. ECCS-1501044).
文摘This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discretetime systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-varying discrete-time systems is presented. Its connections with existing in finite-horizon PI methods are discussed. Then, both data-drive n off-policy PI and Value Iteration (VI) algorithms are derived to find approximate optimal controllers when the system dynamics is completely unknown. Under mild conditions, the proposed data-driven off-policy algorithms converge to the optimal solution. Finally, the effectiveness and feasibility of the developed methods are validated by a practical example of spacecraft attitude control.
基金supported by the National Natural Science Foundation of China (No. 51105034)the Doctoral Thesis Build Project of Beijing 2012 (China)
文摘The classic polynomial chaos method（PCM）, characterized as an intrusive methodology,has been applied to uncertainty propagation（UP） in many dynamic systems. However, the intrusive polynomial chaos method（IPCM） requires tedious modification of the governing equations, which might introduce errors and can be impractical. Alternative to IPCM, the non-intrusive polynomial chaos method（NIPCM） that avoids such modifications has been developed. In spite of the frequent application to dynamic problems, almost all the existing works about NIPCM for dynamic UP fail to elaborate the implementation process in a straightforward way, which is important to readers who are unfamiliar with the mathematics of the polynomial chaos theory. Meanwhile, very few works have compared NIPCM to IPCM in terms of their merits and applicability. Therefore, the mathematic procedure of dynamic UP via both methods considering parametric and initial condition uncertainties are comparatively discussed and studied in the present paper. Comparison of accuracy and efficiency in statistic moment estimation is made by applying the two methods to several dynamic UP problems. The relative merits of both approaches are discussed and summarized. The detailed description and insights gained with the two methods through this work are expected to be helpful to engineering designers in solving dynamic UP problems.
