Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and ...Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.展开更多
In this paper,guaranteed cost attitude tracking con-trol for uncertain quadrotor unmanned aerial vehicle(QUAV)under safety constraints is studied.First,an augmented system is constructed by the tracking error system a...In this paper,guaranteed cost attitude tracking con-trol for uncertain quadrotor unmanned aerial vehicle(QUAV)under safety constraints is studied.First,an augmented system is constructed by the tracking error system and reference system.This transformation aims to convert the tracking control prob-lem into a stabilization control problem.Then,control barrier function and disturbance attenuation function are designed to characterize the violations of safety constraints and tolerance of uncertain disturbances,and they are incorporated into the reward function as penalty items.Based on the modified reward function,the problem is simplified as the optimal regulation problem of the nominal augmented system,and a new Hamilton-Jacobi-Bellman equation is developed.Finally,critic-only rein-forcement learning algorithm with a concurrent learning tech-nique is employed to solve the Hamilton-Jacobi-Bellman equa-tion and obtain the optimal controller.The proposed algorithm can not only ensure the reward function within an upper bound in the presence of uncertain disturbances,but also enforce safety constraints.The performance of the algorithm is evaluated by the numerical simulation.展开更多
In this paper,we present a novel adaptive performance control approach for strict-feedback nonparametric systems with unknown time-varying control coefficients,which mainly includes the following steps.Firstly,by intr...In this paper,we present a novel adaptive performance control approach for strict-feedback nonparametric systems with unknown time-varying control coefficients,which mainly includes the following steps.Firstly,by introducing several key transformation functions and selecting the initial value of the time-varying scaling function,the symmetric prescribed performance with global and semi-global properties can be handled uniformly,without the need for control re-design.Secondly,to handle the problem of unknown time-varying control coefficient with an unknown sign,we propose an enhanced Nussbaum function(ENF)bearing some unique properties and characteristics,with which the complex stability analysis based on specific Nussbaum functions as commonly used is no longer required.Thirdly,by utilizing the core-function information technique,the nonparametric uncertainties in the system are gracefully handled so that no approximator is required.Furthermore,simulation results verify the effectiveness and benefits of the approach.展开更多
In this paper,we consider the practical prescribed-time performance guaranteed tracking control problem for a class of uncertain strict-feedback systems subject to unknown control direction.Due to the existence of unk...In this paper,we consider the practical prescribed-time performance guaranteed tracking control problem for a class of uncertain strict-feedback systems subject to unknown control direction.Due to the existence of unknown nonlinearities and uncertainties,it is challenging to design a controller that can ensure the stability of closed-loop system within a predetermined finite time while maintaining the specified transient performance.The underlying problem becomes further complex as the control directions are unknown.To deal with the above problems,a special translation function as well as Nussbaum type function are introduced in the prescribed performance control(PPC)framework.Finally,a PPC as well as preset finite time tracking control scheme is designed,and its effectiveness is confirmed by both theoretical analysis and numerical simulation.展开更多
Thank you so much for having me with you today.I want to extend a special thank you to the Evergreen Education Foundation,the National Library of China,and the Department of Information Management of Peking University...Thank you so much for having me with you today.I want to extend a special thank you to the Evergreen Education Foundation,the National Library of China,and the Department of Information Management of Peking University for hosting this wonderful conference.I am honored to be able to speak with you today and look forward to展开更多
基金supported in part by the National Natural Science Foundation of China(62222301, 62073085, 62073158, 61890930-5, 62021003)the National Key Research and Development Program of China (2021ZD0112302, 2021ZD0112301, 2018YFC1900800-5)Beijing Natural Science Foundation (JQ19013)。
文摘Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
基金supported in part by the National Science Foundation of China(62173183)。
文摘In this paper,guaranteed cost attitude tracking con-trol for uncertain quadrotor unmanned aerial vehicle(QUAV)under safety constraints is studied.First,an augmented system is constructed by the tracking error system and reference system.This transformation aims to convert the tracking control prob-lem into a stabilization control problem.Then,control barrier function and disturbance attenuation function are designed to characterize the violations of safety constraints and tolerance of uncertain disturbances,and they are incorporated into the reward function as penalty items.Based on the modified reward function,the problem is simplified as the optimal regulation problem of the nominal augmented system,and a new Hamilton-Jacobi-Bellman equation is developed.Finally,critic-only rein-forcement learning algorithm with a concurrent learning tech-nique is employed to solve the Hamilton-Jacobi-Bellman equa-tion and obtain the optimal controller.The proposed algorithm can not only ensure the reward function within an upper bound in the presence of uncertain disturbances,but also enforce safety constraints.The performance of the algorithm is evaluated by the numerical simulation.
基金supported in part by the National Key Research and Development Program of China(2021ZD0201300)in part by the National Natural Science Foundation of China(61860206008,61933012)。
文摘In this paper,we present a novel adaptive performance control approach for strict-feedback nonparametric systems with unknown time-varying control coefficients,which mainly includes the following steps.Firstly,by introducing several key transformation functions and selecting the initial value of the time-varying scaling function,the symmetric prescribed performance with global and semi-global properties can be handled uniformly,without the need for control re-design.Secondly,to handle the problem of unknown time-varying control coefficient with an unknown sign,we propose an enhanced Nussbaum function(ENF)bearing some unique properties and characteristics,with which the complex stability analysis based on specific Nussbaum functions as commonly used is no longer required.Thirdly,by utilizing the core-function information technique,the nonparametric uncertainties in the system are gracefully handled so that no approximator is required.Furthermore,simulation results verify the effectiveness and benefits of the approach.
基金supported in part by the National Key Research and Development Program of China under grant(No.2022YFB4701400/4701401)by the National Natural Science Foundation of China under grant(No.61991400,No.61991403,No.62250710167,No.61860206008,No.61933012,No.62273064,No.62203078)+2 种基金in part by the National Key Research and Development Program of China under grant(No.2021ZD0201300)in part by the Innovation Support Program for International Students Returning to China under grant(No.cx2022016)in part by the Chongqing Medical Scientific Research Project under grant(No.2022DBXM001).
文摘In this paper,we consider the practical prescribed-time performance guaranteed tracking control problem for a class of uncertain strict-feedback systems subject to unknown control direction.Due to the existence of unknown nonlinearities and uncertainties,it is challenging to design a controller that can ensure the stability of closed-loop system within a predetermined finite time while maintaining the specified transient performance.The underlying problem becomes further complex as the control directions are unknown.To deal with the above problems,a special translation function as well as Nussbaum type function are introduced in the prescribed performance control(PPC)framework.Finally,a PPC as well as preset finite time tracking control scheme is designed,and its effectiveness is confirmed by both theoretical analysis and numerical simulation.
文摘Thank you so much for having me with you today.I want to extend a special thank you to the Evergreen Education Foundation,the National Library of China,and the Department of Information Management of Peking University for hosting this wonderful conference.I am honored to be able to speak with you today and look forward to
