Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and ...Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.展开更多
An optimal tracking control problem for a class of nonlinear systems with guaranteed performance and asymmetric input constraints is discussed in this paper.The control policy is implemented by adaptive dynamic progra...An optimal tracking control problem for a class of nonlinear systems with guaranteed performance and asymmetric input constraints is discussed in this paper.The control policy is implemented by adaptive dynamic programming(ADP)algorithm under two event-based triggering mechanisms.It is often challenging to design an optimal control law due to the system deviation caused by asymmetric input constraints.First,a prescribed performance control technique is employed to guarantee the tracking errors within predetermined boundaries.Subsequently,considering the asymmetric input constraints,a discounted non-quadratic cost function is introduced.Moreover,in order to reduce controller updates,an event-triggered control law is developed for ADP algorithm.After that,to further simplify the complexity of controller design,this work is extended to a self-triggered case for relaxing the need for continuous signal monitoring by hardware devices.By employing the Lyapunov method,the uniform ultimate boundedness of all signals is proved to be guaranteed.Finally,a simulation example on a mass–spring–damper system subject to asymmetric input constraints is provided to validate the effectiveness of the proposed control scheme.展开更多
In order to address the output feedback issue for linear discrete-time systems, this work suggests a brand-new adaptive dynamic programming(ADP) technique based on the internal model principle(IMP). The proposed metho...In order to address the output feedback issue for linear discrete-time systems, this work suggests a brand-new adaptive dynamic programming(ADP) technique based on the internal model principle(IMP). The proposed method, termed as IMP-ADP, does not require complete state feedback-merely the measurement of input and output data. More specifically, based on the IMP, the output control problem can first be converted into a stabilization problem. We then design an observer to reproduce the full state of the system by measuring the inputs and outputs. Moreover, this technique includes both a policy iteration algorithm and a value iteration algorithm to determine the optimal feedback gain without using a dynamic system model. It is important that with this concept one does not need to solve the regulator equation. Finally, this control method was tested on an inverter system of grid-connected LCLs to demonstrate that the proposed method provides the desired performance in terms of both tracking and disturbance rejection.展开更多
We applied adaptive dynamics to double slit interference phenomenon using particle model and obtained partial successful results in our previous report. The patterns qualitatively corresponded well with experiments. S...We applied adaptive dynamics to double slit interference phenomenon using particle model and obtained partial successful results in our previous report. The patterns qualitatively corresponded well with experiments. Several properties such as concave single slit pattern and large influence of slight displacement of the emission position were different from the experimental results. In this study we tried other slit conditions and obtained consistent patterns with experiments. We do not claim that the adaptive dynamics is the principle of quantum mechanics, but the present results support the probability of adaptive dynamics as the candidate of the basis of quantum mechanics. We discuss the advantages of the adaptive dynamical view for foundations of quantum mechanics.展开更多
In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagno...In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagnosis performance in sparsely labeled or unlabeled target domain,which has been widely used for cross domain fault diagnosis.However,existing methods focus on either marginal distribution adaptation(MDA)or conditional distribution adaptation(CDA).In practice,marginal and conditional distributions discrepancies both have significant but different influences on the domain divergence.In this paper,a dynamic distribution adaptation based transfer network(DDATN)is proposed for cross domain bearing fault diagnosis.DDATN utilizes the proposed instance-weighted dynamic maximum mean discrepancy(IDMMD)for dynamic distribution adaptation(DDA),which can dynamically estimate the influences of marginal and conditional distribution and adapt target domain with source domain.The experimental evaluation on cross domain bearing fault diagnosis demonstrates that DDATN can outperformance the state-of-the-art cross domain fault diagnosis methods.展开更多
A policy iteration algorithm of adaptive dynamic programming(ADP) is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking prob...A policy iteration algorithm of adaptive dynamic programming(ADP) is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking problem is transformed into an optimal regulation one. The policy iteration algorithm for discrete-time chaotic systems is first described. Then,the convergence and admissibility properties of the developed policy iteration algorithm are presented, which show that the transformed chaotic system can be stabilized under an arbitrary iterative control law and the iterative performance index function simultaneously converges to the optimum. By implementing the policy iteration algorithm via neural networks,the developed optimal tracking control scheme for chaotic systems is verified by a simulation.展开更多
Traditional biomechanical analyses of human movement are generally derived from linear mathematics.While these methods can be useful in many situations,they do not describe behaviors in human systems that are predomin...Traditional biomechanical analyses of human movement are generally derived from linear mathematics.While these methods can be useful in many situations,they do not describe behaviors in human systems that are predominately nonlinear.For this reason,nonlinear analysis methods based on a dynamical systems approach have become more prevalent in recent literature.These analysis techniques have provided new insights into how systems(1) maintain pattern stability,(2) transition into new states,and(3) are governed by short-and long-term(fractal) correlational processes at different spatio-temporal scales.These different aspects of system dynamics are typically investigated using concepts related to variability,stability,complexity,and adaptability.The purpose of this paper is to compare and contrast these different concepts and demonstrate that,although related,these terms represent fundamentally different aspects of system dynamics.In particular,we argue that variability should not uniformly be equated with stability or complexity of movement.In addition,current dynamic stability measures based on nonlinear analysis methods(such as the finite maximal Lyapunov exponent) can reveal local instabilities in movement dynamics,but the degree to which these local instabilities relate to global postural and gait stability and the ability to resist external perturbations remains to be explored.Finally,systematic studies are needed to relate observed reductions in complexity with aging and disease to the adaptive capabilities of the movement system and how complexity changes as a function of different task constraints.展开更多
Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iterati...Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iteration(DTTV)algorithm,is developed.The iterative control law is designed to update the iterative value function which approximates the index function of optimal performance.The admissibility of the iterative control law is analyzed.The results show that the iterative value function is non-increasingly convergent to the Bellman-equation optimal solution.To implement the algorithm,neural networks are employed and a new implementation structure is established,which avoids solving the generalized Bellman equation in each iteration.Finally,the optimal control laws for torsional pendulum and inverted pendulum systems are obtained by using the DTTV policy iteration algorithm,where the mass and pendulum bar length are permitted to be time-varying parameters.The effectiveness of the developed method is illustrated by numerical results and comparisons.展开更多
As a new mode and means of smart manufacturing,smart cloud manufacturing(SCM)faces great challenges in massive supply and demand,dynamic resource collaboration and intelligent adaptation.To address the problem,this pa...As a new mode and means of smart manufacturing,smart cloud manufacturing(SCM)faces great challenges in massive supply and demand,dynamic resource collaboration and intelligent adaptation.To address the problem,this paper proposes an SCM-oriented dynamic supply-demand(SD)intelligent adaptation model for massive manufacturing services.In this model,a collaborative network model is established based on the properties of both the supply-demand and their relationships;in addition,an algorithm based on deep graph clustering(DGC)and aligned sampling(AS)is used to divide and conquer the large adaptation domain to solve the problem of the slow computational speed caused by the high complexity of spatiotemporal search in the collaborative network model.At the same time,an intelligent supply-demand adaptation method driven by the quality of service(QoS)is established,in which the experiences of adaptation are shared among adaptation subdomains through deep reinforcement learning(DRL)powered by a transfer mechanism to improve the poor adaptation results caused by dynamic uncertainty.The results show that the model and the solution proposed in this paper can performcollaborative and intelligent supply-demand adaptation for themassive and dynamic resources in SCM through autonomous learning and can effectively performglobal supply-demand matching and optimal resource allocation.展开更多
We develop an optimal tracking control method for chaotic system with unknown dynamics and disturbances. The method allows the optimal cost function and the corresponding tracking control to update synchronously. Acco...We develop an optimal tracking control method for chaotic system with unknown dynamics and disturbances. The method allows the optimal cost function and the corresponding tracking control to update synchronously. According to the tracking error and the reference dynamics, the augmented system is constructed. Then the optimal tracking control problem is defined. The policy iteration (PI) is introduced to solve the rain-max optimization problem. The off-policy adaptive dynamic programming (ADP) algorithm is then proposed to find the solution of the tracking Hamilton-Jacobi- Isaacs (HJI) equation online only using measured data and without any knowledge about the system dynamics. Critic neural network (CNN), action neural network (ANN), and disturbance neural network (DNN) are used to approximate the cost function, control, and disturbance. The weights of these networks compose the augmented weight matrix, and the uniformly ultimately bounded (UUB) of which is proven. The convergence of the tracking error system is also proven. Two examples are given to show the effectiveness of the proposed synchronous solution method for the chaotic system tracking problem.展开更多
The convergence and stability of a value-iteration-based adaptive dynamic programming (ADP) algorithm are con- sidered for discrete-time nonlinear systems accompanied by a discounted quadric performance index. More ...The convergence and stability of a value-iteration-based adaptive dynamic programming (ADP) algorithm are con- sidered for discrete-time nonlinear systems accompanied by a discounted quadric performance index. More importantly than sufficing to achieve a good approximate structure, the iterative feedback control law must guarantee the closed-loop stability. Specifically, it is firstly proved that the iterative value function sequence will precisely converge to the optimum. Secondly, the necessary and sufficient condition of the optimal value function serving as a Lyapunov function is investi- gated. We prove that for the case of infinite horizon, there exists a finite horizon length of which the iterative feedback control law will provide stability, and this increases the practicability of the proposed value iteration algorithm. Neural networks (NNs) are employed to approximate the value functions and the optimal feedback control laws, and the approach allows the implementation of the algorithm without knowing the internal dynamics of the system. Finally, a simulation example is employed to demonstrate the effectiveness of the developed optimal control method.展开更多
This paper presents a novel cooperative value iteration(VI)-based adaptive dynamic programming method for multi-player differential game models with a convergence proof.The players are divided into two groups in the l...This paper presents a novel cooperative value iteration(VI)-based adaptive dynamic programming method for multi-player differential game models with a convergence proof.The players are divided into two groups in the learning process and adapt their policies sequentially.Our method removes the dependence of admissible initial policies,which is one of the main drawbacks of the PI-based frameworks.Furthermore,this algorithm enables the players to adapt their control policies without full knowledge of others’ system parameters or control laws.The efficacy of our method is illustrated by three examples.展开更多
A dynamics-based adaptive control approach is proposed for a planar dual-arm space robot in the presence of closed-loop constraints and uncertain inertial parameters of the payload. The controller is capable of contro...A dynamics-based adaptive control approach is proposed for a planar dual-arm space robot in the presence of closed-loop constraints and uncertain inertial parameters of the payload. The controller is capable of controlling the po- sition and attitude of both the satellite base and the payload grasped by the manipulator end effectors. The equations of motion in reduced-order form for the constrained system are derived by incorporating the constraint equations in terms of accelerations into Kane's equations of the unconstrained system. Model analysis shows that the resulting equations perfectly meet the requirement of adaptive controller design. Consequently, by using an indirect approach, an adaptive control scheme is proposed to accomplish position/attitude trajectory tracking control with the uncertain parameters be- ing estimated on-line. The actuator redundancy due to the closed-loop constraints is utilized to minimize a weighted norm of the joint torques. Global asymptotic stability is proven by using Lyapunov's method, and simulation results are also presented to demonstrate the effectiveness of the proposed approach.展开更多
This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is int...This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.展开更多
The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable ener...The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming(ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First,the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions.Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.展开更多
In this paper, a data-based fault tolerant control(FTC) scheme is investigated for unknown continuous-time(CT)affine nonlinear systems with actuator faults. First, a neural network(NN) identifier based on particle swa...In this paper, a data-based fault tolerant control(FTC) scheme is investigated for unknown continuous-time(CT)affine nonlinear systems with actuator faults. First, a neural network(NN) identifier based on particle swarm optimization(PSO) is constructed to model the unknown system dynamics. By utilizing the estimated system states, the particle swarm optimized critic neural network(PSOCNN) is employed to solve the Hamilton-Jacobi-Bellman equation(HJBE) more efficiently.Then, a data-based FTC scheme, which consists of the NN identifier and the fault compensator, is proposed to achieve actuator fault tolerance. The stability of the closed-loop system under actuator faults is guaranteed by the Lyapunov stability theorem. Finally, simulations are provided to demonstrate the effectiveness of the developed method.展开更多
In this paper,an adaptive dynamic programming(ADP)strategy is investigated for discrete-time nonlinear systems with unknown nonlinear dynamics subject to input saturation.To save the communication resources between th...In this paper,an adaptive dynamic programming(ADP)strategy is investigated for discrete-time nonlinear systems with unknown nonlinear dynamics subject to input saturation.To save the communication resources between the controller and the actuators,stochastic communication protocols(SCPs)are adopted to schedule the control signal,and therefore the closed-loop system is essentially a protocol-induced switching system.A neural network(NN)-based identifier with a robust term is exploited for approximating the unknown nonlinear system,and a set of switch-based updating rules with an additional tunable parameter of NN weights are developed with the help of the gradient descent.By virtue of a novel Lyapunov function,a sufficient condition is proposed to achieve the stability of both system identification errors and the update dynamics of NN weights.Then,a value iterative ADP algorithm in an offline way is proposed to solve the optimal control of protocol-induced switching systems with saturation constraints,and the convergence is profoundly discussed in light of mathematical induction.Furthermore,an actor-critic NN scheme is developed to approximate the control law and the proposed performance index function in the framework of ADP,and the stability of the closed-loop system is analyzed in view of the Lyapunov theory.Finally,the numerical simulation results are presented to demonstrate the effectiveness of the proposed control scheme.展开更多
The impact dynamics, impact effect, and post-impact unstable motion sup- pression of free-floating space manipulator capturing a satellite on orbit are analyzed. Firstly, the dynamics equation of free-floating space m...The impact dynamics, impact effect, and post-impact unstable motion sup- pression of free-floating space manipulator capturing a satellite on orbit are analyzed. Firstly, the dynamics equation of free-floating space manipulator is derived using the sec- ond Lagrangian equation. Combining the momentum conservation principle, the impact dynamics and effect between the space manipulator end-effector and satellite of the cap- ture process are analyzed with the momentum impulse method. Focusing on the unstable motion of space manipulator due to the above impact effect, a robust adaptive compound control algorithm is designed to suppress the above unstable motion. There is no need to control the free-floating base position to save the jet fuel. Finally, the simulation is proposed to show the impact effect and verify the validity of the control algorithm.展开更多
This paper aims at eliminating the asymmetric and saturated hysteresis nonlinearities by designing hysteresis pseudo inverse compensator and robust adaptive dynamic surface control(DSC)scheme.The"pseudo inverse&q...This paper aims at eliminating the asymmetric and saturated hysteresis nonlinearities by designing hysteresis pseudo inverse compensator and robust adaptive dynamic surface control(DSC)scheme.The"pseudo inverse"means that an on-line calculation mechanism of approximate control signal is developed by applying a searching method to the designed temporary control signal where the true control signal is included.The main contributions are summarized as:1)to our best knowledge,it is the first time to compensate the asymmetric and saturated hysteresis by using hysteresis pseudo inverse compensator because the construction of the true saturated-type hysteresis inverse model is very difficult;2)by designing the saturated-type hysteresis pseudo inverse compensator,the construction of true explicit hysteresis inverse and the identifications of its corresponding unknown parameters are not required when dealing with the saturated-type hysteresis;3)by combining DSC technique with the tracking error transformed function,the"explosion of complexity"problem in backstepping method is overcome and the prespecified tracking performance is achieved.Analysis of stability and experimental results on the hardware-inloop platform illustrate the effectiveness of the proposed adaptive pseudo inverse control scheme.展开更多
Based on consideration of the differential relations between the immeasurable variables and measurable variables in electro-hydraulic servo system,adaptive dynamic recurrent fuzzy neural networks(ADRFNNs) were employe...Based on consideration of the differential relations between the immeasurable variables and measurable variables in electro-hydraulic servo system,adaptive dynamic recurrent fuzzy neural networks(ADRFNNs) were employed to identify the primary uncertainty and the mathematic model of the system was turned into an equivalent linear model with terms of secondary uncertainty.At the same time,gain adaptive sliding mode variable structure control(GASMVSC) was employed to synthesize the control effort.The results show that the unrealization problem caused by some system's immeasurable state variables in traditional fuzzy neural networks(TFNN) taking all state variables as its inputs is overcome.On the other hand,the identification by the ADRFNNs online with high accuracy and the adaptive function of the correction term's gain in the GASMVSC make the system possess strong robustness and improved steady accuracy,and the chattering phenomenon of the control effort is also suppressed effectively.展开更多
基金supported in part by the National Natural Science Foundation of China(62222301, 62073085, 62073158, 61890930-5, 62021003)the National Key Research and Development Program of China (2021ZD0112302, 2021ZD0112301, 2018YFC1900800-5)Beijing Natural Science Foundation (JQ19013)。
文摘Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
基金supported in part by the National Natural Science Foundation of China(62033003,62003093,62373113,U23A20341,U21A20522)the Natural Science Foundation of Guangdong Province,China(2023A1515011527,2022A1515011506).
文摘An optimal tracking control problem for a class of nonlinear systems with guaranteed performance and asymmetric input constraints is discussed in this paper.The control policy is implemented by adaptive dynamic programming(ADP)algorithm under two event-based triggering mechanisms.It is often challenging to design an optimal control law due to the system deviation caused by asymmetric input constraints.First,a prescribed performance control technique is employed to guarantee the tracking errors within predetermined boundaries.Subsequently,considering the asymmetric input constraints,a discounted non-quadratic cost function is introduced.Moreover,in order to reduce controller updates,an event-triggered control law is developed for ADP algorithm.After that,to further simplify the complexity of controller design,this work is extended to a self-triggered case for relaxing the need for continuous signal monitoring by hardware devices.By employing the Lyapunov method,the uniform ultimate boundedness of all signals is proved to be guaranteed.Finally,a simulation example on a mass–spring–damper system subject to asymmetric input constraints is provided to validate the effectiveness of the proposed control scheme.
基金supported by the National Science Fund for Distinguished Young Scholars (62225303)the Fundamental Research Funds for the Central Universities (buctrc202201)+1 种基金China Scholarship Council,and High Performance Computing PlatformCollege of Information Science and Technology,Beijing University of Chemical Technology。
文摘In order to address the output feedback issue for linear discrete-time systems, this work suggests a brand-new adaptive dynamic programming(ADP) technique based on the internal model principle(IMP). The proposed method, termed as IMP-ADP, does not require complete state feedback-merely the measurement of input and output data. More specifically, based on the IMP, the output control problem can first be converted into a stabilization problem. We then design an observer to reproduce the full state of the system by measuring the inputs and outputs. Moreover, this technique includes both a policy iteration algorithm and a value iteration algorithm to determine the optimal feedback gain without using a dynamic system model. It is important that with this concept one does not need to solve the regulator equation. Finally, this control method was tested on an inverter system of grid-connected LCLs to demonstrate that the proposed method provides the desired performance in terms of both tracking and disturbance rejection.
文摘We applied adaptive dynamics to double slit interference phenomenon using particle model and obtained partial successful results in our previous report. The patterns qualitatively corresponded well with experiments. Several properties such as concave single slit pattern and large influence of slight displacement of the emission position were different from the experimental results. In this study we tried other slit conditions and obtained consistent patterns with experiments. We do not claim that the adaptive dynamics is the principle of quantum mechanics, but the present results support the probability of adaptive dynamics as the candidate of the basis of quantum mechanics. We discuss the advantages of the adaptive dynamical view for foundations of quantum mechanics.
基金Supported by National Natural Science Foundation of China(Grant Nos.51875208,51475170)National Key Research and Development Program of China(Grant No.2018YFB1702400).
文摘In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagnosis performance in sparsely labeled or unlabeled target domain,which has been widely used for cross domain fault diagnosis.However,existing methods focus on either marginal distribution adaptation(MDA)or conditional distribution adaptation(CDA).In practice,marginal and conditional distributions discrepancies both have significant but different influences on the domain divergence.In this paper,a dynamic distribution adaptation based transfer network(DDATN)is proposed for cross domain bearing fault diagnosis.DDATN utilizes the proposed instance-weighted dynamic maximum mean discrepancy(IDMMD)for dynamic distribution adaptation(DDA),which can dynamically estimate the influences of marginal and conditional distribution and adapt target domain with source domain.The experimental evaluation on cross domain bearing fault diagnosis demonstrates that DDATN can outperformance the state-of-the-art cross domain fault diagnosis methods.
基金supported by the National Natural Science Foundation of China(Grant Nos.61034002,61233001,61273140,61304086,and 61374105)the Beijing Natural Science Foundation,China(Grant No.4132078)
文摘A policy iteration algorithm of adaptive dynamic programming(ADP) is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking problem is transformed into an optimal regulation one. The policy iteration algorithm for discrete-time chaotic systems is first described. Then,the convergence and admissibility properties of the developed policy iteration algorithm are presented, which show that the transformed chaotic system can be stabilized under an arbitrary iterative control law and the iterative performance index function simultaneously converges to the optimum. By implementing the policy iteration algorithm via neural networks,the developed optimal tracking control scheme for chaotic systems is verified by a simulation.
文摘Traditional biomechanical analyses of human movement are generally derived from linear mathematics.While these methods can be useful in many situations,they do not describe behaviors in human systems that are predominately nonlinear.For this reason,nonlinear analysis methods based on a dynamical systems approach have become more prevalent in recent literature.These analysis techniques have provided new insights into how systems(1) maintain pattern stability,(2) transition into new states,and(3) are governed by short-and long-term(fractal) correlational processes at different spatio-temporal scales.These different aspects of system dynamics are typically investigated using concepts related to variability,stability,complexity,and adaptability.The purpose of this paper is to compare and contrast these different concepts and demonstrate that,although related,these terms represent fundamentally different aspects of system dynamics.In particular,we argue that variability should not uniformly be equated with stability or complexity of movement.In addition,current dynamic stability measures based on nonlinear analysis methods(such as the finite maximal Lyapunov exponent) can reveal local instabilities in movement dynamics,but the degree to which these local instabilities relate to global postural and gait stability and the ability to resist external perturbations remains to be explored.Finally,systematic studies are needed to relate observed reductions in complexity with aging and disease to the adaptive capabilities of the movement system and how complexity changes as a function of different task constraints.
基金supported in part by Fundamental Research Funds for the Central Universities(2022JBZX024)in part by the National Natural Science Foundation of China(61872037,61273167)。
文摘Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iteration(DTTV)algorithm,is developed.The iterative control law is designed to update the iterative value function which approximates the index function of optimal performance.The admissibility of the iterative control law is analyzed.The results show that the iterative value function is non-increasingly convergent to the Bellman-equation optimal solution.To implement the algorithm,neural networks are employed and a new implementation structure is established,which avoids solving the generalized Bellman equation in each iteration.Finally,the optimal control laws for torsional pendulum and inverted pendulum systems are obtained by using the DTTV policy iteration algorithm,where the mass and pendulum bar length are permitted to be time-varying parameters.The effectiveness of the developed method is illustrated by numerical results and comparisons.
基金This paper was supported in part by the National Natural Science Foundation of China under Grant 62172235in part by Natural Science Foundation of Jiangsu Province of China under Grant BK20191381in part by Primary Research&Development Plan of Jiangsu Province Grant BE2019742.
文摘As a new mode and means of smart manufacturing,smart cloud manufacturing(SCM)faces great challenges in massive supply and demand,dynamic resource collaboration and intelligent adaptation.To address the problem,this paper proposes an SCM-oriented dynamic supply-demand(SD)intelligent adaptation model for massive manufacturing services.In this model,a collaborative network model is established based on the properties of both the supply-demand and their relationships;in addition,an algorithm based on deep graph clustering(DGC)and aligned sampling(AS)is used to divide and conquer the large adaptation domain to solve the problem of the slow computational speed caused by the high complexity of spatiotemporal search in the collaborative network model.At the same time,an intelligent supply-demand adaptation method driven by the quality of service(QoS)is established,in which the experiences of adaptation are shared among adaptation subdomains through deep reinforcement learning(DRL)powered by a transfer mechanism to improve the poor adaptation results caused by dynamic uncertainty.The results show that the model and the solution proposed in this paper can performcollaborative and intelligent supply-demand adaptation for themassive and dynamic resources in SCM through autonomous learning and can effectively performglobal supply-demand matching and optimal resource allocation.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.61304079,61673054,and 61374105)the Fundamental Research Funds for the Central Universities,China(Grant No.FRF-TP-15-056A3)the Open Research Project from SKLMCCS,China(Grant No.20150104)
文摘We develop an optimal tracking control method for chaotic system with unknown dynamics and disturbances. The method allows the optimal cost function and the corresponding tracking control to update synchronously. According to the tracking error and the reference dynamics, the augmented system is constructed. Then the optimal tracking control problem is defined. The policy iteration (PI) is introduced to solve the rain-max optimization problem. The off-policy adaptive dynamic programming (ADP) algorithm is then proposed to find the solution of the tracking Hamilton-Jacobi- Isaacs (HJI) equation online only using measured data and without any knowledge about the system dynamics. Critic neural network (CNN), action neural network (ANN), and disturbance neural network (DNN) are used to approximate the cost function, control, and disturbance. The weights of these networks compose the augmented weight matrix, and the uniformly ultimately bounded (UUB) of which is proven. The convergence of the tracking error system is also proven. Two examples are given to show the effectiveness of the proposed synchronous solution method for the chaotic system tracking problem.
文摘The convergence and stability of a value-iteration-based adaptive dynamic programming (ADP) algorithm are con- sidered for discrete-time nonlinear systems accompanied by a discounted quadric performance index. More importantly than sufficing to achieve a good approximate structure, the iterative feedback control law must guarantee the closed-loop stability. Specifically, it is firstly proved that the iterative value function sequence will precisely converge to the optimum. Secondly, the necessary and sufficient condition of the optimal value function serving as a Lyapunov function is investi- gated. We prove that for the case of infinite horizon, there exists a finite horizon length of which the iterative feedback control law will provide stability, and this increases the practicability of the proposed value iteration algorithm. Neural networks (NNs) are employed to approximate the value functions and the optimal feedback control laws, and the approach allows the implementation of the algorithm without knowing the internal dynamics of the system. Finally, a simulation example is employed to demonstrate the effectiveness of the developed optimal control method.
基金supported by the Industry-University-Research Cooperation Fund Project of the Eighth Research Institute of China Aerospace Science and Technology Corporation (USCAST2022-11)Aeronautical Science Foundation of China (20220001057001)。
文摘This paper presents a novel cooperative value iteration(VI)-based adaptive dynamic programming method for multi-player differential game models with a convergence proof.The players are divided into two groups in the learning process and adapt their policies sequentially.Our method removes the dependence of admissible initial policies,which is one of the main drawbacks of the PI-based frameworks.Furthermore,this algorithm enables the players to adapt their control policies without full knowledge of others’ system parameters or control laws.The efficacy of our method is illustrated by three examples.
基金supported by the National Natural Science Foundation of China(11272027)
文摘A dynamics-based adaptive control approach is proposed for a planar dual-arm space robot in the presence of closed-loop constraints and uncertain inertial parameters of the payload. The controller is capable of controlling the po- sition and attitude of both the satellite base and the payload grasped by the manipulator end effectors. The equations of motion in reduced-order form for the constrained system are derived by incorporating the constraint equations in terms of accelerations into Kane's equations of the unconstrained system. Model analysis shows that the resulting equations perfectly meet the requirement of adaptive controller design. Consequently, by using an indirect approach, an adaptive control scheme is proposed to accomplish position/attitude trajectory tracking control with the uncertain parameters be- ing estimated on-line. The actuator redundancy due to the closed-loop constraints is utilized to minimize a weighted norm of the joint torques. Global asymptotic stability is proven by using Lyapunov's method, and simulation results are also presented to demonstrate the effectiveness of the proposed approach.
基金supported in part by the National Key Reseanch and Development Program of China(2018AAA0101502,2018YFB1702300)in part by the National Natural Science Foundation of China(61722312,61533019,U1811463,61533017)in part by the Intel Collaborative Research Institute for Intelligent and Automated Connected Vehicles。
文摘This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.
基金supported in part by the National Natural Science Foundation of China(61533017,U1501251,61374105,61722312)
文摘The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming(ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First,the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions.Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.
基金supported in part by the National Natural ScienceFoundation of China(61533017,61973330,61773075,61603387)the Early Career Development Award of SKLMCCS(20180201)the State Key Laboratory of Synthetical Automation for Process Industries(2019-KF-23-03)。
文摘In this paper, a data-based fault tolerant control(FTC) scheme is investigated for unknown continuous-time(CT)affine nonlinear systems with actuator faults. First, a neural network(NN) identifier based on particle swarm optimization(PSO) is constructed to model the unknown system dynamics. By utilizing the estimated system states, the particle swarm optimized critic neural network(PSOCNN) is employed to solve the Hamilton-Jacobi-Bellman equation(HJBE) more efficiently.Then, a data-based FTC scheme, which consists of the NN identifier and the fault compensator, is proposed to achieve actuator fault tolerance. The stability of the closed-loop system under actuator faults is guaranteed by the Lyapunov stability theorem. Finally, simulations are provided to demonstrate the effectiveness of the developed method.
基金supported in part by the Australian Research Council Discovery Early Career Researcher Award(DE200101128)Australian Research Council(DP190101557)。
文摘In this paper,an adaptive dynamic programming(ADP)strategy is investigated for discrete-time nonlinear systems with unknown nonlinear dynamics subject to input saturation.To save the communication resources between the controller and the actuators,stochastic communication protocols(SCPs)are adopted to schedule the control signal,and therefore the closed-loop system is essentially a protocol-induced switching system.A neural network(NN)-based identifier with a robust term is exploited for approximating the unknown nonlinear system,and a set of switch-based updating rules with an additional tunable parameter of NN weights are developed with the help of the gradient descent.By virtue of a novel Lyapunov function,a sufficient condition is proposed to achieve the stability of both system identification errors and the update dynamics of NN weights.Then,a value iterative ADP algorithm in an offline way is proposed to solve the optimal control of protocol-induced switching systems with saturation constraints,and the convergence is profoundly discussed in light of mathematical induction.Furthermore,an actor-critic NN scheme is developed to approximate the control law and the proposed performance index function in the framework of ADP,and the stability of the closed-loop system is analyzed in view of the Lyapunov theory.Finally,the numerical simulation results are presented to demonstrate the effectiveness of the proposed control scheme.
基金supported by the National Natural Science Foundation of China(Nos.11072061 and 11372073)the Natural Science Foundation of Fujian Province(No.2010J01003)
文摘The impact dynamics, impact effect, and post-impact unstable motion sup- pression of free-floating space manipulator capturing a satellite on orbit are analyzed. Firstly, the dynamics equation of free-floating space manipulator is derived using the sec- ond Lagrangian equation. Combining the momentum conservation principle, the impact dynamics and effect between the space manipulator end-effector and satellite of the cap- ture process are analyzed with the momentum impulse method. Focusing on the unstable motion of space manipulator due to the above impact effect, a robust adaptive compound control algorithm is designed to suppress the above unstable motion. There is no need to control the free-floating base position to save the jet fuel. Finally, the simulation is proposed to show the impact effect and verify the validity of the control algorithm.
基金supported in part by the National Natural Science Foundation of China(61673101,61973131,61733006,U1813201)the Japan Society for the Promotion of Science(C18K04212)+2 种基金the Science and Technology Project of Jilin Province(20180201009SF,20170414011GH,20180201004SF,20180101069JC)the Fundamental Research Funds for the Central Universities(N2008002)“Xing Liao Ying Cai”Program(XLYC1907073)。
文摘This paper aims at eliminating the asymmetric and saturated hysteresis nonlinearities by designing hysteresis pseudo inverse compensator and robust adaptive dynamic surface control(DSC)scheme.The"pseudo inverse"means that an on-line calculation mechanism of approximate control signal is developed by applying a searching method to the designed temporary control signal where the true control signal is included.The main contributions are summarized as:1)to our best knowledge,it is the first time to compensate the asymmetric and saturated hysteresis by using hysteresis pseudo inverse compensator because the construction of the true saturated-type hysteresis inverse model is very difficult;2)by designing the saturated-type hysteresis pseudo inverse compensator,the construction of true explicit hysteresis inverse and the identifications of its corresponding unknown parameters are not required when dealing with the saturated-type hysteresis;3)by combining DSC technique with the tracking error transformed function,the"explosion of complexity"problem in backstepping method is overcome and the prespecified tracking performance is achieved.Analysis of stability and experimental results on the hardware-inloop platform illustrate the effectiveness of the proposed adaptive pseudo inverse control scheme.
基金Project(60634020) supported by the National Natural Science Foundation of China
文摘Based on consideration of the differential relations between the immeasurable variables and measurable variables in electro-hydraulic servo system,adaptive dynamic recurrent fuzzy neural networks(ADRFNNs) were employed to identify the primary uncertainty and the mathematic model of the system was turned into an equivalent linear model with terms of secondary uncertainty.At the same time,gain adaptive sliding mode variable structure control(GASMVSC) was employed to synthesize the control effort.The results show that the unrealization problem caused by some system's immeasurable state variables in traditional fuzzy neural networks(TFNN) taking all state variables as its inputs is overcome.On the other hand,the identification by the ADRFNNs online with high accuracy and the adaptive function of the correction term's gain in the GASMVSC make the system possess strong robustness and improved steady accuracy,and the chattering phenomenon of the control effort is also suppressed effectively.