Transmission Dynamics and Optimal Control Strategies of a Hand-Foot-Mouth Disease Model with Treatment and Vaccination Interventions

In this article, the transmission dynamics of a Hand-Foot-Mouth disease model with treatment and vaccination interventions are studied. We calculated the basic reproduction number and proved the global stability of disease-free equilibrium when R0 R0 > 1. Meanwhile, we obtained the optimal control strategies minimizing the cost of intervention and minimizing the infected person. We also give some numerical simulations to verify our theoretical results.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Parallel Control for Optimal Tracking via Adaptive Dynamic Programming

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.

Inverse Optimal Control for Speed-varying Path Following of Marine Vessels with Actuator Dynamics

A controller which is locally optimal near the origin and globally inverse optimal for the nonlinear system is proposed for path following of over actuated marine crafts with actuator dynamics. The motivation is the existence of undesired signals sent to the actuators, which can result in bad behavior in path following. To attenuate the oscillation of the control signal and obtain smooth thrust outputs, the actuator dynamics are added into the ship maneuvering model. Instead of modifying the Line-of-Sight (LOS) guidance law, this proposed controller can easily adjust the vessel speed to minimize the large cross-track error caused by the high vessel speed when it is turning. Numerical simulations demonstrate the validity of this proposed controller.

OPTIMAL HARVESTING CONTROL PROBLEM FOR LINEAR AGE-DEPENDENT POPULATION DYNAMICS

An optimal harvesting problem for linear age-dependent population dynamics is investigated.By Mazur's Theorem,the existence of solutions of the optimal control problem (OH) is demonstrated.The first order necessary conditions of optimality for problem (OH) is obtained by the conception of normal cone. Finally,under suitable assumptions,the uniqueness of solutions of the optimal control problem (OH) is given.The results extend some known criteria.

Optimal design of dynamic and control performance for planar manipulator

A design and optimization approach of dynamic and control performance for a two-DOF planar manipulator was proposed.After the kinematic and dynamic analysis,several advantages of the mechanism were illustrated,which made it possible to obtain good dynamic and control performances just through mechanism optimization.Based on the idea of design for control(DFC),a novel kind of multi-objective optimization model was proposed.There were three optimization objectives:the index of inertia,the index describing the dynamic coupling effects and the global condition number.Other indexes to characterize the designing requirements such as the velocity of end-effector,the workspace size,and the first mode natural frequency were regarded as the constraints.The cross-section area and length of the linkages were chosen as the design variables.NSGA-II algorithm was introduced to solve this complex multi-objective optimization problem.Additional criteria from engineering experience were incorporated into the selecting of final parameters among the obtained Pareto solution sets.Finally,experiments were performed to validate the linear dynamic structure and control performances of the optimized mechanisms.A new expression for measuring the dynamic coupling degree with clear physical meaning was proposed.The results show that the optimized mechanism has an approximate decoupled dynamics structure,and each active joint can be regarded as a linear SISO system.The control performances of the linear and nonlinear controllers were also compared.It can be concluded that the optimized mechanism can achieve good control performance only using a linear controller.

THEORY AND METHOD OF OPTIMAL CONTROL SOLUTION TODYNAMIC SYSTEM PARAMETERS IDENTIFICATION (Ⅰ)──FUNDAMENTAL CONCEPT AND DETERMINISTIC SYSTEM PARAMETERS IDENTIFICATION

Based on the concept of optimal control solution to dynamic system parameters identification and the optimal control theory of deterministic system,dyna-mics system parameters identfication problem is brought into correspondence with optimal control problem. Then the theory and algorithm of optimal control are introduced into the study of dynamic system parameters identification. According to the theory of Hamilton-Jacobi-Bellman (HJB) equations solution, the existence and uniqueness of optimal control solution to dynamic system parameters identification are resolved in this paper. At last, the parameters identification algorithm of determi-nistic dynamic system is presented also based on above mentioned theory and concept.

THEORY AND ALGORITHM OF OPTIMAL CONTROL SOLUTION TO DYNAMIC SYSTEM PARAMETERS IDENTIFICATION(Ⅱ)──STOCHASTIC SYSTEM PARAMETERS IDENTIFICATION AND APPLICATION EXAMPLE

Based on the contents Of part (Ⅰ) and stochastic optimal control theory, the concept of optimal control solution to parameters identification of stochastic dynamic system is discussed at first. For the completeness of the theory developed in this paper and part (Ⅰ), then the procedure of establishing HamiltonJacobi-Bellman (HJB) equations of parameters identification problem is presented.And then, parameters identification algorithm of stochastic dynamic system is introduced. At last, an application example-local nonlinear parameters identification of dynamic system is presented.

THE OPTIMAL CONTROL VARIATIONAL PRINCIPLE AND FINITEELEMENTS ANALYSIS FOR VISCOPLASTIC DYNAMICS

This paper presents the optimal control variational principle for Perzyna modelwhich is one of the main constitutive relation of viscoplasticity in dynamics. And itcould also be transformed to solve the parametric quadratic programming problem.The FEM form of this problem and its implementation have also been discussed in thepaper.

Global Transmission Dynamics of a Schistosomiasis Model and Its Optimal Control

Drug treatment, snail control, cercariae control, improved sanitation and health education are the effective strategies which are used to control the schistosomiasis. In this paper, we consider a deterministic model for schistosomiasis transmission dynamics in order to explore the role of the several control strategies. The global stability of a schistosomiasis infection model that involves mating structure including male schistosomes, female schistosomes, paired schistosomes and snails is studied by constructing appropriate Lyapunov functions. We derive the basic reproduction number R0 for the deterministic model, and establish that the global dynamics are completely determined by the values of R0. We show that the disease can be eradicated when R0?≤1;otherwise, the system is persistent. In the case where ?R0?>1, we prove the existence, uniqueness and global asymptotic stability of an endemic steady state. Sensitivity analysis and simulations are carried out in order to determine the relative importance of different control strategies for disease transmission and prevalence. Next, optimal control theory is applied to investigate the control strategies for eliminating schistosomiasis using time dependent controls. The characterization of the optimal control is carried out via the Pontryagins Maximum Principle. The simulation results demonstrate that the insecticide is important in the control of schistosomiasis.

Policy Iteration for Optimal Control of Discrete-Time Time-Varying Nonlinear Systems

Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iteration(DTTV)algorithm,is developed.The iterative control law is designed to update the iterative value function which approximates the index function of optimal performance.The admissibility of the iterative control law is analyzed.The results show that the iterative value function is non-increasingly convergent to the Bellman-equation optimal solution.To implement the algorithm,neural networks are employed and a new implementation structure is established,which avoids solving the generalized Bellman equation in each iteration.Finally,the optimal control laws for torsional pendulum and inverted pendulum systems are obtained by using the DTTV policy iteration algorithm,where the mass and pendulum bar length are permitted to be time-varying parameters.The effectiveness of the developed method is illustrated by numerical results and comparisons.

Adaptive Multi-Step Evaluation Design With Stability Guarantee for Discrete-Time Optimal Learning Control

This paper is concerned with a novel integrated multi-step heuristic dynamic programming(MsHDP)algorithm for solving optimal control problems.It is shown that,initialized by the zero cost function,MsHDP can converge to the optimal solution of the Hamilton-Jacobi-Bellman(HJB)equation.Then,the stability of the system is analyzed using control policies generated by MsHDP.Also,a general stability criterion is designed to determine the admissibility of the current control policy.That is,the criterion is applicable not only to traditional value iteration and policy iteration but also to MsHDP.Further,based on the convergence and the stability criterion,the integrated MsHDP algorithm using immature control policies is developed to accelerate learning efficiency greatly.Besides,actor-critic is utilized to implement the integrated MsHDP scheme,where neural networks are used to evaluate and improve the iterative policy as the parameter architecture.Finally,two simulation examples are given to demonstrate that the learning effectiveness of the integrated MsHDP scheme surpasses those of other fixed or integrated methods.

Event-based performance guaranteed tracking control for constrained nonlinear system via adaptive dynamic programming method

An optimal tracking control problem for a class of nonlinear systems with guaranteed performance and asymmetric input constraints is discussed in this paper.The control policy is implemented by adaptive dynamic programming(ADP)algorithm under two event-based triggering mechanisms.It is often challenging to design an optimal control law due to the system deviation caused by asymmetric input constraints.First,a prescribed performance control technique is employed to guarantee the tracking errors within predetermined boundaries.Subsequently,considering the asymmetric input constraints,a discounted non-quadratic cost function is introduced.Moreover,in order to reduce controller updates,an event-triggered control law is developed for ADP algorithm.After that,to further simplify the complexity of controller design,this work is extended to a self-triggered case for relaxing the need for continuous signal monitoring by hardware devices.By employing the Lyapunov method,the uniform ultimate boundedness of all signals is proved to be guaranteed.Finally,a simulation example on a mass–spring–damper system subject to asymmetric input constraints is provided to validate the effectiveness of the proposed control scheme.

Dynamics and adaptive control of a dual-arm space robot with closed-loop constraints and uncertain inertial parameters

A dynamics-based adaptive control approach is proposed for a planar dual-arm space robot in the presence of closed-loop constraints and uncertain inertial parameters of the payload. The controller is capable of controlling the po- sition and attitude of both the satellite base and the payload grasped by the manipulator end effectors. The equations of motion in reduced-order form for the constrained system are derived by incorporating the constraint equations in terms of accelerations into Kane＇s equations of the unconstrained system. Model analysis shows that the resulting equations perfectly meet the requirement of adaptive controller design. Consequently, by using an indirect approach, an adaptive control scheme is proposed to accomplish position/attitude trajectory tracking control with the uncertain parameters be- ing estimated on-line. The actuator redundancy due to the closed-loop constraints is utilized to minimize a weighted norm of the joint torques. Global asymptotic stability is proven by using Lyapunov＇s method, and simulation results are also presented to demonstrate the effectiveness of the proposed approach.

Particle swarm optimization-based algorithm of a symplectic method for robotic dynamics and control

Multibody system dynamics provides a strong tool for the estimation of dynamic performances and the optimization of multisystem robot design. It can be described with differential algebraic equations(DAEs). In this paper, a particle swarm optimization(PSO) method is introduced to solve and control a symplectic multibody system for the first time. It is first combined with the symplectic method to solve problems in uncontrolled and controlled robotic arm systems. It is shown that the results conserve the energy and keep the constraints of the chaotic motion, which demonstrates the efficiency, accuracy, and time-saving ability of the method. To make the system move along the pre-planned path, which is a functional extremum problem, a double-PSO-based instantaneous optimal control is introduced. Examples are performed to test the effectiveness of the double-PSO-based instantaneous optimal control. The results show that the method has high accuracy, a fast convergence speed, and a wide range of applications.All the above verify the immense potential applications of the PSO method in multibody system dynamics.

Optimal Constrained Self-learning Battery Sequential Management in Microgrid Via Adaptive Dynamic Programming

This paper concerns a novel optimal self-learning battery sequential control scheme for smart home energy systems.The main idea is to use the adaptive dynamic programming(ADP) technique to obtain the optimal battery sequential control iteratively. First, the battery energy management system model is established, where the power efficiency of the battery is considered. Next, considering the power constraints of the battery, a new non-quadratic form performance index function is established, which guarantees that the value of the iterative control law cannot exceed the maximum charging/discharging power of the battery to extend the service life of the battery.Then, the convergence properties of the iterative ADP algorithm are analyzed, which guarantees that the iterative value function and the iterative control law both reach the optimums. Finally,simulation and comparison results are given to illustrate the performance of the presented method.

Harmony search algorithm with differential evolution based control parameter co-evolution and its application in chemical process dynamic optimization

A modified harmony search algorithm with co-evolutional control parameters(DEHS), applied through differential evolution optimization, is proposed. In DEHS, two control parameters, i.e., harmony memory considering rate and pitch adjusting rate, are encoded as a symbiotic individual of an original individual(i.e., harmony vector). Harmony search operators are applied to evolving the original population. DE is applied to co-evolving the symbiotic population based on feedback information from the original population. Thus, with the evolution of the original population in DEHS, the symbiotic population is dynamically and self-adaptively adjusted, and real-time optimum control parameters are obtained. The proposed DEHS algorithm has been applied to various benchmark functions and two typical dynamic optimization problems. The experimental results show that the performance of the proposed algorithm is better than that of other HS variants. Satisfactory results are obtained in the application.

Dynamic Control and Optimization of Capital-Constrained Stochastic Inventory Systems

For most firms,especially the small-and medium-sized ones,the operational decisions are affected by their internal capital and ability to obtain external capital.However,the majority of the current studies on dynamic inventory control ignore the firm’s financial status and financing issues completely.An important question that arises is:what are the dynamic optimal inventory and financing policies for firms with limited capital and limited access to external capital?In this paper,we review some of the latest developments in this area.After a brief review of single period models,we focus on multi-period dynamic control of the firm who aims to optimize its xpected terminal wealth.Two cases are discussed in detail:self-finance and short term finance.In the first case,the firm has to rely on its own capital for all ordering decisions,while in the second,the firm can borrow short term loan from lenders.A detailed characterization of the optimal policy is presented and its managerial insights are discussed.Several possible extensions are suggested.