Policy Iteration for Optimal Control of Discrete-Time Time-Varying Nonlinear Systems

Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iteration(DTTV)algorithm,is developed.The iterative control law is designed to update the iterative value function which approximates the index function of optimal performance.The admissibility of the iterative control law is analyzed.The results show that the iterative value function is non-increasingly convergent to the Bellman-equation optimal solution.To implement the algorithm,neural networks are employed and a new implementation structure is established,which avoids solving the generalized Bellman equation in each iteration.Finally,the optimal control laws for torsional pendulum and inverted pendulum systems are obtained by using the DTTV policy iteration algorithm,where the mass and pendulum bar length are permitted to be time-varying parameters.The effectiveness of the developed method is illustrated by numerical results and comparisons.

Robust stability and stabilization for uncertain discrete-time switched singular systems with time-varying delays

The problems of robust stability and stabilization via memoryless state feedback for a class of discrete-time switched singular systems with time-varying delays and linear fractional uncertainties are investigated.By constructing a novel switched Lyapunov-Krasovskii functional,a delay-dependent criterion for the unforced system to be regular,causal and uniformly asymptotically stable is established in terms of linear matrix inequalities（LMIs）.An explicit expression for the desired memoryless state feedback stabilization controller is also given.The merits of the proposed criteria lie in their less conservativeness and relative simplicity,which are achieved by considering additionally useful terms（ignored in previous methods） when estimating the upper bound of the forward difference of the Lyapunov-Krasovskii functional and by avoiding utilizing any model augmentation transformation.Some numerical examples are provided to illustrate the validity of the proposed methods.

Improved Results on Robust H_∞ Control of Uncertain Discrete-time Systems with Time-varying Delay

In this paper, the robust H∞ control problem for uncertain discrete-time systems with time-varying state delay is con- sidered. Based on the Lyapunov functional method, and by resorting to the new technique for estimating the upper bound of the difference of the Lyapunov functional, a new less conservative sufficient condition for the existence of a robust H∞ controller is obtained. Moreover, the cone complementary linearisation procedure is employed to solve the nonconvex feasibility problem. Finally, several numerical examples are presented to show the effectiveness and less conservativeness of the proposed method.

Delay-dependent robust H-infinity control for discrete-time uncertain systems with time-varying state delays

This paper examines the delay-dependent H-infinity control problem for discrete-time linear systems with time-varying state delays and norm-bounded uncertainties. A new inequality for the finite sum of quadratic terms is first established. Then, some new delay-dependent criteria are derived by employing the new inequality to guarantee the robust stability of a closed-loop system with a prescribed H-infinity norm bound for all admissible uncertainties and bounded time-vary delays. A numerical example demonstrates that the proposed method is an improvement over existing ones.

Exponential stability for networked control systems based on the model of nonlinear discrete-time system with time-varying delay

An uncertain nonlinear discrete-time system model with time-varying input delays for networked control systems (NCSs) is presented. The problem of exponential stability for the system is considered and some new criteria of exponential stability are obtained based on norm inequality methods. A numerical example is given todemonstrate that those criteria are useful to analyzing the stability of nonlinear NCSs.

Deterministic and Stochastic Fixed-Time Stability of Discrete-time Autonomous Systems

This paper studies deterministic and stochastic fixedtime stability of autonomous nonlinear discrete-time(DT)systems.Lyapunov conditions are first presented under which the fixed-time stability of deterministic DT systems is certified.Extensions to systems under deterministic perturbations as well as stochastic noise are then considered.For the former,sensitivity to perturbations for fixed-time stable DT systems is analyzed,and it is shown that fixed-time attractiveness results from the presented Lyapunov conditions.For the latter,sufficient Lyapunov conditions for fixed-time stability in probability of nonlinear stochastic DT systems are presented.The fixed upper bound of the settling-time function is derived for both fixed-time stable and fixed-time attractive systems,and a stochastic settling-time function fixed upper bound is derived for stochastic DT systems.Illustrative examples are given along with simulation results to verify the introduced results.

Adaptive dynamic programming for finite-horizon optimal control of linear time-varying discrete-time systems

This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discretetime systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-varying discrete-time systems is presented. Its connections with existing in finite-horizon PI methods are discussed. Then, both data-drive n off-policy PI and Value Iteration (VI) algorithms are derived to find approximate optimal controllers when the system dynamics is completely unknown. Under mild conditions, the proposed data-driven off-policy algorithms converge to the optimal solution. Finally, the effectiveness and feasibility of the developed methods are validated by a practical example of spacecraft attitude control.

Event-Triggered Finite-Time H∞ Filtering for Discrete-Time Nonlinear Stochastic Systems

This paper addresses the problem of event-triggered finite-time H∞ filter design for a class of discrete-time nonlinear stochastic systems with exogenous disturbances. The stochastic Lyapunov-Krasoviskii functional method is adopted to design a filter such that the filtering error system is stochastic finite-time stable (SFTS) and preserves a prescribed performance level according to the pre-defined event-triggered criteria. Based on stochastic differential equations theory, some sufficient conditions for the existence of H∞ filter are obtained for the suggested system by employing linear matrix inequality technique. Finally, the desired H∞ filter gain matrices can be expressed in an explicit form.

Complementary-Label Adversarial Domain Adaptation Fault Diagnosis Network under Time-Varying Rotational Speed and Weakly-Supervised Conditions

Recent research in cross-domain intelligence fault diagnosis of machinery still has some problems,such as relatively ideal speed conditions and sample conditions.In engineering practice,the rotational speed of the machine is often transient and time-varying,which makes the sample annotation increasingly expensive.Meanwhile,the number of samples collected from different health states is often unbalanced.To deal with the above challenges,a complementary-label(CL)adversarial domain adaptation fault diagnosis network(CLADAN)is proposed under time-varying rotational speed and weakly-supervised conditions.In the weakly supervised learning condition,machine prior information is used for sample annotation via cost-friendly complementary label learning.A diagnosticmodel learning strategywith discretized category probabilities is designed to avoidmulti-peak distribution of prediction results.In adversarial training process,we developed virtual adversarial regularization(VAR)strategy,which further enhances the robustness of the model by adding adversarial perturbations in the target domain.Comparative experiments on two case studies validated the superior performance of the proposed method.

On the Application of Mixed Models of Probability and Convex Set for Time-Variant Reliability Analysis

In time-variant reliability problems,there are a lot of uncertain variables from different sources.Therefore,it is important to consider these uncertainties in engineering.In addition,time-variant reliability problems typically involve a complexmultilevel nested optimization problem,which can result in an enormous amount of computation.To this end,this paper studies the time-variant reliability evaluation of structures with stochastic and bounded uncertainties using a mixed probability and convex set model.In this method,the stochastic process of a limit-state function with mixed uncertain parameters is first discretized and then converted into a timeindependent reliability problem.Further,to solve the double nested optimization problem in hybrid reliability calculation,an efficient iterative scheme is designed in standard uncertainty space to determine the most probable point(MPP).The limit state function is linearized at these points,and an innovative random variable is defined to solve the equivalent static reliability analysis model.The effectiveness of the proposed method is verified by two benchmark numerical examples and a practical engineering problem.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

Nuclear magnetic resonance experiments on the time-varying law of oil viscosity and wettability in high-multiple waterflooding sandstone cores

A simulated oil viscosity prediction model is established according to the relationship between simulated oil viscosity and geometric mean value of T2spectrum,and the time-varying law of simulated oil viscosity in porous media is quantitatively characterized by nuclear magnetic resonance(NMR)experiments of high multiple waterflooding.A new NMR wettability index formula is derived based on NMR relaxation theory to quantitatively characterize the time-varying law of rock wettability during waterflooding combined with high-multiple waterflooding experiment in sandstone cores.The remaining oil viscosity in the core is positively correlated with the displacing water multiple.The remaining oil viscosity increases rapidly when the displacing water multiple is low,and increases slowly when the displacing water multiple is high.The variation of remaining oil viscosity is related to the reservoir heterogeneity.The stronger the reservoir homogeneity,the higher the content of heavy components in the remaining oil and the higher the viscosity.The reservoir wettability changes after water injection:the oil-wet reservoir changes into water-wet reservoir,while the water-wet reservoir becomes more hydrophilic;the degree of change enhances with the increase of displacing water multiple.There is a high correlation between the time-varying oil viscosity and the time-varying wettability,and the change of oil viscosity cannot be ignored.The NMR wettability index calculated by considering the change of oil viscosity is more consistent with the tested Amott(spontaneous imbibition)wettability index,which agrees more with the time-varying law of reservoir wettability.

Numerical Simulation of Slurry Diffusion in Fractured Rocks Considering a Time-Varying Viscosity

To analyze the effects of a time-varying viscosity on the penetration length of grouting,in this study cement slur-ries with varying water-cement ratios have been investigated using the Bingham’sfluidflow equation and a dis-crete element method.Afluid-solid coupling numerical model has been introduced accordingly,and its accuracy has been validated through comparison of theoretical and numerical solutions.For different fracture forms(a single fracture,a branch fracture,and a fracture network),the influence of the time-varying viscosity on the slurry length range has been investigated,considering the change in the fracture aperture.The results show that under different fracture forms and the same grouting process conditions,the influence of the time-varying viscosity on the seepage length is 0.350 m.

Adaptive Event-Triggered Time-Varying Output Group Formation Containment Control of Heterogeneous Multiagent Systems

In this paper,a class of time-varying output group formation containment control problem of general linear hetero-geneous multiagent systems(MASs)is investigated under directed topology.The MAS is composed of a number of tracking leaders,formation leaders and followers,where two different types of leaders are used to provide reference trajectories for movement and to achieve certain formations,respectively.Firstly,compen-sators are designed whose states are estimations of tracking lead-ers,based on which,a controller is developed for each formation leader to accomplish the expected formation.Secondly,two event-triggered compensators are proposed for each follower to evalu-ate the state and formation information of the formation leaders in the same group,respectively.Subsequently,a control protocol is designed for each follower,utilizing the output information,to guide the output towards the convex hull generated by the forma-tion leaders within the group.Next,the triggering sequence in this paper is decomposed into two sequences,and the inter-event intervals of these two triggering conditions are provided to rule out the Zeno behavior.Finally,a numerical simulation is intro-duced to confirm the validity of the proposed results.

Set-Membership Filtering Approach to Dynamic Event-Triggered Fault Estimation for a Class of Nonlinear Time-Varying Complex Networks

The present study addresses the problem of fault estimation for a specific class of nonlinear time-varying complex networks,utilizing an unknown-input-observer approach within the framework of dynamic event-triggered mechanism(DETM).In order to optimize communication resource utilization,the DETM is employed to determine whether the current measurement data should be transmitted to the estimator or not.To guarantee a satisfactory estimation performance for the fault signal,an unknown-input-observer-based estimator is constructed to decouple the estimation error dynamics from the influence of fault signals.The aim of this paper is to find the suitable estimator parameters under the effects of DETM such that both the state estimates and fault estimates are confined within two sets of closed ellipsoid domains.The techniques of recursive matrix inequality are applied to derive sufficient conditions for the existence of the desired estimator,ensuring that the specified performance requirements are met under certain conditions.Then,the estimator gains are derived by minimizing the ellipsoid domain in the sense of trace and a recursive estimator parameter design algorithm is then provided.Finally,a numerical example is conducted to demonstrate the effectiveness of the designed estimator.

Identification of time-varying system and energy-based optimization of adaptive control in seismically excited structure

The combination of structural health monitoring and vibration control is of great importance to provide components of smart structures.While synthetic algorithms have been proposed,adaptive control that is compatible with changing conditions still needs to be used,and time-varying systems are required to be simultaneously estimated with the application of adaptive control.In this research,the identification of structural time-varying dynamic characteristics and optimized simple adaptive control are integrated.First,reduced variations of physical parameters are estimated online using the multiple forgetting factor recursive least squares(MFRLS)method.Then,the energy from the structural vibration is simultaneously specified to optimize the control force with the identified parameters to be operational.Optimization is also performed based on the probability density function of the energy under the seismic excitation at any time.Finally,the optimal control force is obtained by the simple adaptive control(SAC)algorithm and energy coefficient.A numerical example and benchmark structure are employed to investigate the efficiency of the proposed approach.The simulation results revealed the effectiveness of the integrated online identification and optimal adaptive control in systems.

Time-varying parameters estimation with adaptive neural network EKF for missile-dual control system

In this paper, a filtering method is presented to estimate time-varying parameters of a missile dual control system with tail fins and reaction jets as control variables. In this method, the long-short-term memory(LSTM) neural network is nested into the extended Kalman filter(EKF) to modify the Kalman gain such that the filtering performance is improved in the presence of large model uncertainties. To avoid the unstable network output caused by the abrupt changes of system states,an adaptive correction factor is introduced to correct the network output online. In the process of training the network, a multi-gradient descent learning mode is proposed to better fit the internal state of the system, and a rolling training is used to implement an online prediction logic. Based on the Lyapunov second method, we discuss the stability of the system, the result shows that when the training error of neural network is sufficiently small, the system is asymptotically stable. With its application to the estimation of time-varying parameters of a missile dual control system, the LSTM-EKF shows better filtering performance than the EKF and adaptive EKF(AEKF) when there exist large uncertainties in the system model.

Improved results on passivity analysis of discrete-time stochastic neural networks with time-varying delay

The problem of passivity analysis for a class of discrete-time stochastic neural networks (DSNNs) with time-varying interval delay was investigated. The delay-dependent sufficient criteria were derived in terms of linear matrix inequalities (LMIs). The results are shown to be generalization of some previous results and are less conservative than the existing works. Meanwhile, the computational complexity of the obtained stability conditions is reduced because less variables are involved. A numerical example is given to show the effectiveness and the benefits of the proposed method.

Synchronization of High-order Discrete-time Linear Complex Networks with Time-varying Delays

Synchronization of high-order discrete-time complex networks with undirected topologies is studied and the impacts of time delays are investigated. Firstly,by the state decomposition,synchronization problems are transformed into asymptotic stability ones of multiple lower dimensional time-delayed subsystems. Then,linear matrix inequality( LMI) criteria for synchronization are given,which can guarantee the scalability of complex networks since they only include three LMI constraints independent of the number of agents. Moreover,an explicit expression of the synchronization function is presented,which can describe the synchronization behavior of all agents in complex networks. Finally,a numerical example is given to demonstrate the theoretical results,where it is shown that if the gain matrices of synchronization protocols satisfy LMI criteria for synchronization,synchronization can be achieved.

Time-Varying Mesh Stiffness Calculation and Dynamic Modeling of Spiral Bevel Gear with Spalling Defects

Time-varying mesh stiffness(TVMS)is a vital internal excitation source for the spiral bevel gear(SBG)transmission system.Spalling defect often causes decrease in gear mesh stiffness and changes the dynamic characteristics of the gear system,which further increases noise and vibration.This paper aims to calculate the TVMS and establish dynamic model of SBG with spalling defect.In this study,a novel analytical model based on slice method is proposed to calculate the TVMS of SBG considering spalling defect.Subsequently,the influence of spalling defect on the TVMS is studied through a numerical simulation,and the proposed analytical model is verified by a finite element model.Besides,an 8-degrees-of-freedom dynamic model is established for SBG transmission system.Incorporating the spalling defect into TVMS,the dynamic responses of spalled SBG are analyzed.The numerical results indicate that spalling defect would cause periodic impact in time domain.Finally,an experiment is designed to verify the proposed dynamic model.The experimental results show that the spalling defect makes the response characterized by periodic impact with the rotating frequency of spalled pinion.