Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Hybrid Dynamic Variables-Dependent Event-Triggered Fuzzy Model Predictive Control

This article focuses on dynamic event-triggered mechanism(DETM)-based model predictive control(MPC) for T-S fuzzy systems.A hybrid dynamic variables-dependent DETM is carefully devised,which includes a multiplicative dynamic variable and an additive dynamic variable.The addressed DETM-based fuzzy MPC issue is described as a “min-max” optimization problem(OP).To facilitate the co-design of the MPC controller and the weighting matrix of the DETM,an auxiliary OP is proposed based on a new Lyapunov function and a new robust positive invariant(RPI) set that contain the membership functions and the hybrid dynamic variables.A dynamic event-triggered fuzzy MPC algorithm is developed accordingly,whose recursive feasibility is analysed by employing the RPI set.With the designed controller,the involved fuzzy system is ensured to be asymptotically stable.Two examples show that the new DETM and DETM-based MPC algorithm have the advantages of reducing resource consumption while yielding the anticipated performance.

Controlling the dynamic behavior of decentralized cluster through centralized approaches

How to control the dynamic behavior of large-scale artificial active matter is a critical concern in experimental research on soft matter, particularly regarding the emergence of collective behaviors and the formation of group patterns. Centralized systems excel in precise control over individual behavior within a group, ensuring high accuracy and controllability in task execution. Nevertheless, their sensitivity to group size may limit their adaptability to diverse tasks. In contrast, decentralized systems empower individuals with autonomous decision-making, enhancing adaptability and system robustness. Yet, this flexibility comes at the cost of reduced accuracy and efficiency in task execution. In this work, we present a unique method for regulating the centralized dynamic behavior of self-organizing clusters based on environmental interactions. Within this environment-coupled robot system, each robot possesses similar dynamic characteristics, and their internal programs are entirely identical. However, their behaviors can be guided by the centralized control of the environment, facilitating the accomplishment of diverse cluster tasks. This approach aims to balance the accuracy and flexibility of centralized control with the robustness and task adaptability of decentralized control. The proactive regulation of dynamic behavioral characteristics in active matter groups, demonstrated in this work through environmental interactions, holds the potential to introduce a novel technological approach and provide experimental references for studying the dynamic behavior control of large-scale artificial active matter systems.

Dynamic Constraint-Driven Event-Triggered Control of Strict-Feedback Systems Without Max/Min Values on Irregular Constraints

This work proposes an event-triggered adaptive control approach for a class of uncertain nonlinear systems under irregular constraints.Unlike the constraints considered in most existing papers,here the external irregular constraints are considered and a constraints switching mechanism(CSM)is introduced to circumvent the difficulties arising from irregular output constraints.Based on the CSM,a new class of generalized barrier functions are constructed,which allows the control results to be independent of the maximum and minimum values(MMVs)of constraints and thus extends the existing results.Finally,we proposed a novel dynamic constraint-driven event-triggered strategy(DCDETS),under which the stress on signal transmission is reduced greatly and no constraints are violated by making a dynamic trade-off among system state,external constraints,and inter-execution intervals.It is proved that the system output is driven to close to the reference trajectory and the semi-global stability is guaranteed under the proposed control scheme,regardless of the external irregular output constraints.Simulation also verifies the effectiveness and benefits of the proposed method.

The Effect of Key Nodes on theMalware Dynamics in the Industrial Control Network

As industrialization and informatization becomemore deeply intertwined,industrial control networks have entered an era of intelligence.The connection between industrial control networks and the external internet is becoming increasingly close,which leads to frequent security accidents.This paper proposes a model for the industrial control network.It includes a malware containment strategy that integrates intrusion detection,quarantine,and monitoring.Basedonthismodel,the role of keynodes in the spreadofmalware is studied,a comparisonexperiment is conducted to validate the impact of the containment strategy.In addition,the dynamic behavior of the model is analyzed,the basic reproduction number is computed,and the disease-free and endemic equilibrium of the model is also obtained by the basic reproduction number.Moreover,through simulation experiments,the effectiveness of the containment strategy is validated,the influence of the relevant parameters is analyzed,and the containment strategy is optimized.In otherwords,selective immunity to key nodes can effectively suppress the spread ofmalware andmaintain the stability of industrial control systems.The earlier the immunization of key nodes,the better.Once the time exceeds the threshold,immunizing key nodes is almost ineffective.The analysis provides a better way to contain the malware in the industrial control network.

Adaptable and Dynamic Access Control Decision-Enforcement Approach Based on Multilayer Hybrid Deep Learning Techniques in BYOD Environment

Organizations are adopting the Bring Your Own Device(BYOD)concept to enhance productivity and reduce expenses.However,this trend introduces security challenges,such as unauthorized access.Traditional access control systems,such as Attribute-Based Access Control(ABAC)and Role-Based Access Control(RBAC),are limited in their ability to enforce access decisions due to the variability and dynamism of attributes related to users and resources.This paper proposes a method for enforcing access decisions that is adaptable and dynamic,based on multilayer hybrid deep learning techniques,particularly the Tabular Deep Neural Network Tabular DNN method.This technique transforms all input attributes in an access request into a binary classification(allow or deny)using multiple layers,ensuring accurate and efficient access decision-making.The proposed solution was evaluated using the Kaggle Amazon access control policy dataset and demonstrated its effectiveness by achieving a 94%accuracy rate.Additionally,the proposed solution enhances the implementation of access decisions based on a variety of resource and user attributes while ensuring privacy through indirect communication with the Policy Administration Point(PAP).This solution significantly improves the flexibility of access control systems,making themmore dynamic and adaptable to the evolving needs ofmodern organizations.Furthermore,it offers a scalable approach to manage the complexities associated with the BYOD environment,providing a robust framework for secure and efficient access management.

Mechanism,prevention,and control of mining-induced dynamic disasters in underground metal mines in China:Challenges and solutions

Metal mineral resources play an indispensable role in the development of the national economy.Dynamic disasters in underground metal mines seriously threaten mining safety,which are major scientific and technological problems to be solved urgently.In this article,the occurrence status and grand challenges of some typical dynamic disasters involving roof falling,spalling,collapse,large deformation,rockburst,surface subsidence,and water inrush in metal mines in China are systematically presented,the characteristics of mining-induced dynamic disasters are analyzed,the examples of dynamic disasters occurring in some metal mines in China are summarized,the occurrence mechanism,monitoring and early warning methods,and prevention and control techniques of these disasters are highlighted,and some new opinions,suggestions,and solutions are proposed simultaneously.Moreover,some shortcomings in current disaster research are pointed out,and the direction of efforts to improve the prevention and control level of dynamic disasters in China’s metal mines in the future is prospected.The integration of forward-looking key innovative theories and technologies in the abovementioned aspects will greatly enhance the cognitive level of disaster prevention and mitigation in China’s metal mining industry and achieve a significant shift from passive disaster relief to active disaster prevention.

Dynamics modeling and optimal control for multi-information diffusion in Social Internet of Things

As an ingenious convergence between the Internet of Things and social networks,the Social Internet of Things(SIoT)can provide effective and intelligent information services and has become one of the main platforms for people to spread and share information.Nevertheless,SIoT is characterized by high openness and autonomy,multiple kinds of information can spread rapidly,freely and cooperatively in SIoT,which makes it challenging to accurately reveal the characteristics of the information diffusion process and effectively control its diffusion.To this end,with the aim of exploring multi-information cooperative diffusion processes in SIoT,we first develop a dynamics model for multi-information cooperative diffusion based on the system dynamics theory in this paper.Subsequently,the characteristics and laws of the dynamical evolution process of multi-information cooperative diffusion are theoretically investigated,and the diffusion trend is predicted.On this basis,to further control the multi-information cooperative diffusion process efficiently,we propose two control strategies for information diffusion with control objectives,develop an optimal control system for the multi-information cooperative diffusion process,and propose the corresponding optimal control method.The optimal solution distribution of the control strategy satisfying the control system constraints and the control budget constraints is solved using the optimal control theory.Finally,extensive simulation experiments based on real dataset from Twitter validate the correctness and effectiveness of the proposed model,strategy and method.

Hierarchical Controller Synthesis Under Linear Temporal Logic Specifications Using Dynamic Quantization

Linear temporal logic(LTL)is an intuitive and expressive language to specify complex control tasks,and how to design an efficient control strategy for LTL specification is still a challenge.In this paper,we implement the dynamic quantization technique to propose a novel hierarchical control strategy for nonlinear control systems under LTL specifications.Based on the regions of interest involved in the LTL formula,an accepting path is derived first to provide a high-level solution for the controller synthesis problem.Second,we develop a dynamic quantization based approach to verify the realization of the accepting path.The realization verification results in the necessity of the controller design and a sequence of quantization regions for the controller design.Third,the techniques of dynamic quantization and abstraction-based control are combined together to establish the local-to-global control strategy.Both abstraction construction and controller design are local and dynamic,thereby resulting in the potential reduction of the computational complexity.Since each quantization region can be considered locally and individually,the proposed hierarchical mechanism is more efficient and can solve much larger problems than many existing methods.Finally,the proposed control strategy is illustrated via two examples from the path planning and tracking problems of mobile robots.

Dynamic Event-Triggered Consensus Control for Input Constrained Multi-Agent Systems With a Designable Minimum Inter-Event Time

This paper investigates the consensus control of multi-agent systems(MASs) with constrained input using the dynamic event-triggered mechanism(ETM).Consider the MASs with small-scale networks where a centralized dynamic ETM with global information of the MASs is first designed.Then,a distributed dynamic ETM which only uses local information is developed for the MASs with large-scale networks.It is shown that the semi-global consensus of the MASs can be achieved by the designed bounded control protocol where the Zeno phenomenon is eliminated by a designable minimum inter-event time.In addition,it is easier to find a trade-off between the convergence rate and the minimum inter-event time by an adjustable parameter.Furthermore,the results are extended to regional consensus of the MASs with the bounded control protocol.Numerical simulations show the effectiveness of the proposed approach.

Transmission Dynamics and Optimal Control Strategies of a Hand-Foot-Mouth Disease Model with Treatment and Vaccination Interventions

In this article, the transmission dynamics of a Hand-Foot-Mouth disease model with treatment and vaccination interventions are studied. We calculated the basic reproduction number and proved the global stability of disease-free equilibrium when R0 R0 > 1. Meanwhile, we obtained the optimal control strategies minimizing the cost of intervention and minimizing the infected person. We also give some numerical simulations to verify our theoretical results.

Dynamic simulation analysis of molten salt reactor-coupled air-steam combined cycle power generation system

A nonlinear dynamic simulation model based on coordinated control of speed and flow rate for the molten salt reactor and combined cycle systems is proposed here to ensure the coordination and stability between the molten salt reactor and power system.This model considers the impact of thermal properties of fluid variation on accuracy and has been validated with Simulink.This study reveals the capability of the control system to compensate for anomalous situations and maintain shaft stability in the event of perturbations occurring in high-temperature molten salt tank outlet parameters.Meanwhile,the control system’s impact on the system’s dynamic characteristics under molten salt disturbance is also analyzed.The results reveal that after the disturbance occurs,the controlled system benefits from the action of the control,and the overshoot and disturbance amplitude are positively correlated,while the system power and frequency eventually return to the initial values.This simulation model provides a basis for utilizing molten salt reactors for power generation and maintaining grid stability.

An active high-static-low-dynamic-stiffness vibration isolator with adjustable buckling beams:theory and experiment

High-static-low-dynamic-stiffness(HSLDS)vibration isolators with buckling beams have been widely used to isolate external vibrations.An active adjustable device composed of proportion integration(PI)active controllers and piezoelectric actuators is proposed for improving the negative stiffness stroke of buckling beams.A nonlinear output frequency response function is used to analyze the effect of the vibration reduction.The prototype of the active HSLDS device is built,and the verification experiment is conducted.The results show that compared with the traditional HSLDS vibration isolator,the active HSLDS device can broaden the isolation frequency bandwidth,and effectively reduce the resonant amplitude by adjusting the active control parameters.The maximum vibration reduction rate of the active HSLDS vibration isolator can attain 89.9%,and the resonant frequency can be reduced from 31.08 Hz to 13.28 Hz.Therefore,this paper devotes to providing a new design scheme for enhanced HSLDS vibration isolators.

Study on vibration reduction of two-scale system coupled with dynamic vibration absorber

The dynamic vibration absorber with inerter and grounded stiffness(IGDVA)is used to control a two-scale system subject to a weak periodic perturbation.The vibration suppression effect is remarkable.The amplitude of the main system coupled with absorber is significantly reduced,and the high frequency vibration completely disappears.First,through the slow-fast analysis and stability theory,it is found that the stability of the autonomous system exerts a notable regulating effect on the vibration response of the non-autonomous system.After adding the dynamic vibrator absorber,the center in the autonomous system changes to an asymptotically stable focus,consequently suppressing the vibration in the non-autonomous system.Further research reveals that the parameters of the absorber affect the real parts of the eigenvalues of the autonomous system,thereby regulating the stability of the system.Transitioning from a qualitative standpoint to a quantitative approach,a comparison of the solutions before and after the introduction of the dynamic absorber reveals that,when the grounded stiffness ratio and the mass ratio of the dynamic absorber are not equal,the high-frequency part in the analytical solution disappears.As a result,this leads to a reduction in the amplitude of the trajectory,achieving a vibration reduction effect.

Vibration Reduction by a Partitioned Dynamic Vibration Absorber with Acoustic Black Hole Features

Vibration quality is a vital indicator for assessing the progress of modern equipment.The dynamic vibration absorber(DVA)based on the acoustic black hole(ABH)feature is a new passive control method that manipulates waves.It offers efficient energy focalization and broad-spectrum vibration suppression,making it highly promising for applications in large equipment such as aircraft,trains,and ships.Despite previous advancements in ABH-DVA development,certain challenges remain,particularly in ensuring effective coupling with host structures during control.To address these issues,this study proposes a partitioned ABH-featured dynamic vibration absorber(PABH-DVA)with partitions in the radial direction of the disc.By employing a plate as the host structure,simulations and experiments were conducted,demonstrating that the PABH-DVA outperforms the original symmetric ABH-DVA in terms of damping performance.The study also calculated and compared the coupling coefficients of the two ABH-DVAs to uncover the mechanism behind the enhanced damping.Simulation results revealed that the PABH-DVA exhibits more coupled modes,occasionally with lower coupling coefficients than the symmetric ABH-DVA.The influence of frequency ratio and modal mass was further analyzed to explain the reasons behind the PABH-DVA's superior damping performance.Additionally,the study discussed the impact of the number of slits and their orientation.This research further explains the coupling mechanism between the ABH-DVA and the controlled structure,and provides new ideas for the further application of ABH in engineering.

Dynamic event-triggered formation control of second-order nonholonomic systems

In this paper,the formation control problem of secondorder nonholonomic mobile robot systems is investigated in a dynamic event-triggered scheme.Event-triggered control protocols combined with persistent excitation(PE)conditions are presented.In event-detecting processes,an inactive time is introduced after each sampling instant,which can ensure a positive minimum sampling interval.To increase the flexibility of the event-triggered scheme,internal dynamic variables are included in event-triggering conditions.Moreover,the dynamic event-triggered scheme plays an important role in increasing the lengths of time intervals between any two consecutive events.In addition,event-triggered control protocols without forward and angular velocities are also presented based on approximate-differentiation(low-pass)filters.The asymptotic convergence results are given based on a nested Matrosov theorem and artificial sampling methods.

Dynamic Event-Triggered Fixed-Time Consensus Control and Its Applications to Magnetic Map Construction

This article deals with the consensus problem of multi-agent systems by developing a fixed-time consensus control approach with a dynamic event-triggered rule. First, a new fixedtime stability condition is obtained where the less conservative settling time is given such that the theoretical settling time can well reflect the real consensus time. Second, a dynamic event-triggered rule is designed to decrease the use of chip and network resources where Zeno behaviors can be avoided after consensus is achieved, especially for finite/fixed-time consensus control approaches. Third, in terms of the developed dynamic event-triggered rule, a fixed-time consensus control approach by introducing a new item is proposed to coordinate the multi-agent system to reach consensus. The corresponding stability of the multi-agent system with the proposed control approach and dynamic eventtriggered rule is analyzed based on Lyapunov theory and the fixed-time stability theorem. At last, the effectiveness of the dynamic event-triggered fixed-time consensus control approach is verified by simulations and experiments for the problem of magnetic map construction based on multiple mobile robots.

Two-layer formation-containment fault-tolerant control of fixed-wing UAV swarm for dynamic target tracking

This paper tackles the formation-containment control problem of fixed-wing unmanned aerial vehicle(UAV)swarm with model uncertainties for dynamic target tracking in three-dimensional space in the faulty case of UAVs’actuator and sensor.The fixed-wing UAV swarm under consideration is organized as a“multi-leader-multi-follower”structure,in which only several leaders can obtain the dynamic target information while others only receive the neighbors’information through the communication network.To simultaneously realize the formation,containment,and dynamic target tracking,a two-layer control framework is adopted to decouple the problem into two subproblems:reference trajectory generation and trajectory tracking.In the upper layer,a distributed finite-time estimator(DFTE)is proposed to generate each UAV’s reference trajectory in accordance with the control objective.Subsequently,a distributed composite robust fault-tolerant trajectory tracking controller is developed in the lower layer,where a novel adaptive extended super-twisting(AESTW)algorithm with a finite-time extended state observer(FTESO)is involved in solving the robust trajectory tracking control problem under model uncertainties,actuator,and sensor faults.The proposed controller simultaneously guarantees rapidness and enhances the system’s robustness with fewer chattering effects.Finally,corresponding simulations are carried out to demonstrate the effectiveness and competitiveness of the proposed two-layer fault-tolerant cooperative control scheme.

Robust missile autopilot design based on dynamic surface control

Since the dynamical system and control system of the missile are typically nonlinear, an effective acceleration tracking autopilot is designed using the dynamic surface control(DSC)technique in order to make the missile control system more robust despite the uncertainty of the dynamical parameters and the presence of disturbances. Firstly, the nonlinear mathematical model of the tail-controlled missile is decomposed into slow acceleration dynamics and fast pitch rate dynamics based on the naturally existing time scale separation. Secondly, the controller based on DSC is designed after obtaining the linear dynamics characteristics of the slow and fast subsystems. An extended state observer is used to detect the uncertainty of the system state variables and aerodynamic parameters to achieve the compensation of the control law. The closed-loop stability of the controller is derived and rigorously analyzed. Finally, the effectiveness and robustness of the design is verified by Monte Carlo simulation considering different initial conditions and parameter uptake. Simulation results illustrate that the missile autopilot based DSC controller achieves better performance and robustness than the other two well-known autopilots.The method proposed in this paper is applied to the design of a missile autopilot, and the results show that the acceleration tracking autopilot based on the DSC controller can ensure accurate tracking of the required commands and has better performance.

Fixed-Time Stabilization of a Class of Strict-Feedback Nonlinear Systems via Dynamic Gain Feedback Control

This paper presents a novel fixed-time stabilization control(FSC)method for a class of strict-feedback nonlinear systems involving unmodelled system dynamics.The key feature of the proposed method is the design of two dynamic parameters.Specifically,a set of auxiliary variables is first introduced through state transformation.These variables combine the original system states and the two introduced dynamic parameters,facilitating the closed-loop system stability analyses.Then,the two dynamic parameters are delicately designed by utilizing the Lyapunov method,ensuring that all the closed-loop system states are globally fixed-time stable.Compared with existing results,the“explosion of complexity”problem of backstepping control is avoided.Moreover,the two designed dynamic parameters are dependent on system states rather than a time-varying function,thus the proposed controller is still valid beyond the given fixedtime convergence instant.The effectiveness of the proposed method is demonstrated through two practical systems.