Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Hybrid Dynamic Variables-Dependent Event-Triggered Fuzzy Model Predictive Control

This article focuses on dynamic event-triggered mechanism(DETM)-based model predictive control(MPC) for T-S fuzzy systems.A hybrid dynamic variables-dependent DETM is carefully devised,which includes a multiplicative dynamic variable and an additive dynamic variable.The addressed DETM-based fuzzy MPC issue is described as a “min-max” optimization problem(OP).To facilitate the co-design of the MPC controller and the weighting matrix of the DETM,an auxiliary OP is proposed based on a new Lyapunov function and a new robust positive invariant(RPI) set that contain the membership functions and the hybrid dynamic variables.A dynamic event-triggered fuzzy MPC algorithm is developed accordingly,whose recursive feasibility is analysed by employing the RPI set.With the designed controller,the involved fuzzy system is ensured to be asymptotically stable.Two examples show that the new DETM and DETM-based MPC algorithm have the advantages of reducing resource consumption while yielding the anticipated performance.

Adaptive Nonlinear PD Controller of Two-Wheeled Self-Balancing Robot with External Force

This paper proposes an adaptive nonlinear proportional-derivative(ANPD)controller for a two-wheeled self-balancing robot(TWSB)modeled by the Lagrange equation with external forces.The proposed control scheme is designed based on the combination of a nonlinear proportional-derivative(NPD)controller and a genetic algorithm,in which the proportional-derivative(PD)parameters are updated online based on the tracking error and the preset error threshold.In addition,the genetic algorithm is employed to adaptively select initial controller parameters,contributing to system stability and improved control accuracy.The proposed controller is basic in design yet simple to implement.The ANPD controller has the advantage of being computationally lightweight and providing high robustness against external forces.The stability of the closed-loop system is rigorously analyzed and verified using Lyapunov theory,providing theoretical assurance of its robustness.Simulations and experimental results show that the TWSB robot with the proposed ANPD controller achieves quick balance and tracks target values with very small errors,demonstrating the effectiveness and performance of the proposed controller.The proposed ANPD controller demonstrates significant improvements in balancing and tracking performance for two-wheeled self-balancing robots,which has great applicability in the field of robot control systems.This represents a promising solution for applications requiring precise and stable motion control under varying external conditions.

Dynamical analysis,control,boundedness,and prediction for a fractional-order financial risk system

This paper delves into the dynamical analysis,chaos control,Mittag–Leffler boundedness(MLB),and forecasting a fractional-order financial risk(FOFR)system through an absolute function term.To this end,the FOFR system is first proposed,and the adomian decomposition method(ADM)is employed to resolve this fractional-order system.The stability of equilibrium points and the corresponding control schemes are assessed,and several classical tools such as Lyapunov exponents(LE),bifurcation diagrams,complexity analysis(CA),and 0–1 test are further extended to analyze the dynamical behaviors of FOFR.Then the global Mittag–Leffler attractive set(MLAS)and Mittag–Leffler positive invariant set(MLPIS)for the proposed financial risk(FR)system are discussed.Finally,a proficient reservoir-computing(RC)method is applied to forecast the temporal evolution of the complex dynamics for the proposed system,and some simulations are carried out to show the effectiveness and feasibility of the present scheme.

On Zero Dynamics and Controllable Cyber-Attacks in Cyber-Physical Systems and Dynamic Coding Schemes as Their Countermeasures

In this paper, we study stealthy cyber-attacks on actuators of cyber-physical systems(CPS), namely zero dynamics and controllable attacks. In particular, under certain assumptions, we investigate and propose conditions under which one can execute zero dynamics and controllable attacks in the CPS. The above conditions are derived based on the Markov parameters of the CPS and elements of the system observability matrix. Consequently, in addition to outlining the number of required actuators to be attacked, these conditions provide one with the minimum system knowledge needed to perform zero dynamics and controllable cyber-attacks. As a countermeasure against the above stealthy cyber-attacks, we develop a dynamic coding scheme that increases the minimum number of the CPS required actuators to carry out zero dynamics and controllable cyber-attacks to its maximum possible value. It is shown that if at least one secure input channel exists, the proposed dynamic coding scheme can prevent adversaries from executing the zero dynamics and controllable attacks even if they have complete knowledge of the coding system. Finally, two illustrative numerical case studies are provided to demonstrate the effectiveness and capabilities of our derived conditions and proposed methodologies.

Controlling the dynamic behavior of decentralized cluster through centralized approaches

How to control the dynamic behavior of large-scale artificial active matter is a critical concern in experimental research on soft matter, particularly regarding the emergence of collective behaviors and the formation of group patterns. Centralized systems excel in precise control over individual behavior within a group, ensuring high accuracy and controllability in task execution. Nevertheless, their sensitivity to group size may limit their adaptability to diverse tasks. In contrast, decentralized systems empower individuals with autonomous decision-making, enhancing adaptability and system robustness. Yet, this flexibility comes at the cost of reduced accuracy and efficiency in task execution. In this work, we present a unique method for regulating the centralized dynamic behavior of self-organizing clusters based on environmental interactions. Within this environment-coupled robot system, each robot possesses similar dynamic characteristics, and their internal programs are entirely identical. However, their behaviors can be guided by the centralized control of the environment, facilitating the accomplishment of diverse cluster tasks. This approach aims to balance the accuracy and flexibility of centralized control with the robustness and task adaptability of decentralized control. The proactive regulation of dynamic behavioral characteristics in active matter groups, demonstrated in this work through environmental interactions, holds the potential to introduce a novel technological approach and provide experimental references for studying the dynamic behavior control of large-scale artificial active matter systems.

Dynamic Constraint-Driven Event-Triggered Control of Strict-Feedback Systems Without Max/Min Values on Irregular Constraints

This work proposes an event-triggered adaptive control approach for a class of uncertain nonlinear systems under irregular constraints.Unlike the constraints considered in most existing papers,here the external irregular constraints are considered and a constraints switching mechanism(CSM)is introduced to circumvent the difficulties arising from irregular output constraints.Based on the CSM,a new class of generalized barrier functions are constructed,which allows the control results to be independent of the maximum and minimum values(MMVs)of constraints and thus extends the existing results.Finally,we proposed a novel dynamic constraint-driven event-triggered strategy(DCDETS),under which the stress on signal transmission is reduced greatly and no constraints are violated by making a dynamic trade-off among system state,external constraints,and inter-execution intervals.It is proved that the system output is driven to close to the reference trajectory and the semi-global stability is guaranteed under the proposed control scheme,regardless of the external irregular output constraints.Simulation also verifies the effectiveness and benefits of the proposed method.

The Effect of Key Nodes on theMalware Dynamics in the Industrial Control Network

As industrialization and informatization becomemore deeply intertwined,industrial control networks have entered an era of intelligence.The connection between industrial control networks and the external internet is becoming increasingly close,which leads to frequent security accidents.This paper proposes a model for the industrial control network.It includes a malware containment strategy that integrates intrusion detection,quarantine,and monitoring.Basedonthismodel,the role of keynodes in the spreadofmalware is studied,a comparisonexperiment is conducted to validate the impact of the containment strategy.In addition,the dynamic behavior of the model is analyzed,the basic reproduction number is computed,and the disease-free and endemic equilibrium of the model is also obtained by the basic reproduction number.Moreover,through simulation experiments,the effectiveness of the containment strategy is validated,the influence of the relevant parameters is analyzed,and the containment strategy is optimized.In otherwords,selective immunity to key nodes can effectively suppress the spread ofmalware andmaintain the stability of industrial control systems.The earlier the immunization of key nodes,the better.Once the time exceeds the threshold,immunizing key nodes is almost ineffective.The analysis provides a better way to contain the malware in the industrial control network.

Adaptable and Dynamic Access Control Decision-Enforcement Approach Based on Multilayer Hybrid Deep Learning Techniques in BYOD Environment

Organizations are adopting the Bring Your Own Device(BYOD)concept to enhance productivity and reduce expenses.However,this trend introduces security challenges,such as unauthorized access.Traditional access control systems,such as Attribute-Based Access Control(ABAC)and Role-Based Access Control(RBAC),are limited in their ability to enforce access decisions due to the variability and dynamism of attributes related to users and resources.This paper proposes a method for enforcing access decisions that is adaptable and dynamic,based on multilayer hybrid deep learning techniques,particularly the Tabular Deep Neural Network Tabular DNN method.This technique transforms all input attributes in an access request into a binary classification(allow or deny)using multiple layers,ensuring accurate and efficient access decision-making.The proposed solution was evaluated using the Kaggle Amazon access control policy dataset and demonstrated its effectiveness by achieving a 94%accuracy rate.Additionally,the proposed solution enhances the implementation of access decisions based on a variety of resource and user attributes while ensuring privacy through indirect communication with the Policy Administration Point(PAP).This solution significantly improves the flexibility of access control systems,making themmore dynamic and adaptable to the evolving needs ofmodern organizations.Furthermore,it offers a scalable approach to manage the complexities associated with the BYOD environment,providing a robust framework for secure and efficient access management.

Mechanism,prevention,and control of mining-induced dynamic disasters in underground metal mines in China:Challenges and solutions

Metal mineral resources play an indispensable role in the development of the national economy.Dynamic disasters in underground metal mines seriously threaten mining safety,which are major scientific and technological problems to be solved urgently.In this article,the occurrence status and grand challenges of some typical dynamic disasters involving roof falling,spalling,collapse,large deformation,rockburst,surface subsidence,and water inrush in metal mines in China are systematically presented,the characteristics of mining-induced dynamic disasters are analyzed,the examples of dynamic disasters occurring in some metal mines in China are summarized,the occurrence mechanism,monitoring and early warning methods,and prevention and control techniques of these disasters are highlighted,and some new opinions,suggestions,and solutions are proposed simultaneously.Moreover,some shortcomings in current disaster research are pointed out,and the direction of efforts to improve the prevention and control level of dynamic disasters in China’s metal mines in the future is prospected.The integration of forward-looking key innovative theories and technologies in the abovementioned aspects will greatly enhance the cognitive level of disaster prevention and mitigation in China’s metal mining industry and achieve a significant shift from passive disaster relief to active disaster prevention.

Dynamics modeling and optimal control for multi-information diffusion in Social Internet of Things

As an ingenious convergence between the Internet of Things and social networks,the Social Internet of Things(SIoT)can provide effective and intelligent information services and has become one of the main platforms for people to spread and share information.Nevertheless,SIoT is characterized by high openness and autonomy,multiple kinds of information can spread rapidly,freely and cooperatively in SIoT,which makes it challenging to accurately reveal the characteristics of the information diffusion process and effectively control its diffusion.To this end,with the aim of exploring multi-information cooperative diffusion processes in SIoT,we first develop a dynamics model for multi-information cooperative diffusion based on the system dynamics theory in this paper.Subsequently,the characteristics and laws of the dynamical evolution process of multi-information cooperative diffusion are theoretically investigated,and the diffusion trend is predicted.On this basis,to further control the multi-information cooperative diffusion process efficiently,we propose two control strategies for information diffusion with control objectives,develop an optimal control system for the multi-information cooperative diffusion process,and propose the corresponding optimal control method.The optimal solution distribution of the control strategy satisfying the control system constraints and the control budget constraints is solved using the optimal control theory.Finally,extensive simulation experiments based on real dataset from Twitter validate the correctness and effectiveness of the proposed model,strategy and method.

Hierarchical Controller Synthesis Under Linear Temporal Logic Specifications Using Dynamic Quantization

Linear temporal logic(LTL)is an intuitive and expressive language to specify complex control tasks,and how to design an efficient control strategy for LTL specification is still a challenge.In this paper,we implement the dynamic quantization technique to propose a novel hierarchical control strategy for nonlinear control systems under LTL specifications.Based on the regions of interest involved in the LTL formula,an accepting path is derived first to provide a high-level solution for the controller synthesis problem.Second,we develop a dynamic quantization based approach to verify the realization of the accepting path.The realization verification results in the necessity of the controller design and a sequence of quantization regions for the controller design.Third,the techniques of dynamic quantization and abstraction-based control are combined together to establish the local-to-global control strategy.Both abstraction construction and controller design are local and dynamic,thereby resulting in the potential reduction of the computational complexity.Since each quantization region can be considered locally and individually,the proposed hierarchical mechanism is more efficient and can solve much larger problems than many existing methods.Finally,the proposed control strategy is illustrated via two examples from the path planning and tracking problems of mobile robots.

Dynamic Event-Triggered Consensus Control for Input Constrained Multi-Agent Systems With a Designable Minimum Inter-Event Time

This paper investigates the consensus control of multi-agent systems(MASs) with constrained input using the dynamic event-triggered mechanism(ETM).Consider the MASs with small-scale networks where a centralized dynamic ETM with global information of the MASs is first designed.Then,a distributed dynamic ETM which only uses local information is developed for the MASs with large-scale networks.It is shown that the semi-global consensus of the MASs can be achieved by the designed bounded control protocol where the Zeno phenomenon is eliminated by a designable minimum inter-event time.In addition,it is easier to find a trade-off between the convergence rate and the minimum inter-event time by an adjustable parameter.Furthermore,the results are extended to regional consensus of the MASs with the bounded control protocol.Numerical simulations show the effectiveness of the proposed approach.

Transmission Dynamics and Optimal Control Strategies of a Hand-Foot-Mouth Disease Model with Treatment and Vaccination Interventions

In this article, the transmission dynamics of a Hand-Foot-Mouth disease model with treatment and vaccination interventions are studied. We calculated the basic reproduction number and proved the global stability of disease-free equilibrium when R0 R0 > 1. Meanwhile, we obtained the optimal control strategies minimizing the cost of intervention and minimizing the infected person. We also give some numerical simulations to verify our theoretical results.

Aerodynamic, Kinematic and Control Coupling Simulation of Aircraft Maneuvering Flight

Aiming at the high angle of attack pull-up and multi-channel roll pull-up coupling problems of high maneuvering aircraft, this paper establishes the flight attitude control rate by means of unsteady flow numerical solution, dynamic unstructured nested mesh assembly method and numerical solution method of flight mechanics equation. On this basis, a virtual flight simulation platform integrating pneumatics, motion and control is established. Based on this virtual flight simulation platform, F-16 aircraft is simulated by high angle of attack pull-up flight mode and multi-channel roll pull-up coupling flight mode. Finally, the influence of rudder on the yaw control channel is investigated. The results show that the numerical virtual flight simulation platform established in this paper has the ability to simulate maneuvering flight of aircraft.

Robust adaptive dynamic surface control for nonlinear uncertain systems

We propose a new method for robust adaptive backstepping control of nonlinear systems with parametric uncertainties and disturbances in the strict feedback form. The method is called dynamic surface control. Traditional backstepping algorithms require repeated differentiations of the modelled nonlinearities. The addition of n first order low pass filters allows the algorithm to be implemented without differentiating any model nonlinearities, thus ending the complexity arising due to the 'explosion of terms' that makes other methods difficult to implement in practice. The combined robust adaptive backstepping/first order filter system is proved to be semiglobally asymptotically stable for sufficiently fast filters by a singular perturbation approach. The simulation results demonstrate the feasibility and effectiveness of the controller designed by the method.

Vibration control of pedestrian-bridge vertical dynamic coupling interaction based on biodynamic model

The human-induced vertical vibration serviceability of low-frequency and lightweight footbridges is studied based on the moving mass-spring-damper（MMSD） biodynamic model, and the mass damper（TMD） with different optimal model parameters being used to control the vertical vibration.First, the MMSD biodynamic model is employed to simulate the pedestrians, and the time-varying control equations of the vertical dynamic coupling system of the pedestrian-bridgeTMD are established with the consideration of pedestrianbridge dynamic interaction; and the equations are solved by using the Runge-Kutta-Felhberg integral method with variable step size. Secondly, the footbridge dynamic response is calculated under the model of pedestrian-structure dynamic interaction and the model of moving load when the pedestrian pace frequency is consistent with the natural frequency of footbridge. Finally, a comparative study and analysis are made on the control effects of the vertical dynamic coupling system in different optimal models of the TMD. The calculation results show that the pedestrian-bridge dynamic interaction cannot be ignored when the vertical human-induced vibration serviceability of low-frequency and light-weight footbridge is evaluated. The TMD can effectively reduce the vibration under the resonance of pedestrian-bridge, and TMD parameters are recommended for the determination by the Warburton optimization model.

Welding of Ti-6Al-4V alloy using dynamically controlled plasma arc welding process

A novel dynamically controlled plasma arc welding process was introduced,which is able tominimize heat input into the workpiece materials while maintaining desired full penetration,and it was used to weld Ti-6Al-4V alloy sheets.The microstructures,facture surfaces and microhardness of the welded joints were characterized by using optical microscope,scanning electron microscope （SEM） and Vickers microhardness tester.Comparing with welds such as gas tungsten arc and conventional plasma arc processes,the experimental results revealed the improvements when using the present process including：1） reducing prior-beta （β） grain size and prohibiting formation of hard martensite phases in the fusion zone due to the decreased heat input;and 2） better toughness and higher hardness.

A High Speed IGBT Based on Dynamic Controlled Anode-Short

IGBT with high switching speed is described based on the dynamic controlled anode- short,which incorpo- rates a normally- on,p- MOSFET controlled by the anode voltage indirectly.This device works just as normal when it is in on- state since the channel of the p- MOSFET is pinched- off.During the course of turning off,the channel of the p- MOSFET will prevent the injection of m inorities and introduce an extra access for the carriers to flow to the anode directly,which m akes the IGBT reach its off- state in a shorter time.The simulation results prove that the new structure can reduce the turn- off time by m ore than75 % compared with the normal one under the same break- down voltage and on- state perform ance.Only two more resistors are needed when using this structure,and the re- quirement of the drive circuits is just the sam e as normal.

Dynamic simulation analysis of molten salt reactor-coupled air-steam combined cycle power generation system

A nonlinear dynamic simulation model based on coordinated control of speed and flow rate for the molten salt reactor and combined cycle systems is proposed here to ensure the coordination and stability between the molten salt reactor and power system.This model considers the impact of thermal properties of fluid variation on accuracy and has been validated with Simulink.This study reveals the capability of the control system to compensate for anomalous situations and maintain shaft stability in the event of perturbations occurring in high-temperature molten salt tank outlet parameters.Meanwhile,the control system’s impact on the system’s dynamic characteristics under molten salt disturbance is also analyzed.The results reveal that after the disturbance occurs,the controlled system benefits from the action of the control,and the overshoot and disturbance amplitude are positively correlated,while the system power and frequency eventually return to the initial values.This simulation model provides a basis for utilizing molten salt reactors for power generation and maintaining grid stability.