Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Self-Triggered Set Stabilization of Boolean Control Networks and Its Applications

Set stabilization is one of the essential problems in engineering systems, and self-triggered control(STC) can save the storage space for interactive information, and can be successfully applied in networked control systems with limited communication resources. In this study, the set stabilization problem and STC design of Boolean control networks are investigated via the semi-tensor product technique. On the one hand, the largest control invariant subset is calculated in terms of the strongly connected components of the state transition graph, by which a graph-theoretical condition for set stabilization is derived. On the other hand, a characteristic function is exploited to determine the triggering mechanism and feasible controls. Based on this, the minimum-time and minimum-triggering open-loop, state-feedback and output-feedback STCs for set stabilization are designed,respectively. As classic applications of self-triggered set stabilization, self-triggered synchronization, self-triggered output tracking and self-triggered output regulation are discussed as well. Additionally, several practical examples are given to illustrate the effectiveness of theoretical results.

The Application and Design of the Economical Nonlinear Controller

This anticle gives a design method of the economical nonlinear controller. The controller is composed of an expert intelligent coordination controller, a fuzzy prediction controller, a fuzzy feedforward controller, a nonlinear controller and so on. The consistence of a distributed control system based on this controller is also shown briefly.

H~∞ Theory and its Application to the Position of Rolling Mill Control System

The controller designed according to classical or modern control theory will not satisfy the performance requirements when the controlled object in industrial field can not be described by exact mathematical model or the disturbance of the controlled system. In order to make the controlled system stable and having good performance, H∞ control theory was put forward to solve this practical problem. Taking the position of a rolling mill as the controlled object, it was rectified by optimal engineering way. Then, three different plans were put forward according to Bang-Bang control, LQ control and H∞ control, respectively. The result of the simulation shows that the controller designed according to H∞ method whose robust performance and ability to restrain colors disturbance is satisfactory.

A Survey on Type-3 Fuzzy Logic Systems and Their Control Applications

In this paper,we offer a review of type-3 fuzzy logic systems and their applications in control.The main objective of this work is to observe and analyze in detail the applications in the control area using type-3 fuzzy logic systems.In this case,we review their most important applications in control and other related topics with type-3 fuzzy systems.Intelligent algorithms have been receiving increasing attention in control and for this reason a review in this area is important.This paper reviews the main applications that make use of Intelligent Computing methods.Specifically,type-3 fuzzy logic systems.The aim of this research is to be able to appreciate,in detail,the applications in control systems and to point out the scientific trends in the use of Intelligent Computing techniques.This is done with the construction and visualization of bibliometric networks,developed with VosViewer Software,which it is a free Java-based program,mainly intended to be used for analyzing and visualizing bibliometric networks.With this tool,we can create maps of publications,authors,or journals based on a co-citation network or construct maps of keywords,countries based on a co-occurrence networks,research groups,etc.

Establishment and Application of Prevention and Control Techniques of New Sugarcane White Leaf Disease

Aiming at the basic and key technical problems in prevention and control of sugarcane white leaf disease(SCWL),this study systematically overcame key technical bottleneck of prevention and control of new SCWL after 10 years of collaborative research,and achieved several innovative achievements.SCWL phytoplasmas newly recorded in China and the new subgroup of SCWL phytoplasmas(16SrXI-D)were discovered for the first time in Yunnan,and the whole genome analysis of the epidemic subgroup was completed.The main transmission source of SCWL pathogens has been identified as infected seed canes,and Tettigoniella viridis and Clovia conifer were newly discovered as vectors for virus transmission.The disease resistance of 25 main varieties was identified,and 10 control varieties were selected.The prevention and control strategy of"emphasizing early warning,strictly carrying out quarantine,blocking the vectors and controlling residual plants"was put forward,and a comprehensive prevention technique was established through integration of various techniques,and standardized technical regulations were formulated for demonstration application.The promotion and application of these achievements have realized scientific prevention and control of SCWL,effectively curbed the spread of SCWL,and ensured the safety of sugarcane producing areas in China,achieving great economic,social and ecological benefits and providing technical support for high-quality development,loss reduction and efficiency improvement of China's sugar industry.

Migration and Transformation of Coated Controlled Release Nitrogen in Soil under Co-situs Application Mode

By using field trials, the migration and transformation of coated controlled release nitrogen （YZS80） in soil under the co-situs application mode were studied. The results indicated that YZS80 nitrogen dissolved out in average speed of about 0.4%/d and 32% totally in 80 d compared with common compound fertUizer. For YZS80, in the vertical downward direction of application points, urea nitrogen content increased significantly （P 〈0.05） in the 45 -80 d and the 30 -60 cm soil layers; nitrate nitrogen content was little change range and in moderate （10 -100 mg/kg） in 0 -80 d and 10 -60 cm soil layers, but increased significantly （ P 〈0.05） 45 -80 d compared to 0 -45 d; ammonium nitrogen content was significant lower （P〈0.05） before 45 d in 10 -30 cm soil layers, but significant higher（P〈0.05） after 45 d in 10 -60 cm soil layers; NO3^- -N/NH4^＋ -N meets gradually the needs of the crop with the extension of time. Under the co-situs application mode, the possibility of burning root and salt injury and loss dsk of nitrate leaching is a significant reduction.

Review of the Tuned Mass Damper Inerter(TMDI)in Energy Harvesting and Vibration Control:Designs,Analysis and Applications

Tuned mass damper inerter(TMDI)is a device that couples traditional tuned mass dampers(TMD)with an inertial device.The inertial device produces resistance proportional to the relative acceleration at its two ends through its“inertial”constant.Due to its unique mechanical properties,TMDI has received widespread attention and application in the past twenty years.As different configurations are required in different practical situations,TMDI is still active in the research on vibration control and energy harvesting in structures.This paper provides a comprehensive review of the research status of TMDI.This work first examines the generation and important vibration control characteristics of TMDI.Then,the energy harvesting performance of electromagnetic tuned mass damper inerter(EM-TMDI)is discussed.This work emphasizes the formation of a passive dynamic vibration absorber by coupling traditional TMD with inertial devices.This paper also summarizes the design and implementation of optimal vibration control and energy harvesting for TMDI.Furthermore,this paper details the applications of TMDI in the fields of bridges and building engineering.Finally,this paper summarizes the necessity of research on tuned mass-damper-inertia,the challenges faced currently,and future research directions,such as control of parameters in electromagnetic energy harvesting TMDI systems and low-cost TMDI.

An Overview of Finite/Fixed-Time Control and Its Application in Engineering Systems

The finite/fixed-time stabilization and tracking control is currently a hot field in various systems since the faster convergence can be obtained. By contrast to the asymptotic stability,the finite-time stability possesses the better control performance and disturbance rejection property. Different from the finite-time stability, the fixed-time stability has a faster convergence speed and the upper bound of the settling time can be estimated. Moreover, the convergent time does not rely on the initial information.This work aims at presenting an overview of the finite/fixed-time stabilization and tracking control and its applications in engineering systems. Firstly, several fundamental definitions on the finite/fixed-time stability are recalled. Then, the research results on the finite/fixed-time stabilization and tracking control are reviewed in detail and categorized via diverse input signal structures and engineering applications. Finally, some challenging problems needed to be solved are presented.

Research on the Application of Evidence-Based Quality Control Circle to Improve the Implementation Rate of Airway Management Measures in Adult Critically Ill Patients

Objective: To explore the effect of evidence-based quality control circle (QCC) in improving the implementation rate of airway management measures in adult critically ill patients. Methods: Based on the Joanna Briggs Institute (JBI) evidence-based health care model, the best evidence of airway management in adult critically ill patients was obtained and applied to the clinic. Results: The total implementation rate of airway management measures in adult critically ill patients increased from 23.62% before the implementation of quality control circle to 88.82%, and the pulmonary infection rate in critically ill patients decreased from 42.31% to 21.74%, with statistical significance between the two groups (P 0.05). Conclusion: Evidence-based quality control circle activities can standardize the practice standards of airway management in critically ill patients, reduce the occurrence of patients’ airway related complications, and improve clinical outcomes.

Design and Application of Intelligent Control System for Molten Iron Transportation Based on 5G Technology

Molten transport is an important link in the iron and steel enterprise production,involves many complex factors,artificial management is difficult.Therefore,puts forward a kind of molten iron transport wisdom control system based on 5G technology,which mainly contains the intelligent identification tracking system,equipment status collection information acquisition system,locomotive vehicle terminal system,etc.Combined with the analysis of the actual application situation,the system could integrate all the processes and elements of molten iron produc-tion and transportation,realize the integration of operation and management,and also promote the improvement of the turnover efficiency of molten iron tank,reduce the demand for personnel,and reduce the labor cost.

High-Order Volterra Model Predictive Control and Its Application to a Nonlinear Polymerisation Process

Model Predictive Control (MPC) has recently found wide acceptance in the process industry, but existing design and implementation methods are restricted to linear process models. A chemical process, however, involves severe nonlinearity which cannot be ignored in practice. This paper aims to solve this nonlinear control problem by extending MPC to accommodate nonlinear models. It develops an analytical framework for nonlinear model predictive control (NMPC). It also offers a third-order Volterra series based nonparametric nonlinear modelling technique for NMPC design, which relieves practising engineers from the need for deriving a physical-principles based model first. An on-line realisation technique for implementing NMPC is then developed and applied to a Mitsubishi Chemicals polymerisation reaction process. Results show that this nonlinear MPC technique is feasible and very effective. It considerably outperforms linear and low-order Volterra model based methods. The advantages of the developed approach lie not only in control performance superior to existing NMPC methods, but also in eliminating the need for converting an analytical model and then convert it to a Volterra model obtainable only up to the second order. Keywords Model predictive control - Volterra series - process control - nonlinear control Yun Li is a senior lecturer at University of Glasgow, UK, where has taught and researched in evolutionary computation and control engineering since 1991. He worked in the UK National Engineering Laboratory and Industrial Systems and Control Ltd, Glasgow in 1989 and 1990. In 1998, he established the IEEE CACSD Evolutionary Computation Working Group and the European Network of Excellence in Evolutionary Computing (EvoNet) Workgroup on Systems, Control, and Drives. In summer 2002, he served as a visiting professor to Kumamoto University, Japan. He is also a visiting professor at University of Electronic Science and Technology of China. His research interests are in parallel processing, design automation and discovery of engineering systems using evolutionary learning and intelligent search techniques. Applications include control, system modelling and prediction, circuit design, microwave engineering, and operations management. He has advised 12 Ph.D.s in evolutionary computation and has 140 publications.Hiroshi Kashiwagi received B.E, M.E. and Ph.D. degrees in measurement and control engineering from the University of Tokyo, Japan, in 1962, 1964 and 1967 respectively. In 1967 he became an Associate Professor and in 1976 a Professor at Kumamoto University. From 1973 to 1974, he served as a visiting Associate Professor at Purdue University, Indiana, USA. From 1990 to 1994, he was the Director at Computer Center of Kumamoto University. He has also served as a member of Board of Trustees of Society of Instrument and Control Engineers (SICE), Japan, Chairman of Kyushu Branch of SICE and General Chair of many international conferences held in Japan, Korea, Chin and India. In 1994, he was awarded SICE Fellow for his contributions to the field of measurement and control engineering through his various academic activities. He also received the Gold Medal Prize at ICAUTO’95 held in India. In 1997, he received the “Best Book Award” from SICE for his new book entitled “M-sequence and its application” written in Japanese and published in 1996 by Shoukoudou Publishing Co. in Japan. In 1999, he received the “Best Paper Award” from SICE for his paper “M-transform and its application to system identification”. His research interests include signal processing and applications, especially pseudorandom sequence and its applications to measurement and control engineering.

ADAPTIVE VARIABLE STRUCTURE CONTROLLERS AND APPLICATION TO ENGINE IDLE SPEED CONTROL SIMULATION

A neural-network-based adaptive variable structure control methodology isproposed for the tracking problem of nonlinear discrete-time input-output systems. The unknowndynamics of the system are approximated via radial basis function neural networks. The control lawis based on sliding modes and simple to implement. The discrete-time adaptive law for tuning theweight of neural networks is presented using the adaptive filtering algorithm with residueupper-bound compensation. The application of the proposed controller to engine idle speed controldesign is discussed. The results indicate the validation and effectiveness of this approach.

Clinical study on the treatment of acute pancreatitis with external application of Yi-Dan Therapy based on the method of“stagnation requiring dispersion”:study protocol for a randomized controlled trial

Background:Acute pancreatitis is an unpredictable and potentially lethal disease,causing tremendous pain in patients.The initial treatment of acute pancreatitis in modern medicine is supportive,but it is generally ineffective in relieving abdominal pain and distension.Traditional Chinese medicine has been shown to be more effective in regulating the body’s homeostasis and reducing the clinical symptoms of pancreatitis.Yi-Dan ointment,derived from Dahuang-Mudan Decoction,is an effective external ointment for treating acute pancreatitis.The aim of this trial is to investigate the clinical efficacy of Yi-Dan ointment,providing a valuable complement to existing treatment options for pancreatitis.Methods:This is a randomized controlled clinical trial with two parallel groups.Patients in the control group were given basic treatment and nursing for 7 days;in the treatment group,Yi-Dan ointment was applied externally in addition to basic treatment and nursing.The main indicator is the overall efficacy,serum amylase,acute physiology and chronic health evaluationⅡscore,symptom score,inflammatory markers,and classification of computed tomography.Conclusion:The trial results will provide high-quality evidence for Yi-Dan ointment,and provide a complement to existing treatment options for pancreatitis.

Modeling of Microstructure Evolution and Mechanical Properties of Steel Plates Produced by Thermo-Mechanical Control Process and Its On-line Application

An integrated metallurgical model was developed to predict microstructure evolution and mechanical properties of low-carbon steel plates produced by TMCP. The metallurgical phenomena occurring during TMCP and mechanical properties were predicted for different process parameters. In the later passes full recrystallization becomes difficult to occur and higher residual strain remains in austenite after rolling. For the reasonable temperature and cooling schedule, yield strength of 30 mm plain carbon steel plate can reach 310 MPa. The first on-line application of prediction and control of microstructure and properties (PCMP) in the medium plate production was achieved. The predictions of the system are in good agreement with measurements.

Robust Tracking Control of Uncertain MIMO Nonlinear Systems with Application to UAVs

In this paper, we consider the robust adaptive tracking control of uncertain multi-input and multi-output (MIMO) nonlinear systems with input saturation and unknown external disturbance. The nonlinear disturbance observer (NDO) is employed to tackle the system uncertainty as well as the external disturbance. To handle the input saturation, an auxiliary system is constructed as a saturation compensator. By using the backstepping technique and the dynamic surface method, a robust adaptive tracking control scheme is developed. The closed-loop system is proved to be uniformly ultimately bounded thorough Lyapunov stability analysis. Simulation results with application to an unmanned aerial vehicle (UAV) demonstrate the effectiveness of the proposed robust control scheme. © 2014 Chinese Association of Automation.

Cloud Control System Architectures,Technologies and Applications on Intelligent and Connected Vehicles:a Review

The electrification of vehicle helps to improve its operation efficiency and safety.Due to fast development of network,sensors,as well as computing technology,it becomes realizable to have vehicles driving autonomously.To achieve autonomous driving,several steps,including environment perception,path-planning,and dynamic control,need to be done.However,vehicles equipped with on-board sensors still have limitations in acquiring necessary environmental data for optimal driving decisions.Intelligent and connected vehicles(ICV)cloud control system(CCS)has been introduced as a new concept as it is a potentially synthetic solution for high level automated driving to improve safety and optimize traffic flow in intelligent transportation.This paper systematically investigated the concept of cloud control system from cloud related applications on ICVs,and cloud control system architecture design,as well as its core technologies development.Based on the analysis,the challenges and suggestions on cloud control system development have been addressed.

Neural-network adaptive controller for nonlinear systems and its application in pneumatic servo systems

In this paper, a novel control law is presented, which uses neural-network techniques to approximate the affine class nonlinear system having unknown or uncertain dynamics and noise disturbances. It adopts an adaptive control law to adjust the network parameters online and adds another control component according to H-infinity control theory to attenuate the disturbance. This control law is applied to the position tracking control of pneumatic servo systems. Simulation and experimental results show that the tracking precision and convergence speed is obviously superior to the results by using the basic BP-network controller and self-tuning adaptive controller.

Multi-objective nonlinear model predictive control through switching cost functions and its applications to chemical processes

This paper proposes a switching multi-objective model predictive control(MOMPC) algorithm for constrained nonlinear continuous-time process systems.Different cost functions to be minimized in MPC are switched to satisfy different performance criteria imposed at different sampling times.In order to ensure recursive feasibility of the switching MOMPC and stability of the resulted closed-loop system,the dual-mode control method is used to design the switching MOMPC controller.In this method,a local control law with some free-parameters is constructed using the control Lyapunov function technique to enlarge the terminal state set of MOMPC.The correction term is computed if the states are out of the terminal set and the free-parameters of the local control law are computed if the states are in the terminal set.The recursive feasibility of the MOMPC and stability of the resulted closed-loop system are established in the presence of constraints and arbitrary switches between cost functions.Finally,implementation of the switching MOMPC controller is demonstrated with a chemical process example for the continuous stirred tank reactor.

ACS algorithm-based adaptive fuzzy PID controller and its application to CIP-I intelligent leg

Based on the ant colony system （ACS） algorithm and fuzzy logic control, a new design method for optimal fuzzy PID controller was proposed. In this method, the ACS algorithm was used to optimize the input/output scaling factors of fuzzy PID controller to generate the optimal fuzzy control rules and optimal real-time control action on a given controlled object. The designed controller, called the Fuzzy-ACS PID controller, was used to control the CIP-Ⅰ intelligent leg. The simulation experiments demonstrate that this controller has good control performance. Compared with other three optimal PID controllers designed respectively by using the differential evolution algorithm, the real-coded genetic algorithm, and the simulated annealing, it was verified that the Fuzzy-ACS PID controller has better control performance. Furthermore, the simulation results also verify that the proposed ACS algorithm has quick convergence speed, small solution variation, good dynamic convergence behavior, and high computation efficiency in searching for the optimal input/output scaling factors.