Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Control performance monitoring and advanced control towards energy conservation

High performance control of an interactive process such as iron and steel plant relies on ability to honor safety and operational constraints;reduce the standard deviations of variables that need to be controlled(e.g.product quantity,quality );de-bottlenecking the process;and,maximize profitability or lower cost(e.g.energy savings, improve hot metal content).These objectives may be prioritized in this order,but can vary and are very difficult to achieve optimally through conventional control.A multivariable predictive controller solution,along with its extensive inferential sensor and built-in optimizer,provides online closed loop control and optimization for many interactive metal and mining processes to lower the energy cost,increase throughput,and optimize product quality and yield. Control loop performance is also a key factor to improve iron and steel plant automation and operation result; Honeywell CPM offers vender-independent product which provides monitoring,tuning,modeling of control loop and sustainable loop performance analysis and maintenance solution towards operation stability and energy saving.

Overview of Advanced Control Strategies for Electric Machines

Control strategies play a key role for operation of electric machines,which would directly affect the whole system performance.In fact,different control strategies have been executed and explored for electric machines,which bring great impacts to industrial development and human society.This paper investigates and discusses the advantages control strategies for electric machines,including the field oriented control(FOC),direct torque control(DTC),finite control set model predictive control(FCS-MPC),sensorless control,and fault tolerant control(FTC).The corresponding control principles,control targets,fundamental approaches,advanced approaches,methodologies,merits and shortcomings are revealed and analyzed in detail.

Improved Disturbance Observer (DOB) Based Advanced Feedback Control for Optimal Operation of a Mineral Grinding Process

Advanced feedback control for optimal operation of mineral grinding process is usually based on the model predictive control (MPC) dynamic optimization. Since the MPC does not handle disturbances directly by controller design, it cannot achieve satisfactory effects in controlling complex grinding processes in the presence of strong disturbances and large uncertainties. In this paper, an improved disturbance observer (DOB) based MPC advanced feedback control is proposed to control the multivariable grinding operation. The improved DOB is based on the optimal achievable H 2 performance and can deal with disturbance observation for the nonminimum-phase delay systems. In this DOB-MPC advanced feedback control, the higher-level optimizer computes the optimal operation points by maximize the profit function and passes them to the MPC level. The MPC acts as a presetting controller and is employed to generate proper pre-setpoint for the lower-level basic feedback control system. The DOB acts as a compensator and improves the operation performance by dynamically compensating the setpoints for the basic control system according to the observed various disturbances and plant uncertainties. Several simulations are performed to demonstrate the proposed control method for grinding process operation.

Problems and Strategies in Applications of Advanced Process Control in China

Advanced Process Control （APC） is necessary for oil refining and chemical process in China, but some problems have emerged in the application of APC techniques in this field. This paper discusses the conditions of APC application concerning process design, distributed control system （DCS） choice and regular control. It analyzes the problems and strategies in APC application. Some suggestions are proposed for the enterprise to benefit from APC application.

Optimized model-based control of main mine ventilation air flows with minimized energy consumption

In early 2018,the Boliden Garpenberg operation implemented an optimized control strategy as an addition to the existing ventilation on demand system.The purpose of the strategy is to further minimize energy use for main and booster fans,whilst also fulfilling airflow setpoints without violating constraints such as min/max differential pressure over fans and interaction of air between areas in mines.Using air flow measurements and a dynamical model of the ventilation system,a mine-wide coordination control of fans can be carried out.The numerical model is data driven and derived from historical operational data or step changes experiments.This makes both initial deployment and lifetime model maintenance,as the mine evolves,a comparably easy operation.The control has been proven to operate in a stable manner over long periods without having to re-calibrate the model.Results prove a 40%decrease in energy use for the fans involved and a greater controllability of air flow.Moreover,a 15%decrease of the total air flow into the mine will give additional proportional heating savings during winter periods.All in all,the multivariable controller shows a correlation between production in the mine and the ventilation system performance superior to all of its predecessors.

Special Issue on "Recent Advances in Power System Control" for International Journal of Control,Automation,and Systems

Today the high quality power supply is of essential in the economic development in a country. With the development of modem power systems and increasing demand for power supply, the electric power industry is facing a great challenge in meeting the increased load demand with highest reliability and security with minimum transmission expenditure. Power system stability analysis and control is one of the most important issues in power systems. The problem becomes more and more serious in power systems with the development of modem power systems. The ability of a power system

Experimental and Simulation-based Investigation of a Velocity Controller Extension on a Ball Screw System

The background of this contribution is the enhancement of the achievable accuracy of servo-screw-presses. Therefore, the paper is concerned with the improvement of the dynamic precision for direct driven servo axes which is still restricted by structural vibrations. For this purpose, a ball screw test rig was analyzed for which the standard cascade control structure was extended by an additional velocity feedback. This structural extension has the potential to improve the controller performance significantly due to a better damping of low frequent vibrations. Furthermore, a parametric dynamic model for the control structure was derived to investigate the effects of the controller extension. For this analysis, the influences of the used tuning factors and filters is discussed in the frequency domain based on bode plots. The results of these cognitions are transferred to the time domain and illustrated by step responses. In addition, an evaluation by the criterion of the IAE （integral of absolute error） and the Prony analysis is carried out. Finally, the results are experimentally verified at the ball screw test rig. The paper closes with the conclusions.

Call for Papers:Special Issue on "Recent Advances and New Directions in Switched Control Systems" ——Jounal of Control Theory and Application

Recently, switched control systems have been attracting much attention m the control community because the problems are not only academically challenging for the inherent complexity, but also of practical importance due to its wide ranges of applications in nature, engineering, and social sciences.

Special Issue on Recent Advances and New Directions in Switched Control Systems

As guest editors, it is our great honor to bring this special issue of the Journal of Control Theory and Applications devoted to Recent Advances and New Directions in Switched Control Systems.

STRATEGY AND IMPLEMENTATION FOR HIGH-SPEED TORQUE ENHANCEMENT OF BRUSHLESS DC MOTOR

Serious commutation lag occurs when a Brushless DC Motor(BLDCM) operates at high speeds,and this leads to torque decline with ripple.In this paper,an advanced conduction control scheme is proposed which can accelerate the commutation and enhance the torque production remarkably.Besides,an on line adjusting algorithm based on the Golden Section Method is adopted to search the optimal advanced conduction angle.Simulation and experimental results verify the feasibility and effectivity of the scheme proposed.

Editorial for Special Issue on Intelligent Control and Computing in Advanced Robotics

Artificial intelligence （AI）, robotics, and intelligent systems are increasingly penetrating our society. With recent advances in AI, especially in intelligent control and computing, robots are gaining the ability to learn, make decisions, and operate in ways much similar to humans.To a large extent.

Call for Papers

Recently,switched control systems have been attracting much attention in the control community because the problems are not only academically challenging for the inherent complexity, but also of practical importance due to its wide ranges of applications in nature,engineering,and social sciences. For survival,natural biological systems switch in strategies in accordance to environmental changes. For improved perfor-mance , switching has been extensively utilized/exploited in many engineering systems such as electronics, power systems, and traffic control,among others.In addition,switched behaviors exhibit in social systems.

New advances in EMG control methods of anthropomorphic prosthetic hand

Prosthetic hands are becoming more and more popular among people without a hand and it is able to perform certain manual operations in function.In biology,action potentials trigger muscle contractions,and surface electromyography（sEMG）signals are the sum of action potentials under the skin exposed by the electrodes（via sebum and solution conduction）.

Improved Synergetic Current Control for Grid-connected Microgrids and Distributed Generation Systems

This paper presents the development of improved synergetic current control for the injected current of an inverter in the grid-connected microgrid and the distributed generation system(DGS). This paper introduces new formulas of the macro-variable functions for integral synergetic control(SC)and integral fast terminal SC, which both have an integral term to guarantee zero steady-state error. The proposed integral SC and integral fast terminal SC achieve a seamless performance such as the fast convergence, minimal overshoot, zero steady-state error, and chattering-free operation. To demonstrate the meritorious performance of the proposed scheme for injected current control, it is compared with the performance of a proportional-integral(PI) controller and advanced exponential sliding mode control(SMC). Finally, the practicality of the proposed scheme is justified by experimental results obtained through rapid control prototyping(RCP) using the dSPACESCALEXIO platform.

ADVANCEMENT OF TUBERCULOSIS CONTROL AND RESEARCH IN CHINA

Since the founding of new China, the status of tuberculosis (TB) has been greatly improved through effective control measures and application of research achievements. In the early 1950s, more than 90% of adults in Shanghai and Tianjin had been infected. Tuberculosis ranked the first among the leading causes of death in big cities and the mortality rate reached 169-259/100000 in Beijing, Shanghai, Tianjin, Guangzhou and Hangzhou. The estimated prevalence of tuberculosis in the urban area (3500/100000) was higher than that in the countryside (1500/100000).

Bioreactor control systems in the biopharmaceutical industry:a critical perspective

Industrial-scale bioprocessing underpins much of the production of pharmaceuticals,nutraceuticals,food,and beverage processing industries of the modern world.The proftability of these processes increasingly leverages the economies of scale and scope that are critically dependent on the product yields,titers,and productivity.Most of the processes are controlled using classical control approaches and represent over 90%of the industrial controls used in bioprocessing industries.However,with the advances in the production processes,especially in the biopharmaceutical and nutraceutical industries,monitoring and control of bioprocesses such as fermentations with GMO organisms,and downstream processing has become increasingly complex and the inadequacies of the classical and some of the modern control systems techniques is becoming apparent.Therefore,with increasing research complexity,nonlinearity,and digitization in process,there has been a critical need for advanced process control that is more efective,and easier process intensifcation and product yield(both by quality and quantity)can be achieved.In this review,industrial aspects of a process and automation along with various commercial control strategies have been extensively discussed to give an insight into the future prospects of industrial development and possible new strategies for process control and automation with a special focus on the biopharmaceutical industry.

Editorial Special issue on recent advances in hybrid control theory and applications

The demands of a rapidly advancing technology for faster and more accurate controllers have always had a strong influence on the progress of automatic control theory. In recent years control problems have been arising with increasing frequency in widely different areas, which cannot be addressed using conventional control techniques. The principal reason for this is the fact that a highly competitive economy is forcing systems to operate in regimes where presence of large uncertainties as well as nonlinearities plays a major role. For instance, the design of unmanned aircraft, unmanned underwater vehicles, and automatic driving systems for automobiles are all typical examples of such difficult control problems. The large uncertainties concerning the plant may arise due to large external disturbances, large parameter variations because of faults in the system, or failure of some of the subsystems. In such cases, the controller has to determine the specific situation that exists at any instant, and take the appropriate control action. Accomplishing this rapidly, accurately, and in a stable fashion is the objective of control design.

Power Electronics:The Enabling Technology for Renewable Energy Integration

The markedly increased integration of renewable energy in the power grid is of significance in the transition to a sustainable energy future.The grid integration of renewables will be continuously enhanced in the future.According to the International Renewable Energy Agency(IRENA),renewable technology is the main pathway to reach zero carbon dioxide(CO_(2))emissions by 2060.Power electronics have played and will continue to play a significant role in this energy transition by providing efficient electrical energy conversion,distribution,transmission,and utilization.Consequently,the development of power electronics technologies,i.e.,new semiconductor devices,flexible converters,and advanced control schemes,is promoted extensively across the globe.Among various renewables,wind energy and photovoltaic(PV)are the most widely used,and accordingly these are explored in this paper to demonstrate the role of power electronics.The development of renewable energies and the demands of power electronics are reviewed first.Then,the power conversion and control technologies as well as grid codes for wind and PV systems are discussed.Future trends in terms of power semiconductors,reliability,advanced control,grid-forming operation,and security issues for largescale grid integration of renewables,and intelligent and full user engagement are presented at the end.

Cyber security of railway cyber-physical system(CPS)-A risk management methodology

Along with the increasing application of different cyber-physical systems(CPSs)to connect various components in the rail industry,rising connectivity through communication technologies has also introduced cyber threats against rail-CPSs,causing failures with huge consequences.Implementations of rail-CPSs demand proactive identification,clear-cut definition,and proper handling of their cyber security risks.In this paper,we formulate a risk management methodology for cyber security in rail-CPSs and conduct a retrospective case study on the Advanced Train Control System(ATCS)that has been deployed in many U.S.freight railways.The methodology provides two alternative approaches to fill knowledge gaps in contingency preparation,threat prevention,consequence analysis,and security risk mitigation.In the case study,we demonstrate two cyber threats of ATCS,using attack sequence modeling and consequence analysis,and provide recommendations for risk mitigation.By practicing the methodology with the case study,this work can be used as a general reference to conduct cyber risk management and cyber-robustness evaluations for other existing systems.