Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Special Issue on "Recent Advances in Power System Control" for International Journal of Control,Automation,and Systems

Today the high quality power supply is of essential in the economic development in a country. With the development of modem power systems and increasing demand for power supply, the electric power industry is facing a great challenge in meeting the increased load demand with highest reliability and security with minimum transmission expenditure. Power system stability analysis and control is one of the most important issues in power systems. The problem becomes more and more serious in power systems with the development of modem power systems. The ability of a power system

Problems and Strategies in Applications of Advanced Process Control in China

Advanced Process Control （APC） is necessary for oil refining and chemical process in China, but some problems have emerged in the application of APC techniques in this field. This paper discusses the conditions of APC application concerning process design, distributed control system （DCS） choice and regular control. It analyzes the problems and strategies in APC application. Some suggestions are proposed for the enterprise to benefit from APC application.

Control performance monitoring and advanced control towards energy conservation

High performance control of an interactive process such as iron and steel plant relies on ability to honor safety and operational constraints;reduce the standard deviations of variables that need to be controlled(e.g.product quantity,quality );de-bottlenecking the process;and,maximize profitability or lower cost(e.g.energy savings, improve hot metal content).These objectives may be prioritized in this order,but can vary and are very difficult to achieve optimally through conventional control.A multivariable predictive controller solution,along with its extensive inferential sensor and built-in optimizer,provides online closed loop control and optimization for many interactive metal and mining processes to lower the energy cost,increase throughput,and optimize product quality and yield. Control loop performance is also a key factor to improve iron and steel plant automation and operation result; Honeywell CPM offers vender-independent product which provides monitoring,tuning,modeling of control loop and sustainable loop performance analysis and maintenance solution towards operation stability and energy saving.

Call for Papers:Special Issue on "Recent Advances and New Directions in Switched Control Systems" ——Jounal of Control Theory and Application

Recently, switched control systems have been attracting much attention m the control community because the problems are not only academically challenging for the inherent complexity, but also of practical importance due to its wide ranges of applications in nature, engineering, and social sciences.

Special Issue on Recent Advances and New Directions in Switched Control Systems

As guest editors, it is our great honor to bring this special issue of the Journal of Control Theory and Applications devoted to Recent Advances and New Directions in Switched Control Systems.

Optimized model-based control of main mine ventilation air flows with minimized energy consumption

In early 2018,the Boliden Garpenberg operation implemented an optimized control strategy as an addition to the existing ventilation on demand system.The purpose of the strategy is to further minimize energy use for main and booster fans,whilst also fulfilling airflow setpoints without violating constraints such as min/max differential pressure over fans and interaction of air between areas in mines.Using air flow measurements and a dynamical model of the ventilation system,a mine-wide coordination control of fans can be carried out.The numerical model is data driven and derived from historical operational data or step changes experiments.This makes both initial deployment and lifetime model maintenance,as the mine evolves,a comparably easy operation.The control has been proven to operate in a stable manner over long periods without having to re-calibrate the model.Results prove a 40%decrease in energy use for the fans involved and a greater controllability of air flow.Moreover,a 15%decrease of the total air flow into the mine will give additional proportional heating savings during winter periods.All in all,the multivariable controller shows a correlation between production in the mine and the ventilation system performance superior to all of its predecessors.

Overview of Advanced Control Strategies for Electric Machines

Control strategies play a key role for operation of electric machines,which would directly affect the whole system performance.In fact,different control strategies have been executed and explored for electric machines,which bring great impacts to industrial development and human society.This paper investigates and discusses the advantages control strategies for electric machines,including the field oriented control(FOC),direct torque control(DTC),finite control set model predictive control(FCS-MPC),sensorless control,and fault tolerant control(FTC).The corresponding control principles,control targets,fundamental approaches,advanced approaches,methodologies,merits and shortcomings are revealed and analyzed in detail.

基于TDC和ARM的高精度北斗授时系统设计

针对已有的电力、通信等授时系统精度较低的问题,设计了一种基于TDC和ARM的高精度北斗授时系统。该系统以STM32F2xx/STM32F4xx芯片作为核心信号处理器,对比标准1PPS与1PPS的相位差,结合TDC对时间延时的高精度测量技术,采用PID算法调整本地OCXO压控值,使OCXO输出稳定的时钟频率,进而达到高精度授时目的。目前,北斗接收机授时精度一般优于50ns,而授时精度低于20ns即达到高精度授时标准。实验结果表明,该方法授时精度能达到13ns,能够实现北斗高精度授时。

ADVANCEMENT OF TUBERCULOSIS CONTROL AND RESEARCH IN CHINA

Since the founding of new China, the status of tuberculosis (TB) has been greatly improved through effective control measures and application of research achievements. In the early 1950s, more than 90% of adults in Shanghai and Tianjin had been infected. Tuberculosis ranked the first among the leading causes of death in big cities and the mortality rate reached 169-259/100000 in Beijing, Shanghai, Tianjin, Guangzhou and Hangzhou. The estimated prevalence of tuberculosis in the urban area (3500/100000) was higher than that in the countryside (1500/100000).

Editorial Special issue on recent advances in hybrid control theory and applications

The demands of a rapidly advancing technology for faster and more accurate controllers have always had a strong influence on the progress of automatic control theory. In recent years control problems have been arising with increasing frequency in widely different areas, which cannot be addressed using conventional control techniques. The principal reason for this is the fact that a highly competitive economy is forcing systems to operate in regimes where presence of large uncertainties as well as nonlinearities plays a major role. For instance, the design of unmanned aircraft, unmanned underwater vehicles, and automatic driving systems for automobiles are all typical examples of such difficult control problems. The large uncertainties concerning the plant may arise due to large external disturbances, large parameter variations because of faults in the system, or failure of some of the subsystems. In such cases, the controller has to determine the specific situation that exists at any instant, and take the appropriate control action. Accomplishing this rapidly, accurately, and in a stable fashion is the objective of control design.

Editorial for Special Issue on Intelligent Control and Computing in Advanced Robotics

Artificial intelligence （AI）, robotics, and intelligent systems are increasingly penetrating our society. With recent advances in AI, especially in intelligent control and computing, robots are gaining the ability to learn, make decisions, and operate in ways much similar to humans.To a large extent.

Call for Papers

Recently,switched control systems have been attracting much attention in the control community because the problems are not only academically challenging for the inherent complexity, but also of practical importance due to its wide ranges of applications in nature,engineering,and social sciences. For survival,natural biological systems switch in strategies in accordance to environmental changes. For improved perfor-mance , switching has been extensively utilized/exploited in many engineering systems such as electronics, power systems, and traffic control,among others.In addition,switched behaviors exhibit in social systems.

Bioreactor control systems in the biopharmaceutical industry:a critical perspective

Industrial-scale bioprocessing underpins much of the production of pharmaceuticals,nutraceuticals,food,and beverage processing industries of the modern world.The proftability of these processes increasingly leverages the economies of scale and scope that are critically dependent on the product yields,titers,and productivity.Most of the processes are controlled using classical control approaches and represent over 90%of the industrial controls used in bioprocessing industries.However,with the advances in the production processes,especially in the biopharmaceutical and nutraceutical industries,monitoring and control of bioprocesses such as fermentations with GMO organisms,and downstream processing has become increasingly complex and the inadequacies of the classical and some of the modern control systems techniques is becoming apparent.Therefore,with increasing research complexity,nonlinearity,and digitization in process,there has been a critical need for advanced process control that is more efective,and easier process intensifcation and product yield(both by quality and quantity)can be achieved.In this review,industrial aspects of a process and automation along with various commercial control strategies have been extensively discussed to give an insight into the future prospects of industrial development and possible new strategies for process control and automation with a special focus on the biopharmaceutical industry.

Improved Synergetic Current Control for Grid-connected Microgrids and Distributed Generation Systems

This paper presents the development of improved synergetic current control for the injected current of an inverter in the grid-connected microgrid and the distributed generation system(DGS). This paper introduces new formulas of the macro-variable functions for integral synergetic control(SC)and integral fast terminal SC, which both have an integral term to guarantee zero steady-state error. The proposed integral SC and integral fast terminal SC achieve a seamless performance such as the fast convergence, minimal overshoot, zero steady-state error, and chattering-free operation. To demonstrate the meritorious performance of the proposed scheme for injected current control, it is compared with the performance of a proportional-integral(PI) controller and advanced exponential sliding mode control(SMC). Finally, the practicality of the proposed scheme is justified by experimental results obtained through rapid control prototyping(RCP) using the dSPACESCALEXIO platform.

Power Electronics:The Enabling Technology for Renewable Energy Integration

The markedly increased integration of renewable energy in the power grid is of significance in the transition to a sustainable energy future.The grid integration of renewables will be continuously enhanced in the future.According to the International Renewable Energy Agency(IRENA),renewable technology is the main pathway to reach zero carbon dioxide(CO_(2))emissions by 2060.Power electronics have played and will continue to play a significant role in this energy transition by providing efficient electrical energy conversion,distribution,transmission,and utilization.Consequently,the development of power electronics technologies,i.e.,new semiconductor devices,flexible converters,and advanced control schemes,is promoted extensively across the globe.Among various renewables,wind energy and photovoltaic(PV)are the most widely used,and accordingly these are explored in this paper to demonstrate the role of power electronics.The development of renewable energies and the demands of power electronics are reviewed first.Then,the power conversion and control technologies as well as grid codes for wind and PV systems are discussed.Future trends in terms of power semiconductors,reliability,advanced control,grid-forming operation,and security issues for largescale grid integration of renewables,and intelligent and full user engagement are presented at the end.

Cyber security of railway cyber-physical system(CPS)-A risk management methodology

Along with the increasing application of different cyber-physical systems(CPSs)to connect various components in the rail industry,rising connectivity through communication technologies has also introduced cyber threats against rail-CPSs,causing failures with huge consequences.Implementations of rail-CPSs demand proactive identification,clear-cut definition,and proper handling of their cyber security risks.In this paper,we formulate a risk management methodology for cyber security in rail-CPSs and conduct a retrospective case study on the Advanced Train Control System(ATCS)that has been deployed in many U.S.freight railways.The methodology provides two alternative approaches to fill knowledge gaps in contingency preparation,threat prevention,consequence analysis,and security risk mitigation.In the case study,we demonstrate two cyber threats of ATCS,using attack sequence modeling and consequence analysis,and provide recommendations for risk mitigation.By practicing the methodology with the case study,this work can be used as a general reference to conduct cyber risk management and cyber-robustness evaluations for other existing systems.