Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

An intelligent control method based on artificial neural network for numerical flight simulation of the basic finner projectile with pitching maneuver

In this paper,an intelligent control method applying on numerical virtual flight is proposed.The proposed algorithm is verified and evaluated by combining with the case of the basic finner projectile model and shows a good application prospect.Firstly,a numerical virtual flight simulation model based on overlapping dynamic mesh technology is constructed.In order to verify the accuracy of the dynamic grid technology and the calculation of unsteady flow,a numerical simulation of the basic finner projectile without control is carried out.The simulation results are in good agreement with the experiment data which shows that the algorithm used in this paper can also be used in the design and evaluation of the intelligent controller in the numerical virtual flight simulation.Secondly,combined with the real-time control requirements of aerodynamic,attitude and displacement parameters of the projectile during the flight process,the numerical simulations of the basic finner projectile’s pitch channel are carried out under the traditional PID(Proportional-Integral-Derivative)control strategy and the intelligent PID control strategy respectively.The intelligent PID controller based on BP(Back Propagation)neural network can realize online learning and self-optimization of control parameters according to the acquired real-time flight parameters.Compared with the traditional PID controller,the concerned control variable overshoot,rise time,transition time and steady state error and other performance indicators have been greatly improved,and the higher the learning efficiency or the inertia coefficient,the faster the system,the larger the overshoot,and the smaller the stability error.The intelligent control method applying on numerical virtual flight is capable of solving the complicated unsteady motion and flow with the intelligent PID control strategy and has a strong promotion to engineering application.

Intelligent Fractional-Order Controller for SMES Systems in Renewable Energy-Based Microgrid

An autonomous microgrid that runs on renewable energy sources is presented in this article.It has a supercon-ducting magnetic energy storage(SMES)device,wind energy-producing devices,and an energy storage battery.However,because such microgrids are nonlinear and the energy they create varies with time,controlling and managing the energy inside them is a difficult issue.Fractional-order proportional integral(FOPI)controller is recommended for the current research to enhance a standalone microgrid’s energy management and performance.The suggested dedicated control for the SMES comprises two loops:the outer loop,which uses the FOPI to regulate the DC-link voltage,and the inner loop,responsible for regulating the SMES current,is constructed using the intelligent FOPI(iFOPI).The FOPI+iFOPI parameters are best developed using the dandelion optimizer(DO)approach to achieve the optimum performance.The suggested FOPI+iFOPI controller’s performance is contrasted with a conventional PI controller for variations in wind speed and microgrid load.The optimal FOPI+iFOPI controller manages the voltage and frequency of the load.The behavior of the microgrid as a reaction to step changes in load and wind speed was measured using the proposed controller.MATLAB simulations were used to evaluate the recommended system’s performance.The results of the simulations showed that throughout all interruptions,the recommended microgrid provided the load with AC power with a constant amplitude and frequency.In addition,the required load demand was accurately reduced.Furthermore,the microgrid functioned incredibly well despite SMES and varying wind speeds.Results obtained under identical conditions were compared with and without the best FOPI+iFOPI controller.When utilizing the optimal FOPI+iFOPI controller with SMES,it was found that the microgrid performed better than the microgrid without SMES.

Design and Application of Intelligent Control System for Molten Iron Transportation Based on 5G Technology

Molten transport is an important link in the iron and steel enterprise production,involves many complex factors,artificial management is difficult.Therefore,puts forward a kind of molten iron transport wisdom control system based on 5G technology,which mainly contains the intelligent identification tracking system,equipment status collection information acquisition system,locomotive vehicle terminal system,etc.Combined with the analysis of the actual application situation,the system could integrate all the processes and elements of molten iron produc-tion and transportation,realize the integration of operation and management,and also promote the improvement of the turnover efficiency of molten iron tank,reduce the demand for personnel,and reduce the labor cost.

Clinical application of flexible ureteroscopic sheath with controllable intraluminal pressure in treating ureteral stones

Objective:The purpose of the study was to assess the clinical efficacy and safety of a combined perfusion suction platform with pressure feedback control function and an ureteroscopic suction sheath that can measure the ureteropelvic pressure in implementing lithotripsies.Methods:Fifty-two patients who underwent lithotripsy under intelligent monitoring of ureteral intraluminal pressure from June 2016 to January 2018 were retrospectively recruited.The inclusion standard was stone diameter>1.5 cm but<2.5 cm.After the 12/14 Fr suction sheath was placed,manometer interface and suction interface of the sheath were connected to the platform via the pressure sensor and suction tube,respectively.The ureteroscope was connected to the platform perfusion pump,and the crushed stones were aspirated out under negative pressure.Results:According to the location of the stone,21(40.4%)cases were classified as upper ureteral stones,19(36.5%)were midureteral stones,and 12(23.1%)were lower ureteral stones.Forty-seven patients underwent successful primary sheath placement and lithotripsy with a mean operative time of 34.5(standard deviation 18.3)min.Retrograde stone migration did not occur.There were eight patients with hematuria postoperatively.Serious complication was 1.9% with one case of ureteral perforation.Stone clearance was 95.7% at Day 1e2 postoperatively,and 100% at Day 30 postoperatively.Conclusion:Ureteroscopic lithotripsy with intelligent pressure control using our device improved the efficiency of the lithotripsy and rate of stone clearance.The safety of the operation can be ensured.It is worth popularization and application in clinical practice.

Mechanoluminescent-Triboelectric Bimodal Sensors for Self-Powered Sensing and Intelligent Control

Self-powered flexible devices with skin-like multiple sensing ability have attracted great attentions due to their broad applications in the Internet of Things(IoT).Various methods have been proposed to enhance mechano-optic or electric performance of the flexible devices;however,it remains challenging to realize the display and accurate recognition of motion trajectories for intelligent control.Here,we present a fully self-powered mechanoluminescent-triboelectric bimodal sensor based on micronanostructured mechanoluminescent elastomer,which can patterned-display the force trajectories.The deformable liquid metals used as stretchable electrode make the stress transfer stable through overall device to achieve outstanding mechanoluminescence(with a gray value of 107 under a stimulus force as low as 0.3 N and more than 2000 cycles reproducibility).Moreover,a microstructured surface is constructed which endows the resulted composite with significantly improved triboelectric performances(voltage increases from 8 to 24 V).Based on the excellent bimodal sensing performances and durability of the obtained composite,a highly reliable intelligent control system by machine learning has been developed for controlling trolley,providing an approach for advanced visual interaction devices and smart wearable electronics in the future IoT era.

An Intelligent Control Method for the Low-Carbon Operation of Energy-Intensive Equipment

Based on an analysis of the operational control behavior of operation experts on energy-intensive equipment,this paper proposes an intelligent control method for low-carbon operation by combining mechanism analysis with deep learning,linking control and optimization with prediction,and integrating decision-making with control.This method,which consists of setpoint control,self-optimized tuning,and tracking control,ensures that the energy consumption per tonne is as low as possible,while remaining within the target range.An intelligent control system for low-carbon operation is developed by adopting the end-edge-cloud collaboration technology of the Industrial Internet.The system is successfully applied to a fused magnesium furnace and achieves remarkable results in reducing carbon emissions.

Iteration dependent interval based open‐closed‐loop iterative learning control for time varying systems with vector relative degree

For linear time varying(LTV)multiple input multiple output(MIMO)systems with vector relative degree,an open‐closed‐loop iterative learning control(ILC)strategy is developed in this article,where the time interval of operation is iteration dependent.To compensate the missing tracking signal caused by iteration dependent interval,the feedback control is introduced in ILC design.As the tracking signal of many continuous iterations is lost in a certain interval,the feedback control part can employ the tracking signal of current iteration for compensation.Under the assumption that the initial state vibrates around the desired initial state uniformly in mathematical expectation sense,the expectation of ILC tracking error can converge to zero as the number of iteration tends to infinity.Under the circumstance that the initial state varies around the desired initial state with a bound,as the number of iteration tends to infinity,the expectation of ILC tracking error can be driven to a bounded range,whose upper bound is proportional to the fluctuation.It is revealed that the convergence condition is dependent on the feed-forward control gains,while the feedback control can accelerate convergence speed by selecting appropriate feedback control gains.As a special case,the controlled system with integrated high relative degree is also addressed by proposing a simplified iteration dependent interval based open‐closed‐loop ILC method.Finally,the effectiveness of the developed iteration dependent interval based open‐closed‐loop ILC is illustrated by a simulation example with two cases on initial state.

Design of Diversified Intelligent Control System for Energy-saving Optimization of Solar Greenhouse in North China

Intelligent greenhouse can promote the development of modern agriculture, realize the high quality and high yield of crops, and also bring greater economic benefits. In accordance with the climate conditions in northwest China, a set of intelligent control system for diversified environment of solar greenhouse was designed. The system divides the annual greenhouse control into six stages according to the optimal energy saving. It uses modern detection technology to collect the greenhouse environmental temperature, environmental humidity, soil humidity, CO_(2) concentration and illumination parameters under different working modes. It uses programmable logic control technology to realize the data processing of various parameters and the action control of rolling film, wet curtain fan and other actuators. It uses KingView monitoring software to realize the monitoring and manual control of greenhouse environment parameters. The operation results indicate that the control system runs stably and basically meets the control requirements.

A review of artificial intelligence applications in high-speed railway systems

In recent years,the global surge of High-speed Railway(HSR)revolutionized ground transportation,providing secure,comfortable,and punctual services.The next-gen HSR,fueled by emerging services like video surveillance,emergency communication,and real-time scheduling,demands advanced capabilities in real-time perception,automated driving,and digitized services,which accelerate the integration and application of Artificial Intelligence(AI)in the HSR system.This paper first provides a brief overview of AI,covering its origin,evolution,and breakthrough applications.A comprehensive review is then given regarding the most advanced AI technologies and applications in three macro application domains of the HSR system:mechanical manufacturing and electrical control,communication and signal control,and transportation management.The literature is categorized and compared across nine application directions labeled as intelligent manufacturing of trains and key components,forecast of railroad maintenance,optimization of energy consumption in railroads and trains,communication security,communication dependability,channel modeling and estimation,passenger scheduling,traffic flow forecasting,high-speed railway smart platform.Finally,challenges associated with the application of AI are discussed,offering insights for future research directions.

Analysis of Current Research and Future Development Trends of Applying Solar Energy in Street Lighting

In the context of promoting green energy transition and addressing climate change globally,solar energy,as a clean and renewable energy source,has gradually become a hot topic for research.Solar streetlight systems realize energy self-sufficiency and environment-friendly lighting by integrating photovoltaic power generation technology and efficient LED lighting technology.By comprehensively analyzing the current status of the application of solar streetlights at home and abroad,this paper discusses its technical advantages,market penetration,and challenges in its development.In terms of technical characteristics,this paper focuses on analyzing the key technologies such as energy conversion efficiency and intelligent control systems of solar streetlights.

INTELLIGENT VIRTUAL CONTROL:MEASURING INSTRUMENT FROM WHOLE TO PART

A new concept called intelligent virtual control (IVC), which can be drivenby measuring functions, is put forward. This small 'intelligent measurement instrument unit (IMIU)',carrying with functions of instrument, consists of different types of intelligent virtualinstrument (IVI) through individual components together as building blocks and can be displayeddirectly on the computer screen. This is a new concept of measuring instrument, and also animportant breakthrough after virtual instrument (VI). Virtual control makes instrument resourcesobtain further exploitation. It brings about a fundamental change to the design and manufacturingmode. The instrument therefore, can not only be produced directly inside a PC, but the product isinvolved in the 'green product' system. So far, all the present digital instruments will grow to bereplaced by intelligent control with green characteristics.

Mathematical Model and Its Hybrid Dynamic Mechanism in Intelligent Control of Ironmaking

A hybrid dynamic model was proposed, which considered both the hydrokinetic and the chaotic properties of the blast furnace ironmaking process; and great emphasis was put on its mechanism. The new model took the high complexity of the blast furnace as well as the effects of main parameters of the model into account, and the predicted results were in very good agreement with actual data.

Nonlinear Intelligent Flight Control for Quadrotor Unmanned Helicopter

Quadrotor unmanned helicopter is a new popular research platform for unmanned aerial vehicle(UAV),thanks to its simple construction,vertical take-off and landing(VTOL)capability.Here a nonlinear intelligent flight control system is developed for quadrotor unmanned helicopter,including trajectory control loop composed of co-controller and state estimator,and attitude control loop composed of brain emotional learning(BEL)intelligent controller.BEL intelligent controller based on mammalian middle brain is characterized as self-learning capability,model-free and robustness.Simulation results of a small quadrotor unmanned helicopter show that the BEL intelligent controller-based flight control system has faster dynamical responses with higher precision than the traditional controller-based system.

Flight Control System of Unmanned Aerial Vehicle

To date unmanned aerial system(UAS)technologies have attracted more and more attention from countries in the world.Unmanned aerial vehicles(UAVs)play an important role in reconnaissance,surveillance,and target tracking within military and civil fields.Here one briefly introduces the development of UAVs,and reviews its various subsystems including autopilot,ground station,mission planning and management subsystem,navigation system and so on.Furthermore,an overview is provided for advanced design methods of UAVs control system,including the linear feedback control,adaptive and nonlinear control,and intelligent control techniques.Finally,the future of UAVs flight control techniques is forecasted.

RESEARCH ON MECHANICAL MEASUREMENT-ORIENTED INTELLIGENT VIRTUAL CONTROLS

Intelligent virtual control (IVC) is an intelligent measurement instrumentunit with the function of actual measurement instruments, and the unit can be used as basic buildingblock for a variety of more complex virtual measurement instruments on a PC. IVC is a furtheradvancement from virtual instrument (VI), and it fuses the function modules and the controls modulesso that the relationship between the functions and controls of an instrument is imbedded in one ormore units. The design, implementation and optimization methods of IVCs are introduced. The computersoftware representation of IVCs is discussed. An example of an actual VI constructed with thebuilding blocks of IVCs is given.

Non-Minimum Phase Nonlinear System Predictive Control Based on Local Recurrent Neural Networks

After a recursive multi-step-ahead predictor for nonlinear systems based on local recurrent neural networks is introduced, an intelligent FID controller is adopted to correct the errors including identified model errors and accumulated errors produced in the recursive process. Characterized by predictive control, this method can achieve a good control accuracy and has good robustness. A simulation study shows that this control algorithm is very effective.

Application of Fuzzy Logic Controller to Level Control of Twin-Roll Strip Casting

An intelligent fuzzy-PID controller consisting of fuzzy logic controller and PID controller was developed to control the molten steel level of twin-roll strip caster.Additionally,a feedforward differential PID controller was used for stopper position control in order to avoid differential kick.It is proved by simulation that the proposed intelligent controller is able to obtain zero steady state error asymptotically and the control system is robust due to its fuggy behavior of the controller.

Hybrid intelligent control of combustion process for ore-roasting furnace

Because of its synthetic and complex characteristics, the combustion process of the shaft ore-roasting furnace is very difficult to control stably. A hybrid intelligent control approach is developed which consists of two systems： one is a cascade fuzzy control system with a temperature soft-sensor, and the other is a ratio control system for air flow with a compensation model for heating gas flow and air-fuel ratio. This approach combined intelligent control, soft-sensing and fault diagnosis with conventional control. It can adjust both the heating gas flow and the air-fuel ratio in real time. By this way, the difficulty of online measurement of the furnace temperature is solved, the fault ratios during combustion process is decreased, the steady control of the furnace temperature is achieved, and the gas consumption is reduced. The successful application in shaft furnaces of a mineral processing plant in China indicates its effectiveness.

Intelligent coordinated control of power-plant main steam pressure and power output

An intelligent coordinated control strategy has been proposed and successfully applied to a 300MW boiler-turbine unit i. e. Unit 1 of Yuanbaoshan power plant in China. Load following operation of coal-fired boiler-turbine unit in the power plant leads to changes in operating points which result in nonlinear variations of the plant variables and parameters. For the variation of operating condition and slowly varying dynamics, an intelligent control scheme has been developed by combining fuzzy self-tuning with adaptive control and auto-tuning techniques. As there exist strong couplings between control loops of main steam pressure and power output in the unit, a new design for static decoupler aimed at decoupling for setpoints and unmeasured pulverized coal disturbance of the system at the same time is presented. Satisfactory industrial application results show that such a control system has enhanced adaptability and robustness to the complex process, and better control performance and high economic benefit have been obtained.