Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

An intelligent control method based on artificial neural network for numerical flight simulation of the basic finner projectile with pitching maneuver

In this paper,an intelligent control method applying on numerical virtual flight is proposed.The proposed algorithm is verified and evaluated by combining with the case of the basic finner projectile model and shows a good application prospect.Firstly,a numerical virtual flight simulation model based on overlapping dynamic mesh technology is constructed.In order to verify the accuracy of the dynamic grid technology and the calculation of unsteady flow,a numerical simulation of the basic finner projectile without control is carried out.The simulation results are in good agreement with the experiment data which shows that the algorithm used in this paper can also be used in the design and evaluation of the intelligent controller in the numerical virtual flight simulation.Secondly,combined with the real-time control requirements of aerodynamic,attitude and displacement parameters of the projectile during the flight process,the numerical simulations of the basic finner projectile’s pitch channel are carried out under the traditional PID(Proportional-Integral-Derivative)control strategy and the intelligent PID control strategy respectively.The intelligent PID controller based on BP(Back Propagation)neural network can realize online learning and self-optimization of control parameters according to the acquired real-time flight parameters.Compared with the traditional PID controller,the concerned control variable overshoot,rise time,transition time and steady state error and other performance indicators have been greatly improved,and the higher the learning efficiency or the inertia coefficient,the faster the system,the larger the overshoot,and the smaller the stability error.The intelligent control method applying on numerical virtual flight is capable of solving the complicated unsteady motion and flow with the intelligent PID control strategy and has a strong promotion to engineering application.

Intelligent Fractional-Order Controller for SMES Systems in Renewable Energy-Based Microgrid

An autonomous microgrid that runs on renewable energy sources is presented in this article.It has a supercon-ducting magnetic energy storage(SMES)device,wind energy-producing devices,and an energy storage battery.However,because such microgrids are nonlinear and the energy they create varies with time,controlling and managing the energy inside them is a difficult issue.Fractional-order proportional integral(FOPI)controller is recommended for the current research to enhance a standalone microgrid’s energy management and performance.The suggested dedicated control for the SMES comprises two loops:the outer loop,which uses the FOPI to regulate the DC-link voltage,and the inner loop,responsible for regulating the SMES current,is constructed using the intelligent FOPI(iFOPI).The FOPI+iFOPI parameters are best developed using the dandelion optimizer(DO)approach to achieve the optimum performance.The suggested FOPI+iFOPI controller’s performance is contrasted with a conventional PI controller for variations in wind speed and microgrid load.The optimal FOPI+iFOPI controller manages the voltage and frequency of the load.The behavior of the microgrid as a reaction to step changes in load and wind speed was measured using the proposed controller.MATLAB simulations were used to evaluate the recommended system’s performance.The results of the simulations showed that throughout all interruptions,the recommended microgrid provided the load with AC power with a constant amplitude and frequency.In addition,the required load demand was accurately reduced.Furthermore,the microgrid functioned incredibly well despite SMES and varying wind speeds.Results obtained under identical conditions were compared with and without the best FOPI+iFOPI controller.When utilizing the optimal FOPI+iFOPI controller with SMES,it was found that the microgrid performed better than the microgrid without SMES.

Design and Application of Intelligent Control System for Molten Iron Transportation Based on 5G Technology

Molten transport is an important link in the iron and steel enterprise production,involves many complex factors,artificial management is difficult.Therefore,puts forward a kind of molten iron transport wisdom control system based on 5G technology,which mainly contains the intelligent identification tracking system,equipment status collection information acquisition system,locomotive vehicle terminal system,etc.Combined with the analysis of the actual application situation,the system could integrate all the processes and elements of molten iron produc-tion and transportation,realize the integration of operation and management,and also promote the improvement of the turnover efficiency of molten iron tank,reduce the demand for personnel,and reduce the labor cost.

Ultra-precision positioning control technique based on neural network

Due to the non-linearity behavior of the precision positioning system, an accurate mathematical control model is difficult to set up, a novel control method for ultra-precision alignment is presented. This method relies on neural network and alignment marks that are in the form of 100μm pitch gratings. The 0-th order Moire signals＇ intensity and its intensity rate are chosen as input variables of the neural network. The characteristics of the neural network make it possible to perform self-training and self-adjusting so as to achieve automatic precision alignment. A neural network model for precision positioning is set up. The model is composed of three neural layers, i.e. input layer, hidden layer and output layer. Driving signal is obtained by mapping Moire signals＇ intensity and its intensity rate. The experimental results show that neural network control for precision positioning can effectively improve positioning speed with high accuracy. It has the advantages of fast, stable response and good robustness. The device based on neural network can achieve the positioning accuracy of ± 0. 5μm.

INTELLIGENT PRECISION CONTROL IN CNC GRINDING

The development of intelligent control techniques provides powerful means for the control of machine tools. In this paper, a intelligent control technique and an algorithm for precision control of CNC grinding of ceramic chips are introduced. In the process of ceramic chip CNC grinding, the dimension of the chips tends to get larger and the dimensional error to exceed the tolerance as the number of the chips increases which are machined on the same part program. There are many factors leading to the occurrence of the error and the law of error variation is very complicated. With the introduced intelligent self learning error compensation technique, the CNC system can improve the control strategy to compensate the error automatically. The simulational result is also given.

The Application and Design of the Economical Nonlinear Controller

This anticle gives a design method of the economical nonlinear controller. The controller is composed of an expert intelligent coordination controller, a fuzzy prediction controller, a fuzzy feedforward controller, a nonlinear controller and so on. The consistence of a distributed control system based on this controller is also shown briefly.

Precision Point to Point Control of Proportional Valve Controlled Motor with a Time Varying Load

Aim In accordance with the positioning control for valve controlled motor electrohydraulic proportional servo systems driving the static load torque, the positioning performance was studied in the presence of the time? varying deadzone and gain. Methods The large positioning errors caused by the time varying deadzone were significantly reduced by using the dynamic compensation method for the deadzone; and the large overshoot caused by the time varying gain were dramatically reduced by using the three section intelligent control schemes. Results Experimental results demonstrated that the positioning performance of rapid response, high accuracy and smaller or even no overshoot was achieved under a wide variations of load torque. Conclusion The good positioning performance for valve controlled motor servo systems has been achieved in the presence of the time varying deadzone and gain.

Open multilevel hierarchic intelligent control system for eco-industrial system

In allusion to the characteristics of the open complex giant system, an open multilevel hierarchic intelligent control system is established for the eco-industrial system. With the idea of the open engineering system, using the hall for workshop of metasynthetic engineering （HWME）, intelligent control techniques, the expert system and the design of experiments are integrated within the framework of the nonlinear multiobjective decision support system to develop a robust, top-level design specification so as to make the system have the quality of adaptive control, self-organizing, self-learning and robustness. Finally, an illustrative example is given to clarify the effectiveness of the method.

INTELLIGENT VIRTUAL CONTROL:MEASURING INSTRUMENT FROM WHOLE TO PART

A new concept called intelligent virtual control (IVC), which can be drivenby measuring functions, is put forward. This small 'intelligent measurement instrument unit (IMIU)',carrying with functions of instrument, consists of different types of intelligent virtualinstrument (IVI) through individual components together as building blocks and can be displayeddirectly on the computer screen. This is a new concept of measuring instrument, and also animportant breakthrough after virtual instrument (VI). Virtual control makes instrument resourcesobtain further exploitation. It brings about a fundamental change to the design and manufacturingmode. The instrument therefore, can not only be produced directly inside a PC, but the product isinvolved in the 'green product' system. So far, all the present digital instruments will grow to bereplaced by intelligent control with green characteristics.

Study on Intelligent Control and 3D Real-time Distributed Animation Simulation for Super-maneuver Attack of the New Generation Fighter

The super-maneuver flight performance has a very high tactical value, and the development of this tactical value has great significance. A discussion is devoted to the study of intelligent control methods and technologies of real-time distributed 3-dimensional animation simulation for the super-maneuverable attack of new generational fighter in this paper. A flight control system of super-maneuver is reconstructed by adopting three layers BP neural networks of number 3, and the fire/flight coupler is designed by introducing a fuzzy control rule whose universe of discourse and gain are regulated adaptively on the line. Furthermore, a new method of real-time distributed 3-dimensional animation simulation is put forward, and a real-time distributed 3-dimensional animation simulation tool platform is constructed in this paper. The simulation result is lifelike, perceivable directly and useful.

Nonlinear Intelligent Flight Control for Quadrotor Unmanned Helicopter

Quadrotor unmanned helicopter is a new popular research platform for unmanned aerial vehicle(UAV),thanks to its simple construction,vertical take-off and landing(VTOL)capability.Here a nonlinear intelligent flight control system is developed for quadrotor unmanned helicopter,including trajectory control loop composed of co-controller and state estimator,and attitude control loop composed of brain emotional learning(BEL)intelligent controller.BEL intelligent controller based on mammalian middle brain is characterized as self-learning capability,model-free and robustness.Simulation results of a small quadrotor unmanned helicopter show that the BEL intelligent controller-based flight control system has faster dynamical responses with higher precision than the traditional controller-based system.

Mathematical Model and Its Hybrid Dynamic Mechanism in Intelligent Control of Ironmaking

A hybrid dynamic model was proposed, which considered both the hydrokinetic and the chaotic properties of the blast furnace ironmaking process; and great emphasis was put on its mechanism. The new model took the high complexity of the blast furnace as well as the effects of main parameters of the model into account, and the predicted results were in very good agreement with actual data.

Flight Control System of Unmanned Aerial Vehicle

To date unmanned aerial system(UAS)technologies have attracted more and more attention from countries in the world.Unmanned aerial vehicles(UAVs)play an important role in reconnaissance,surveillance,and target tracking within military and civil fields.Here one briefly introduces the development of UAVs,and reviews its various subsystems including autopilot,ground station,mission planning and management subsystem,navigation system and so on.Furthermore,an overview is provided for advanced design methods of UAVs control system,including the linear feedback control,adaptive and nonlinear control,and intelligent control techniques.Finally,the future of UAVs flight control techniques is forecasted.

Mechanoluminescent-Triboelectric Bimodal Sensors for Self-Powered Sensing and Intelligent Control

Self-powered flexible devices with skin-like multiple sensing ability have attracted great attentions due to their broad applications in the Internet of Things(IoT).Various methods have been proposed to enhance mechano-optic or electric performance of the flexible devices;however,it remains challenging to realize the display and accurate recognition of motion trajectories for intelligent control.Here,we present a fully self-powered mechanoluminescent-triboelectric bimodal sensor based on micronanostructured mechanoluminescent elastomer,which can patterned-display the force trajectories.The deformable liquid metals used as stretchable electrode make the stress transfer stable through overall device to achieve outstanding mechanoluminescence(with a gray value of 107 under a stimulus force as low as 0.3 N and more than 2000 cycles reproducibility).Moreover,a microstructured surface is constructed which endows the resulted composite with significantly improved triboelectric performances(voltage increases from 8 to 24 V).Based on the excellent bimodal sensing performances and durability of the obtained composite,a highly reliable intelligent control system by machine learning has been developed for controlling trolley,providing an approach for advanced visual interaction devices and smart wearable electronics in the future IoT era.

RESEARCH ON MECHANICAL MEASUREMENT-ORIENTED INTELLIGENT VIRTUAL CONTROLS

Intelligent virtual control (IVC) is an intelligent measurement instrumentunit with the function of actual measurement instruments, and the unit can be used as basic buildingblock for a variety of more complex virtual measurement instruments on a PC. IVC is a furtheradvancement from virtual instrument (VI), and it fuses the function modules and the controls modulesso that the relationship between the functions and controls of an instrument is imbedded in one ormore units. The design, implementation and optimization methods of IVCs are introduced. The computersoftware representation of IVCs is discussed. An example of an actual VI constructed with thebuilding blocks of IVCs is given.

Clinical application of flexible ureteroscopic sheath with controllable intraluminal pressure in treating ureteral stones

Objective:The purpose of the study was to assess the clinical efficacy and safety of a combined perfusion suction platform with pressure feedback control function and an ureteroscopic suction sheath that can measure the ureteropelvic pressure in implementing lithotripsies.Methods:Fifty-two patients who underwent lithotripsy under intelligent monitoring of ureteral intraluminal pressure from June 2016 to January 2018 were retrospectively recruited.The inclusion standard was stone diameter>1.5 cm but<2.5 cm.After the 12/14 Fr suction sheath was placed,manometer interface and suction interface of the sheath were connected to the platform via the pressure sensor and suction tube,respectively.The ureteroscope was connected to the platform perfusion pump,and the crushed stones were aspirated out under negative pressure.Results:According to the location of the stone,21(40.4%)cases were classified as upper ureteral stones,19(36.5%)were midureteral stones,and 12(23.1%)were lower ureteral stones.Forty-seven patients underwent successful primary sheath placement and lithotripsy with a mean operative time of 34.5(standard deviation 18.3)min.Retrograde stone migration did not occur.There were eight patients with hematuria postoperatively.Serious complication was 1.9% with one case of ureteral perforation.Stone clearance was 95.7% at Day 1e2 postoperatively,and 100% at Day 30 postoperatively.Conclusion:Ureteroscopic lithotripsy with intelligent pressure control using our device improved the efficiency of the lithotripsy and rate of stone clearance.The safety of the operation can be ensured.It is worth popularization and application in clinical practice.

A Task-Resource Joint Management Model with Intelligent Control for Mission-Aware Dispersed Computing

Dispersed computing can link all devices with computing capabilities on a global scale to form a fully decentralized network,which can make full use of idle computing resources.Realizing the overall resource allocation of the dispersed computing system is a significant challenge.In detail,by jointly managing the task requests of external users and the resource allocation of the internal system to achieve dynamic balance,the efficient and stable operation of the system can be guaranteed.In this paper,we first propose a task-resource joint management model,which quantifies the dynamic transformation relationship between the resources consumed by task requests and the resources occupied by the system in dispersed computing.Secondly,to avoid downtime caused by an overload of resources,we introduce intelligent control into the task-resource joint management model.The existence and stability of the positive periodic solution of the model can be obtained by theoretical analysis,which means that the stable operation of dispersed computing can be guaranteed through the intelligent feedback control strategy.Additionally,to improve the system utilization,the task-resource joint management model with bi-directional intelligent control is further explored.Setting control thresholds for the two resources not only reverse restrains the system resource overload,but also carries out positive incentive control when a large number of idle resources appear.The existence and stability of the positive periodic solution of the model are proved theoretically,that is,the model effectively avoids the two extreme cases and ensure the efficient and stable operation of the system.Finally,numerical simulation verifies the correctness and validity of the theoretical results.

Non-Minimum Phase Nonlinear System Predictive Control Based on Local Recurrent Neural Networks

After a recursive multi-step-ahead predictor for nonlinear systems based on local recurrent neural networks is introduced, an intelligent FID controller is adopted to correct the errors including identified model errors and accumulated errors produced in the recursive process. Characterized by predictive control, this method can achieve a good control accuracy and has good robustness. A simulation study shows that this control algorithm is very effective.

Application of Fuzzy Logic Controller to Level Control of Twin-Roll Strip Casting

An intelligent fuzzy-PID controller consisting of fuzzy logic controller and PID controller was developed to control the molten steel level of twin-roll strip caster.Additionally,a feedforward differential PID controller was used for stopper position control in order to avoid differential kick.It is proved by simulation that the proposed intelligent controller is able to obtain zero steady state error asymptotically and the control system is robust due to its fuggy behavior of the controller.