Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Short-term displacement prediction for newly established monitoring slopes based on transfer learning

This study makes a significant progress in addressing the challenges of short-term slope displacement prediction in the Universal Landslide Monitoring Program,an unprecedented disaster mitigation program in China,where lots of newly established monitoring slopes lack sufficient historical deformation data,making it difficult to extract deformation patterns and provide effective predictions which plays a crucial role in the early warning and forecasting of landslide hazards.A slope displacement prediction method based on transfer learning is therefore proposed.Initially,the method transfers the deformation patterns learned from slopes with relatively rich deformation data by a pre-trained model based on a multi-slope integrated dataset to newly established monitoring slopes with limited or even no useful data,thus enabling rapid and efficient predictions for these slopes.Subsequently,as time goes on and monitoring data accumulates,fine-tuning of the pre-trained model for individual slopes can further improve prediction accuracy,enabling continuous optimization of prediction results.A case study indicates that,after being trained on a multi-slope integrated dataset,the TCN-Transformer model can efficiently serve as a pretrained model for displacement prediction at newly established monitoring slopes.The three-day average RMSE is significantly reduced by 34.6%compared to models trained only on individual slope data,and it also successfully predicts the majority of deformation peaks.The fine-tuned model based on accumulated data on the target newly established monitoring slope further reduced the three-day RMSE by 37.2%,demonstrating a considerable predictive accuracy.In conclusion,taking advantage of transfer learning,the proposed slope displacement prediction method effectively utilizes the available data,which enables the rapid deployment and continual refinement of displacement predictions on newly established monitoring slopes.

Continuous glucose monitoring metrics in pregnancy with type 1 diabetes mellitus

Managing diabetes during pregnancy is challenging,given the significant risk it poses for both maternal and foetal health outcomes.While traditional methods involve capillary self-monitoring of blood glucose level monitoring and periodic HbA1c tests,the advent of continuous glucose monitoring(CGM)systems has revolutionized the approach.These devices offer a safe and reliable means of tracking glucose levels in real-time,benefiting both women with diabetes during pregnancy and the healthcare providers.Moreover,CGM systems have shown a low rate of side effects and high feasibility when used in pregnancies complicated by diabetes,especially when paired with continuous subcutaneous insulin infusion pump as hybrid closed loop device.Such a combined approach has been demonstrated to improve overall blood sugar control,lessen the occurrence of preeclampsia and neonatal hypoglycaemia,and minimize the duration of neonatal intensive care unit stays.This paper aims to offer a comprehensive evaluation of CGM metrics specifically tailored for pregnancies impacted by type 1 diabetes mellitus.

A HybridManufacturing ProcessMonitoringMethod Using Stacked Gated Recurrent Unit and Random Forest

This study proposed a new real-time manufacturing process monitoring method to monitor and detect process shifts in manufacturing operations.Since real-time production process monitoring is critical in today’s smart manufacturing.The more robust the monitoring model,the more reliable a process is to be under control.In the past,many researchers have developed real-time monitoring methods to detect process shifts early.However,thesemethods have limitations in detecting process shifts as quickly as possible and handling various data volumes and varieties.In this paper,a robust monitoring model combining Gated Recurrent Unit(GRU)and Random Forest(RF)with Real-Time Contrast(RTC)called GRU-RF-RTC was proposed to detect process shifts rapidly.The effectiveness of the proposed GRU-RF-RTC model is first evaluated using multivariate normal and nonnormal distribution datasets.Then,to prove the applicability of the proposed model in a realmanufacturing setting,the model was evaluated using real-world normal and non-normal problems.The results demonstrate that the proposed GRU-RF-RTC outperforms other methods in detecting process shifts quickly with the lowest average out-of-control run length(ARL1)in all synthesis and real-world problems under normal and non-normal cases.The experiment results on real-world problems highlight the significance of the proposed GRU-RF-RTC model in modern manufacturing process monitoring applications.The result reveals that the proposed method improves the shift detection capability by 42.14%in normal and 43.64%in gamma distribution problems.

Contribution of the MERISE-Type Conceptual Data Model to the Construction of Monitoring and Evaluation Indicators of the Effectiveness of Training in Relation to the Needs of the Labor Market in the Republic of Congo

This study proposes the use of the MERISE conceptual data model to create indicators for monitoring and evaluating the effectiveness of vocational training in the Republic of Congo. The importance of MERISE for structuring and analyzing data is underlined, as it enables the measurement of the adequacy between training and the needs of the labor market. The innovation of the study lies in the adaptation of the MERISE model to the local context, the development of innovative indicators, and the integration of a participatory approach including all relevant stakeholders. Contextual adaptation and local innovation: The study suggests adapting MERISE to the specific context of the Republic of Congo, considering the local particularities of the labor market. Development of innovative indicators and new measurement tools: It proposes creating indicators to assess skills matching and employer satisfaction, which are crucial for evaluating the effectiveness of vocational training. Participatory approach and inclusion of stakeholders: The study emphasizes actively involving training centers, employers, and recruitment agencies in the evaluation process. This participatory approach ensures that the perspectives of all stakeholders are considered, leading to more relevant and practical outcomes. Using the MERISE model allows for: • Rigorous data structuring, organization, and standardization: Clearly defining entities and relationships facilitates data organization and standardization, crucial for effective data analysis. • Facilitation of monitoring, analysis, and relevant indicators: Developing both quantitative and qualitative indicators helps measure the effectiveness of training in relation to the labor market, allowing for a comprehensive evaluation. • Improved communication and common language: By providing a common language for different stakeholders, MERISE enhances communication and collaboration, ensuring that all parties have a shared understanding. The study’s approach and contribution to existing research lie in: • Structured theoretical and practical framework and holistic approach: The study offers a structured framework for data collection and analysis, covering both quantitative and qualitative aspects, thus providing a comprehensive view of the training system. • Reproducible methodology and international comparison: The proposed methodology can be replicated in other contexts, facilitating international comparison and the adoption of best practices. • Extension of knowledge and new perspective: By integrating a participatory approach and developing indicators adapted to local needs, the study extends existing research and offers new perspectives on vocational training evaluation.

Determination of Monitoring Control Value for Concrete Gravity Dam Spatial Deformation Based on POT Model

Deformation can directly reflect the working behavior of the dam,so determining the deformation monitoring control value can effectively monitor the safety of dam operation.The traditional dam deformation monitoring control value only considers the single measuring point.In order to overcome the limitation,this paper presents a new method to determine the monitoring control value for concrete gravity dam based on the deformations of multi-measuring points.A dam’s comprehensive deformation displacement is determined by the measured values at different measuring points on the positive inverted vertical line and the corresponding weight of eachmeasuring point.The projection pursuit method(PPM)combined with the grey wolf optimization(GWO)algorithm is used to determine the weight of each measuring point according to the spatial correlation distribution characteristics of dam deformation.The peaks over threshold(POT)model based on the extreme value theory is adopted to determine the monitoring control value with the obtained dam comprehensive deformation displacement.In addition,the POTmodel is improved with the automatic threshold determinationmethod based on the 3σcriterion in probability theory and the GWO algorithm,which can avoid subjectivity and randomness in determining the threshold.The results of the engineering application show the feasibility and applicability of the proposed method.

Tool Condition Monitoring Based on Nonlinear Output Frequency Response Functions and Multivariate Control Chart

Tool condition monitoring(TCM)is a key technology for intelligent manufacturing.The objective is to monitor the tool operation status and detect tool breakage so that the tool can be changed in time to avoid significant damage to workpieces and reduce manufacturing costs.Recently,an innovative TCM approach based on sensor data modelling and model frequency analysis has been proposed.Different from traditional signal feature-based monitoring,the data from sensors are utilized to build a dynamic process model.Then,the nonlinear output frequency response functions,a concept which extends the linear system frequency response function to the nonlinear case,over the frequency range of the tooth passing frequency of the machining process are extracted to reveal tool health conditions.In order to extend the novel sensor data modelling and model frequency analysis to unsupervised condition monitoring of cutting tools,in the present study,a multivariate control chart is proposed for TCM based on the frequency domain properties of machining processes derived from the innovative sensor data modelling and model frequency analysis.The feature dimension is reduced by principal component analysis first.Then the moving average strategy is exploited to generate monitoring variables and overcome the effects of noises.The milling experiments of titanium alloys are conducted to verify the effectiveness of the proposed approach in detecting excessive flank wear of solid carbide end mills.The results demonstrate the advantages of the new approach over conventional TCM techniques and its potential in industrial applications.

Microseismic monitoring and forecasting of dynamic disasters in underground hydropower projects in southwest China:A review

The underground hydropower projects in Southwest China is characterized by large excavation sizes,high geostresses,complicated geological conditions and multiple construction processes.Various disasters such as collapses,large deformations,rockbursts are frequently encountered,resulting in serious casualties and huge economic losses.This review mainly presents some representative results on microseismic(MS)monitoring and forecasting for disasters in hydropower underground engineering.First,a set of new denoising,spectral analysis,and location methods were developed for better identification and location of MS signals.Then,the tempo-spatial characteristics of MS events were analyzed to understand the relationship between field construction and damages of surrounding rocks.Combined with field construction,geological data,numerical simulation and parametric analysis of MS sources,the focal mechanism of MS events was revealed.A damage constitutive model considering MS fracturing size was put forward and feedback analysis considering the MS damage of underground surrounding rocks was conducted.Next,an MS multi-parameter based risk assessment and early warning method for dynamic disasters were proposed.The technology for control of the damage and deformation of underground surrounding rocks was proposed for underground caverns.Finally,two typical underground powerhouses were selected as case studies.These achievements can provide significant references for prevention and control of dynamic disasters for underground engineering with similar complicated geological conditions.

A blast furnace fault monitoring algorithm with low false alarm rate:Ensemble of greedy dynamic principal component analysis-Gaussian mixture model

The large blast furnace is essential equipment in the process of iron and steel manufacturing. Due to the complex operation process and frequent fluctuations of variables, conventional monitoring methods often bring false alarms. To address the above problem, an ensemble of greedy dynamic principal component analysis-Gaussian mixture model(EGDPCA-GMM) is proposed in this paper. First, PCA-GMM is introduced to deal with the collinearity and the non-Gaussian distribution of blast furnace data.Second, in order to explain the dynamics of data, the greedy algorithm is used to determine the extended variables and their corresponding time lags, so as to avoid introducing unnecessary noise. Then the bagging ensemble is adopted to cooperate with greedy extension to eliminate the randomness brought by the greedy algorithm and further reduce the false alarm rate(FAR) of monitoring results. Finally, the algorithm is applied to the blast furnace of a large iron and steel group in South China to verify performance.Compared with the basic algorithms, the proposed method achieves lowest FAR, while keeping missed alarm rate(MAR) remain stable.

A wireless solution for greenhouse monitoring and control system based on ZigBee technology

With the rapid development of wireless technologies, it is possible for Chinese greenhouses to be equipped with wireless sensor networks due to their low-cost, simplicity and mobility. In the current study, we compared the advantages of ZigBee with other two similar wireless networking protocols, Wi-Fi and Bluetooth, and proposed a wireless solution for green- house monitoring and control system based on ZigBee technology. As an explorative application of ZigBee technology in Chinese greenhouse, it may promote Chinese protected agriculture.

Underground ground control monitoring and interpretation,and numerical modeling, and shield capacity design

Mine or longwall panel layout is a 3D structure with highly non-uniform stress distribution. Recognition of such fact will facilitate underground problem identification/investigation and solving by numerical modeling through proper model construction. Due to its versatility, numerical modeling is the most popular method for ground control design and problem solving. However numerical modeling results require highly experienced professionals to interpret its validity/applicability to actual mining operations due to complicated mining and geological conditions. Underground ground control monitoring is routinely performed to predict roof behavior such as weighting and weighting interval without matching observation of face mining condition while the mining pressures are being monitored, resulting in unrealistic interpretation of the obtained data on mining pressure. The importance of ground control pressure monitoring and simultaneous observation of mining and geological conditions is illustrated by an example of shield leg pressure monitoring and interpretation in an U.S. longwall coal mine: it was found that the roof strata act like a plate, not an individual block of the size of a shield dimension, as commonly assumed by all researchers and shield capacity is not a fixed property for a longwall panel or a mine or a coal seam. A new mechanism on the interaction between shield's hydraulic leg pressure and roof strata for shield loading is proposed.

Nonparametric Control Scheme for Monitoring Phase Ⅱ Nonlinear Profiles with Varied Argument Values

Profile monitoring is used to check the stability of the quality of a product over time when the product quality is best represented by a function at each time point.However,most previous monitoring approaches have not considered that the argument values may vary from profile to profile,which is common in practice.A novel nonparametric control scheme based on profile error is proposed for monitoring nonlinear profiles with varied argument values.The proposed scheme uses the metrics of profile error as the statistics to construct the control charts.More details about the design of this nonparametric scheme are also discussed.The monitoring performance of the combined control scheme is compared with that of alternative nonparametric methods via simulation.Simulation studies show that the combined scheme is effective in detecting parameter error and is sensitive to small shifts in the process.In addition,due to the properties of the charting statistics,the out-of-control signal can provide diagnostic information for the users.Finally,the implementation steps of the proposed monitoring scheme are given and applied for monitoring the blade manufacturing process.With the application in blade manufacturing of aircraft engines,the proposed nonparametric control scheme is effective,interpretable,and easy to apply.

Performance Monitoring and Diagnosis of Multivariable Model Predictive Control Using Statistical Analysis

A statistic-based benchmark was proposed for performance assessment and monitoring of model predic- tive control; the benchmark was straightforward and achievable by recording a set of output data only when the control performance was good according to the user’s selection. Principal component model was built and an auto- regressive moving average filter was identified to monitor the performance; an improved T2 statistic was selected as the performance monitor index. When performance changes were detected, diagnosis was done by model validation using recursive analysis and generalized likelihood ratio (GLR) method. This distinguished the fact that the per- formance change was due to plant model mismatch or due to disturbance term. Simulation was done about a heavy oil fractionator system and good results were obtained. The diagnosis result was helpful for the operator to improve the system performance.

Multivariate Statistical Process Monitoring and Control: Recent Developments and Applications to Chemical Industry

Multivariate statistical process monitoring and control (MSPM&C) methods for chemical process monitoring with statistical projection techniques such as principal component analysis (PCA) and partial least squares (PLS) are surveyed in this paper. The four-step procedure of performing MSPM&C for chemical process, modeling of processes, detecting abnormal events or faults, identifying the variable(s) responsible for the faults and diagnosing the source cause for the abnormal behavior, is analyzed. Several main research directions of MSPM&C reported in the literature are discussed, such as multi-way principal component analysis (MPCA) for batch process, statistical monitoring and control for nonlinear process, dynamic PCA and dynamic PLS, and on-line quality control by inferential models. Industrial applications of MSPM&C to several typical chemical processes, such as chemical reactor, distillation column, polymerization process, petroleum refinery units, are summarized. Finally, some concluding remarks and future considerations are made.

Improved Smith prediction monitoring AGC system based on feedback-assisted iterative learning control

The performance of Smith prediction monitoring automatic gauge control(AGC) system is influenced by model mismatching greatly in strip rolling process. Aiming at this problem, a feedback-assisted iterative learning control strategy, which learned unknown modeling error by using previous control information repeatedly, was introduced into Smith prediction monitoring AGC system. Firstly, conventional Smith predictor and improved Smith predictor with PI-P controller were analyzed. Secondly, on the basis of establishing of feedback-assisted iterative learning control strategy for improved Smith predictor, process control signal update law and control error were deduced, then convergence condition of this strategy was put forward and proved. Finally, after modeling the automatic position control system, the PI-P Smith prediction monitoring AGC system with feedback-assisted iterative learning control was researched through simulation. Simulation results indicate that this system remains stable during model mismatching. The robustness and response of monitoring AGC is improved by development of feedback-assisted iterative learning control strategy for PI-P Smith predictor.

Research on Remote Monitoring and Fault Diagnosis Technology of Numerical Control Machine

Based on the internet technology,it has become possible to complete remote monitoring and fault diagnosis for the numerical control machine.In order to capture the micro-shock signal induced by the incipient fault on the rotating parts,the reso- nance demodulation technology is utilized in the system.As a subsystem of the remote monitoring system,the embedded data acquisi- tion instrument not only integrates the demodulation board but also complete the collection and preprocess of monitoring data from different machines.Furthermore,through connecting to the internet,the data can be transferred to the remote diagnosis center and data reading and writing function can be finished in the database.At the same time,the problem of the IP address floating in the dial-up of web server is solved by the dynamic DNS technology.Finally,the remote diagnosis software developed on the Lab VIEW platform can analyze the monitoring data from manufacturing field.The research results have indicated that the equipment status can be monitored by the system effectively.

A hierarchical architecture of centralized monitoring and controlling system and its high-performance and interoperability protocol

This paper describes a hierarchical architecture and a high-performance and interoperability protocol for centralized monitoring and controlling systems (CMCS) . The protocol we proposed can interoperate different monitoring and controlling systems constructed by different companies, each with different functions and communication protocols. The protocol reduces the amount of traffic and has real-time and high-perfor-mance advantages. The protocol was implemented in CMCS for telecommunication power supply and air-condi-tioner used by the Telecommunication Bureau of Zhejiang Province. This paper deals with the hierarchical architecture and function of CMCS and packet format, command ID, and SDL description of its protocol. We also discuss the properties of the interoperability and performance of the protocol in this paper.

Management of chronic heart failure:Role of home echocardiography in monitoring care programs

AIM:To identify a possible role of home echocardiography for monitoring chronic heart failure(CHF)patients.METHODS:We prospectively investigated 118 patients hospitalized during the last year for CHF who could not easily reach the pertaining District Healthcare Center.The patients were followed up with 2 home management programs:one including clinical and electrocardiographic evaluations and also periodic home echocardiographic examinations(group A),the other including clinical and electrocardiographic evaluations only(group B).RESULTS:At the end of the 18-mo follow-up no significant differences were observed between the 2 groups as regards the primary endpoint:rehospitalization occurred in 4 patients of the group A and in 6 patients of the group B;major cardiovascular events occurred in 2 and in 3 patients,respectively.No significant differences were observed with respect to the secondary endpoints:one vascular event appeared in both the groups,3 cardiovascular deaths occurred in group A and 2 in group B.No significant differences were observed between the 2 groups as regards the composite endpoint of death plus hospitalization.CONCLUSION:Home echocardiography for monitoring of CHF patients does not improve the cardiovascular endpoints.In our CHF patients,a low incidence of vascular events was observed.

Monitoring of Organochlorine Pesticide Residues──The GEMS/Food Program in China

This paper covers the monitoring activities of organochlorine (OCS) pesticide residues in the GEMS/Food program, which China took part in for the first time in 1992. The levels of HCH and DDT were determined in 355 samples of eight main varieties of food , taken from five provinces and cities. The results show that the positive rate of HCH is 69%, of which 99. 44 % is in accord with the National Standards of Food Hygiene of China. As to DDT, the two rates are 42 % and 100 % respectively. The daily intakes of HCH and DDT are 0. 22 mg/ kg.bw and 0. 54 mg/kg. bw respectively, which are far below their corresponding ADIs.Compared with those in 1970s, the OCS residues have decreased significantly. Therefore the monitored varieties of food are safe to people's health

A Multi-Interface Remote Monitoring and Control System Architecture Based on Embedded Server

This paper presents a multi-interface embedded server architecture for remote real-time monitoring system and distributed monitoring applications. In the scheme,an embedded microprocessor( LPC3250 from NXP) is chosen as the CPU of the embedded server with a linux operation system( OS) environment. The embedded server provides multiple interfaces for supporting various application scenarios. The whole network is based on local area network and adopts the Browser / Server( B / S) model. The monitoring and control node is as a browser endpoint and the remote node with an embedded server is as a server endpoint. Users can easily acquire various sensors information through writing Internet protocol address of remote node on the computer browser. Compared with client / server( C / S) mode,B / S model needs less maintain and can be applicable to large user group. In addition,a simple network management protocol( SNMP) is used for management of devices in Internet protocol( IP) networks. The results of the demonstration experiment show that the proposed system gives good support to manage the network from different user terminals and allows the users to better interact with the ambient environment.