Approximate Dynamic Programming for Self-Learning Control

This paper introduces a self-learning control approach based on approximate dynamic programming. Dynamic programming was introduced by Bellman in the 1950's for solving optimal control problems of nonlinear dynamical systems. Due to its high computational complexity, the applications of dynamic programming have been limited to simple and small problems. The key step in finding approximate solutions to dynamic programming is to estimate the performance index in dynamic programming. The optimal control signal can then be determined by minimizing (or maximizing) the performance index. Artificial neural networks are very efficient tools in representing the performance index in dynamic programming. This paper assumes the use of neural networks for estimating the performance index in dynamic programming and for generating optimal control signals, thus to achieve optimal control through self-learning.

A self-learning human-machine cooperative control method based on driver intention recognition

Human-machine cooperative control has become an important area of intelligent driving,where driver intention recognition and dynamic control authority allocation are key factors for improving the performance of cooperative decision-making and control.In this paper,an online learning method is proposed for human-machine cooperative control,which introduces a priority control parameter in the reward function to achieve optimal allocation of control authority under different driver intentions and driving safety conditions.Firstly,a two-layer LSTM-based sequence prediction algorithm is proposed to recognise the driver's lane change(LC)intention for human-machine cooperative steering control.Secondly,an online reinforcement learning method is developed for optimising the steering authority to reduce driver workload and improve driving safety.The driver-in-the-loop simulation results show that our method can accurately predict the driver's LC intention in cooperative driving and effectively compensate for the driver's non-optimal driving actions.The experimental results on a real intelligent vehicle further demonstrate the online optimisation capability of the proposed RL-based control authority allocation algorithm and its effectiveness in improving driving safety.

STUDY ON FUZZY SELF-LEARNING CONTROL SYSTEM FOR SHIP STEERING

Fuzzy control has shown success in some application areas and emerged as analternative to some conventional control schemes. There are also some drawbacks in this approach,for example it is hard to justify the choice of fuzzy controller parameters and control rules, andcontrol precision is low, and so on. Fuzzy control is developing towards self-learning and adaptive.The ship steering motion is a nonlinear, coupling, time-delay complicated system. How to control iteffectively is the problem that many scholars are studying. In this paper, based on the repeatedcontrol of the robot, the self-learning arithmetic was worked out. The arithmetic was realized infuzzy logic way and used in cargo steering. It is the first time for the arithmetic to be used incargo steering. Our simulation results show that the arithmetic is effective and has severalpotential advantages over conventional fuzzy control. This work lays a foundation in modeling andanalyzing the fuzzy learning control system.

SELF-LEARNING FUZZY CONTROL RULES USING GENETIC ALGORITHMS

This papcr presents a new genetic algorithms(GAs)-based method for self-learniag fuzzy control rules. An improved GA is used to learn to optimally select the fuzzy membership functions of the linguistic labels in the condition portion of each rule, and to automatically generate fuzzy control actions under each condition. The dynamics of the controlled system is unknown to the GA. The only information for evaluating performance is a failure signal indicating that the controlled system is out of control. We compare its performance with that of other learning methods for the same problem. We also examine the ability of the algorithm to adapt to changing conditions. Simulation results show that such an approach for self-learning fuzzy control rules is both effective and robust.

Self-learning Fuzzy Controllers Based On a Real-time Reinforcement Genetic Algorithm

This paper presents a novel method for constructing fuzzy controllers based on a real time reinforcement genetic algorithm. This methodology introduces the real-time learning capability of neural networks into globally searching process of genetic algorithm, aiming to enhance the convergence rate and real-time learning ability of genetic algorithm, which is then used to construct fuzzy controllers for complex dynamic systems without any knowledge about system dynamics and prior control experience. The cart-pole system is employed as a test bed to demonstrate the effectiveness of the proposed control scheme, and the robustness of the acquired fuzzy controller with comparable result.

Neuron self-learning PSD control for backside width of weld pool in pulsed GTAW with wire filler

In this paper, the weld pool shape control by intelligent strategy was studied. A neuron self-learning PSD controller for backside width of weld pool in pulsed GTAW with wire filler was designed. The PSD control arithmetic was analyzed, simulating experiment by MATLAB software was done, and the validating experiments on varied heat sink workpiece and varied gap workpiece were successfully implemented. The study results show that the neuron self-learning PSD control method can attain a perfect control effect under different set values and conditions, and is suitable for the welding process with the varied structure and coefficients of control model.

Self-learning fuzzy neural network control for backside width of weld pool in pulsed GTAW with wire filler

The weld pool shape control by intelligent strategy was studied. In order to improve the ability of self-learning and self-adaptation of the ordinary fuzzy control, a self-learning fuzzy neural network controller (FNNC) for backside width of weld pool in pulsed gas tungsten arc welding (GTAW) with wire filler was designed. In FNNC, the fuzzy system was expressed by an equivalence neural network, the membership functions and inference rulers were decided through the learning of the neural network. Then, the FNNC control arithmetic was analyzed, simulating experiment was done, and the validating experiments on varied heat sink workpiece and varied gap workpiece were implemented. The maximum error between the real value and the given one was 0.39mm, the mean error was 0.014mm, and the root-mean-square was 0.14mm. The real backside width was maintained around the given value. The results show that the self-learning fuzzy neural network control strategy can achieve a perfect control effect under different set values and conditions, and is suitable for the welding process with the varied structure and coefficients of control model.

Advanced Functional Electromagnetic Shielding Materials:A Review Based on Micro‑Nano Structure Interface Control of Biomass Cell Walls

Research efforts on electromagnetic interference(EMI)shielding materials have begun to converge on green and sustainable biomass materials.These materials offer numerous advantages such as being lightweight,porous,and hierarchical.Due to their porous nature,interfacial compatibility,and electrical conductivity,biomass materials hold significant potential as EMI shielding materials.Despite concerted efforts on the EMI shielding of biomass materials have been reported,this research area is still relatively new compared to traditional EMI shielding materials.In particular,a more comprehensive study and summary of the factors influencing biomass EMI shielding materials including the pore structure adjustment,preparation process,and micro-control would be valuable.The preparation methods and characteristics of wood,bamboo,cellulose and lignin in EMI shielding field are critically discussed in this paper,and similar biomass EMI materials are summarized and analyzed.The composite methods and fillers of various biomass materials were reviewed.this paper also highlights the mechanism of EMI shielding as well as existing prospects and challenges for development trends in this field.

Passive activity enhances residual control ability in patients with complete spinal cord injury

Patients with complete spinal cord injury retain the potential for volitional muscle activity in muscles located below the spinal injury level.However,because of prolonged inactivity,initial attempts to activate these muscles may not effectively engage any of the remaining neurons in the descending pathway.A previous study unexpectedly found that a brief clinical round of passive activity significantly increased volitional muscle activation,as measured by surface electromyography.In this study,we further explored the effect of passive activity on surface electromyographic signals during volitional control tasks among individuals with complete spinal cord injury.Eleven patients with chronic complete thoracic spinal cord injury were recruited.Surface electromyography data from eight major leg muscles were acquired and compared before and after the passive activity protocol.The results indicated that the passive activity led to an increased number of activated volitional muscles and an increased frequency of activation.Although the cumulative root mean square of surface electromyography amplitude for volitional control of movement showed a slight increase after passive activity,the difference was not statistically significant.These findings suggest that brief passive activity may enhance the ability to initiate volitional muscle activity during surface electromyography tasks and underscore the potential of passive activity for improving residual motor control among patients with motor complete spinal cord injury.

A matrix metalloproteinase-responsive hydrogel system controls angiogenic peptide release for repair of cerebral ischemia/reperfusion injury

Vascular endothelial growth factor and its mimic peptide KLTWQELYQLKYKGI(QK)are widely used as the most potent angiogenic factors for the treatment of multiple ischemic diseases.However,conventional topical drug delivery often results in a burst release of the drug,leading to transient retention(inefficacy)and undesirable diffusion(toxicity)in vivo.Therefore,a drug delivery system that responds to changes in the microenvironment of tissue regeneration and controls vascular endothelial growth factor release is crucial to improve the treatment of ischemic stroke.Matrix metalloproteinase-2(MMP-2)is gradually upregulated after cerebral ischemia.Herein,vascular endothelial growth factor mimic peptide QK was self-assembled with MMP-2-cleaved peptide PLGLAG(TIMP)and customizable peptide amphiphilic(PA)molecules to construct nanofiber hydrogel PA-TIMP-QK.PA-TIMP-QK was found to control the delivery of QK by MMP-2 upregulation after cerebral ischemia/reperfusion and had a similar biological activity with vascular endothelial growth factor in vitro.The results indicated that PA-TIMP-QK promoted neuronal survival,restored local blood circulation,reduced blood-brain barrier permeability,and restored motor function.These findings suggest that the self-assembling nanofiber hydrogel PA-TIMP-QK may provide an intelligent drug delivery system that responds to the microenvironment and promotes regeneration and repair after cerebral ischemia/reperfusion injury.

Application of Fructo-oligosaccharides Quality Control Chart in Milk Powder

The quality control chart of fructo-oligosaccharides in milk powder was established to determine whether the detection process and results are in control state.The content of fructo-oligosaccharides in milk powder control samples was determined by ion chromatography,and the quality control chart of fructo-oligosaccharides was established to analyze the controlled state.The results indicate that the median of the quality control chart is 1613.14 mg/100 g,and the standard deviation is 85.57 mg/100 g.The new quality control points were evaluated and analyzed,and the precision changed,but the mean value did not change.Further F test was conducted to determine that the precision did not change significantly,indicating that the test was in a statistical control state,and the detection process,method and results were controlled.

The chemical composition, quality control and pharmacological effects of Gualou-Xiebai-Banxia Decoction: a review

Gualou-Xiebai-Banxia Decoction(GXBD)is a traditional Chinese herbal formula including four traditional Chinese medicines:Gualou(Trichosanthis Fructus,TF),Xiebai(Allii Macrostemonis Bulbus,AMB),Banxia(Pinelliae Rhizoma,PR)and yellow wine.It is a classical therapy for chest stuffiness and pain syndrome and is widely used in the clinical treatment of coronary heart disease.It also shows significant therapeutic effects on pulmonary heart disease,hyperlipidemia,and arrhythmia.This study conducted a literature review and collected information on GXBD from databases such as PubMed,Web of Science,China National Knowledge Infrastructure,and ScienceDirect.The result indicated that the main active ingredients of GXBD are steroids,flavonoids,terpenoids,alkaloids,amino acids,and organic acids.Trigonelline,macrostemonoside and cucurbitacin B can provide reference for its quality control.GXBD may exert therapeutic effects on coronary heart disease through AMPK,PI3K-AKT,oxLDL,VEGF,and NF-κB signal pathways.This review provides a comprehensive analysis and summary of the chemical composition and in vivo metabolism of three traditional Chinese medicines(TF,AMB,and PR),along with an evaluation of the chemical composition,quality control,pharmacological effects,and clinical application of GXBD.Based on these,areas requiring further research on GXBD have been proposed to provide a reference for its further development and new drug research.

Optimizing risk management for post-amputation wound complications in diabetic patients: Focus on glycemic and immunosuppressive control

This study highlights the importance of identifying and addressing risk factors associated with wound complications following transtibial amputation in diabetic patients.These amputations,often necessitated by severe diabetic foot ulcers,carry significant risks of postoperative complications such as infection and delayed wound healing.Elevated hemoglobin A1c levels,indicative of poor glycemic control,and a history of kidney transplantation,due to required immunosuppressive therapy,are key factors influencing these outcomes.This paper emphasizes the need for enhanced glycemic management and personalized postoperative care,particularly for immunocompromised individuals,to minimize complications and improve patient prognosis.Future research should focus on prospective studies to validate targeted interventions and optimize care strategies,ultimately aiming to reduce the healthcare burden associated with diabetic foot complications.

Two-layer networked learning control using self-learning fuzzy control algorithms

Since the existing single-layer networked control systems have some inherent limitations and cannot effectively handle the problems associated with unreliable networks, a novel two-layer networked learning control system (NLCS) is proposed in this paper. Its lower layer has a number of local controllers that are operated independently, and its upper layer has a learning agent that communicates with the independent local controllers in the lower layer. To implement such a system, a packet-discard strategy is firstly developed to deal with network-induced delay and data packet loss. A cubic spline interpolator is then employed to compensate the lost data. Finally, the output of the learning agent based on a novel radial basis function neural network (RBFNN) is used to update the parameters of fuzzy controllers. A nonlinear heating, ventilation and air-conditioning (HVAC) system is used to demonstrate the feasibility and effectiveness of the proposed system.

Self-Learning and Its Application to Laminar Cooling Model of Hot Rolled Strip

The mathematical model for online controlling hot rolled steel cooling on run-out table （ROT for abbreviation） was analyzed, and water cooling is found to be the main cooling mode for hot rolled steel. The calculation of the drop in strip temperature by both water cooling and air cooling is summed up to obtain the change of heat transfer coefficient. It is found that the learning coefficient of heat transfer coefficient is the kernel coefficient of coiler temperature control （CTC） model tuning. To decrease the deviation between the calculated steel temperature and the measured one at coiler entrance, a laminar cooling control self-learning strategy is used. Using the data acquired in the field, the results of the self-learning model used in the field were analyzed. The analyzed results show that the self-learning function is effective.

Optimal Constrained Self-learning Battery Sequential Management in Microgrid Via Adaptive Dynamic Programming

This paper concerns a novel optimal self-learning battery sequential control scheme for smart home energy systems. The main idea is to use the adaptive dynamic programming U+0028 ADP U+0029 technique to obtain the optimal battery sequential control iteratively. First, the battery energy management system model is established, where the power efficiency of the battery is considered. Next, considering the power constraints of the battery, a new non-quadratic form performance index function is established, which guarantees that the value of the iterative control law cannot exceed the maximum charging/discharging power of the battery to extend the service life of the battery. Then, the convergence properties of the iterative ADP algorithm are analyzed, which guarantees that the iterative value function and the iterative control law both reach the optimums. Finally, simulation and comparison results are given to illustrate the performance of the presented method. © 2017 Chinese Association of Automation.

Self-Learning of Multivariate Time Series Using Perceptually Important Points

In machine learning,positive-unlabelled(PU)learning is a special case within semi-supervised learning.In positiveunlabelled learning,the training set contains some positive examples and a set of unlabelled examples from both the positive and negative classes.Positive-unlabelled learning has gained attention in many domains,especially in time-series data,in which the obtainment of labelled data is challenging.Examples which originate from the negative class are especially difficult to acquire.Self-learning is a semi-supervised method capable of PU learning in time-series data.In the self-learning approach,observations are individually added from the unlabelled data into the positive class until a stopping criterion is reached.The model is retrained after each addition with the existent labels.The main problem in self-learning is to know when to stop the learning.There are multiple,different stopping criteria in the literature,but they tend to be inaccurate or challenging to apply.This publication proposes a novel stopping criterion,which is called Peak evaluation using perceptually important points,to address this problem for time-series data.Peak evaluation using perceptually important points is exceptional,as it does not have tunable hyperparameters,which makes it easily applicable to an unsupervised setting.Simultaneously,it is flexible as it does not make any assumptions on the balance of the dataset between the positive and the negative class.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Where Have Network-based Self-learning Classes Gone?——Reflections & Expectations on the Employment of Network-based Self-learning Classes

To respond to the further development of college English reforms,many universities employed network-based selflearning classes to aid the traditional classroom teaching,especially in teaching listening,but as time went by,some universities gradually gave them up.The paper intends to reflect on the employment of network-based self-learning listening classes,analyz ing the learning with and without its aid,and meanwhile introduce the need to re-employ it,and discuss how we can improve the network-based self-learning classes to help with students' listening.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.