Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Vibration Control of A Flexible Marine Riser System Subject to Input Dead Zone and Extraneous Disturbances

An observer-based adaptive backstepping boundary control is proposed for vibration control of flexible offshore riser systems with unknown nonlinear input dead zone and uncertain environmental disturbances.The control algorithm can update the control law online through real-time data to make the controller adapt to the environment and improve the control precision.Specifically,based on the adaptive backstepping framework,virtual control laws and Lyapunov functions are designed for each subsystem.Three direction interference observers are designed to track the timevarying boundary disturbance.On this basis,the inverse of the dead zone and linear state transformation are used to compensate for the original system and eliminate the adverse effects of the dead zone.In addition,the stability of the closed-loop system is proven by Lyapunov stability theory.All the system states are bounded,and the vibration offset of the riser converges to a small area of the initial position.Finally,four examples of flexible marine risers are simulated in MATLAB to verify the effectiveness of the proposed controller.

An Integrated Control Framework for Torque Vectoring and Active Suspension System

Four-wheel independently driven electric vehicles(FWID-EV)endow a flexible and scalable control framework to improve vehicle performance.This paper integrates the torque vectoring and active suspension system(ASS)to enhance the vehicle’s longitudinal and vertical motion control performance.While the nonlinear characteristic of the tire model leads to a relatively heavier computational burden.To facilitate the controller design and ease the load,a half-vehicle dynamics system is built and simplified to the linear-time-varying(LTV)model.Then a model predictive controller is developed by formulating the objective function by comprehensively considering the safety,energy-saving and comfort requirements.The in-wheel motor efficiency and the power loss of tire slip are treated as optimization indices in this work to reduce energy consumption.Finally,the effectiveness of the proposed controller is verified through the rapid-control-prototype(RCP)test.The results demonstrate the enhancement of the energy-saving as well as comfort on the basis of vehicle stability.

Mitochondrial dysfunction and quality control lie at the heart of subarachnoid hemorrhage

The dramatic increase in intracranial pressure after subarachnoid hemorrhage leads to a decrease in cerebral perfusion pressure and a reduction in cerebral blood flow.Mitochondria are directly affected by direct factors such as ischemia,hypoxia,excitotoxicity,and toxicity of free hemoglobin and its degradation products,which trigger mitochondrial dysfunction.Dysfunctional mitochondria release large amounts of reactive oxygen species,inflammatory mediators,and apoptotic proteins that activate apoptotic pathways,further damaging cells.In response to this array of damage,cells have adopted multiple mitochondrial quality control mechanisms through evolution,including mitochondrial protein quality control,mitochondrial dynamics,mitophagy,mitochondrial biogenesis,and intercellular mitochondrial transfer,to maintain mitochondrial homeostasis under pathological conditions.Specific interventions targeting mitochondrial quality control mechanisms have emerged as promising therapeutic strategies for subarachnoid hemorrhage.This review provides an overview of recent research advances in mitochondrial pathophysiological processes after subarachnoid hemorrhage,particularly mitochondrial quality control mechanisms.It also presents potential therapeutic strategies to target mitochondrial quality control in subarachnoid hemorrhage.

A 16-bit 18-MSPS flash-assisted SAR ADC with hybrid synchronous and asynchronous control logic

This paper presents a 16-bit,18-MSPS(million samples per second)flash-assisted successive-approximation-register(SAR)analog-to-digital converter(ADC)utilizing hybrid synchronous and asynchronous(HYSAS)timing control logic based on an on-chip delay-locked loop(DLL).The HYSAS scheme can provide a longer settling time for the capacitive digital-to-analog converter(CDAC)than the synchronous and asynchronous SAR ADC.Therefore,the issue of incomplete settling or ringing in the DAC voltage for cases of either on-chip or off-chip reference voltage can be solved to a large extent.In addition,the fore-ground calibration of the CDAC’s mismatch is performed with a finite-impulse-response bandpass filter(FIR-BPF)based least-mean-square(LMS)algorithm in an off-chip FPGA(field programmable gate array).Fabricated in 40-nm CMOS process,the proto-type ADC achieves 94.02-dB spurious-free dynamic range(SFDR),and 75.98-dB signal-to-noise-and-distortion ratio(SNDR)for a 2.88-MHz input under 18-MSPS sampling rate.

Path Planning and Tracking Control for Parking via Soft Actor-Critic Under Non-Ideal Scenarios

Parking in a small parking lot within limited space poses a difficult task. It often leads to deviations between the final parking posture and the target posture. These deviations can lead to partial occupancy of adjacent parking lots, which poses a safety threat to vehicles parked in these parking lots. However, previous studies have not addressed this issue. In this paper, we aim to evaluate the impact of parking deviation of existing vehicles next to the target parking lot(PDEVNTPL) on the automatic ego vehicle(AEV) parking, in terms of safety, comfort, accuracy, and efficiency of parking. A segmented parking training framework(SPTF) based on soft actor-critic(SAC) is proposed to improve parking performance. In the proposed method, the SAC algorithm incorporates strategy entropy into the objective function, to enable the AEV to learn parking strategies based on a more comprehensive understanding of the environment. Additionally, the SPTF simplifies complex parking tasks to maintain the high performance of deep reinforcement learning(DRL). The experimental results reveal that the PDEVNTPL has a detrimental influence on the AEV parking in terms of safety, accuracy, and comfort, leading to reductions of more than 27%, 54%, and 26%respectively. However, the SAC-based SPTF effectively mitigates this impact, resulting in a considerable increase in the parking success rate from 71% to 93%. Furthermore, the heading angle deviation is significantly reduced from 2.25 degrees to 0.43degrees.

Quantization and Event-Triggered Policy Design for Encrypted Networked Control

This paper proposes a novel event-driven encrypted control framework for linear networked control systems(NCSs),which relies on two modified uniform quantization policies,the Paillier cryptosystem,and an event-triggered strategy.Due to the fact that only integers can work in the Pailler cryptosystem,both the real-valued control gain and system state need to be first quantized before encryption.This is dramatically different from the existing quantized control methods,where only the quantization of a single value,e.g.,the control input or the system state,is considered.To handle this issue,static and dynamic quantization policies are presented,which achieve the desired integer conversions and guarantee asymptotic convergence of the quantized system state to the equilibrium.Then,the quantized system state is encrypted and sent to the controller when the triggering condition,specified by a state-based event-triggered strategy,is satisfied.By doing so,not only the security and confidentiality of data transmitted over the communication network are protected,but also the ciphertext expansion phenomenon can be relieved.Additionally,by tactfully designing the quantization sensitivities and triggering error,the proposed event-driven encrypted control framework ensures the asymptotic stability of the overall closedloop system.Finally,a simulation example of the secure motion control for an inverted pendulum cart system is presented to evaluate the effectiveness of the theoretical results.

Versatile and controlled quantum teleportation network

A quantum teleportation network involving multiple users is essential for future quantum internet.So far,controlled quantum teleportation has been demonstrated in a three-user network.However,versatile and controlled quantum teleportation network involving more users is in demand,which satisfies different combinations of users for practical requirements.Here we propose a highly versatile and controlled teleportation network that can switch among various combinations of different users.We use a single continuous-variable six-partite Greenberger-Horne-Zeilinger(GHZ)state to realize such a task by choosing the different measurement and feedback operations.The controlled teleportation network,which includes one sub-network,two sub-networks and three sub-networks,can be realized for different application of user combinations.Furthermore,the coherent feedback control(CFC)can manipulate and improve the teleportation performance.Our approach is flexible and scalable,and would provide a versatile platform for demonstrations of complex quantum communication and quantum computing protocols.

Nonlinear robust adaptive control for bidirectional stabilization system of all-electric tank with unknown actuator backlash compensation and disturbance estimation

Since backlash nonlinearity is inevitably existing in actuators for bidirectional stabilization system of allelectric tank,it behaves more drastically in high maneuvering environments.In this work,the accurate tracking control for bidirectional stabilization system of moving all-electric tank with actuator backlash and unmodeled disturbance is solved.By utilizing the smooth adaptive backlash inverse model,a nonlinear robust adaptive feedback control scheme is presented.The unknown parameters and unmodelled disturbance are addressed separately through the derived parametric adaptive function and the continuous nonlinear robust term.Because the unknown backlash parameters are updated via adaptive function and the backlash effect can be suppressed successfully by inverse operation,which ensures the system stability.Meanwhile,the system disturbance in the high maneuverable environment can be estimated with the constructed adaptive law online improving the engineering practicality.Finally,Lyapunov-based analysis proves that the developed controller can ensure the tracking error asymptotically converges to zero even with unmodeled disturbance and unknown actuator backlash.Contrast co-simulations and experiments illustrate the advantages of the proposed approach.

Blood Pressure Profile and Glycemic Control of Type 2 Diabetics and Hypertensives at the Yalgado Ouedraogo University Hospital: A Review of 116 Cases

Objective: The association hypertension and diabetes is important. The two pathologies may influence each other. The aim was to study the correlation between glycemic control and blood pressure control and to determine the factors associated with blood pressure control. Methodology: This was a descriptive cross-sectional study with an analytical focus over 7 months. Patients were recruited as outpatients and all underwent ambulatory blood pressure measure, glycated hemoglobin and creatinine measurements, and assessment of compliance with treatment. Results: During this period 116 patients were collected. The predominance was female 69%. The mean age of the patients was 62 ± 7 years with a peak between 60 and 70 years. The average age of hypertension was 12 years and that of diabetes 6 1/2 years. The most frequently associated cardiovascular risk factor was a sedentary lifestyle (71.5%) after age. 57.8% of patients were not controlled at the office, with a predominance of systolic hypertension (58.2%). 61.6% of patients were controlled by ambulatory blood pressure measure, a rate of 47.8% of white coat hypertension. Glycemic control was observed in 42.2% of cases and 87% of patients had good renal function (glomerular filter rate ≥ 60 ml/mn). Therapeutic compliance was good in 53.4% of cases and dual therapy was the most used therapeutic modality 44.8% (52 patients) followed by triple therapy. The factors associated with poor blood pressure control were glycemic imbalance, non-compliance and monotherapy. Dual therapy had a protective effect. Conclusion: The association of hypertension and type 2 diabetes is frequent. The risk of occurrence increases with age. Ambulatory blood pressure measure is the best method to assess blood pressure control. Optimization of blood pressure control should also include optimization of glycemic control.

Command filtered integrated estimation guidance and control for strapdown missiles with circular field of view

In this paper,an integrated estimation guidance and control(IEGC)system is designed based on the command filtered backstepping approach for circular field-of-view(FOV)strapdown missiles.The threedimensional integrated estimation guidance and control nonlinear model with limited actuator deflection angle is established considering the seeker's FOV constraint.The boundary time-varying integral barrier Lyapunov function(IBLF)is employed in backstepping design to constrain the body line-of-sight(BLOS)in IEGC system to fit a circular FOV.Then,the nonlinear adaptive controller is designed to estimate the changing aerodynamic parameters.The generalized extended state observer(GESO)is designed to estimate the acceleration of the maneuvering targets and the unmatched time-varying disturbances for improving tracking accuracy.Furthermore,the command filters are used to solve the"differential expansion"problem during the backstepping design.The Lyapunov theory is used to prove the stability of the overall closed-loop IEGC system.Finally,the simulation results validate the integrated system's effectiveness,achieving high accuracy strikes against maneuvering targets.

The prediction of projectile-target intersection for moving tank based on adaptive robust constraint-following control and interval uncertainty analysis

To improve the hit probability of tank at high speed,a prediction method of projectile-target intersection based on adaptive robust constraint-following control and interval uncertainty analysis is proposed.The method proposed provides a novel way to predict the impact point of projectile for moving tank.First,bidirectional stability constraints and stability constraint-following error are constructed using the Udwadia-Kalaba theory,and an adaptive robust constraint-following controller is designed considering uncertainties.Second,the exterior ballistic ordinary differential equation with uncertainties is integrated into the controller,and the pointing control of stability system is extended to the impact-point control of projectile.Third,based on the interval uncertainty analysis method combining Chebyshev polynomial expansion and affine arithmetic,a prediction method of projectile-target intersection is proposed.Finally,the co-simulation experiment is performed by establishing the multi-body system dynamic model of tank and mathematical model of control system.The results demonstrate that the prediction method of projectile-target intersection based on uncertainty analysis can effectively decrease the uncertainties of system,improve the prediction accuracy,and increase the hit probability.The adaptive robust constraint-following control can effectively restrain the uncertainties caused by road excitation and model error.

On-Chip Micro Temperature Controllers Based on Freestanding Thermoelectric Nano Films for Low-Power Electronics

Multidimensional integration and multifunctional com-ponent assembly have been greatly explored in recent years to extend Moore’s Law of modern microelectronics.However,this inevitably exac-erbates the inhomogeneity of temperature distribution in microsystems,making precise temperature control for electronic components extremely challenging.Herein,we report an on-chip micro temperature controller including a pair of thermoelectric legs with a total area of 50×50μm^(2),which are fabricated from dense and flat freestanding Bi2Te3-based ther-moelectric nano films deposited on a newly developed nano graphene oxide membrane substrate.Its tunable equivalent thermal resistance is controlled by electrical currents to achieve energy-efficient temperature control for low-power electronics.A large cooling temperature difference of 44.5 K at 380 K is achieved with a power consumption of only 445μW,resulting in an ultrahigh temperature control capability over 100 K mW^(-1).Moreover,an ultra-fast cooling rate exceeding 2000 K s^(-1) and excellent reliability of up to 1 million cycles are observed.Our proposed on-chip temperature controller is expected to enable further miniaturization and multifunctional integration on a single chip for microelectronics.

Real-Time Co-optimization of Gear Shifting and Engine Torque for Predictive Cruise Control of Heavy-Duty Trucks

Fuel consumption is one of the main concerns for heavy-duty trucks.Predictive cruise control(PCC)provides an intriguing opportunity to reduce fuel consumption by using the upcoming road information.In this study,a real-time implementable PCC,which simultaneously optimizes engine torque and gear shifting,is proposed for heavy-duty trucks.To minimize fuel consumption,the problem of the PCC is formulated as a nonlinear model predictive control(MPC),in which the upcoming road elevation information is used.Finding the solution of the nonlinear MPC is time consuming;thus,a real-time implementable solver is developed based on Pontryagin’s maximum principle and indirect shooting method.Dynamic programming(DP)algorithm,as a global optimization algorithm,is used as a performance benchmark for the proposed solver.Simulation,hardware-in-the-loop and real-truck experiments are conducted to verify the performance of the proposed controller.The results demonstrate that the MPC-based solution performs nearly as well as the DP-based solution,with less than 1%deviation for testing roads.Moreover,the proposed co-optimization controller is implementable in a real-truck,and the proposed MPC-based PCC algorithm achieves a fuel-saving rate of 7.9%without compromising the truck’s travel time.

Self-Triggered Set Stabilization of Boolean Control Networks and Its Applications

Set stabilization is one of the essential problems in engineering systems, and self-triggered control(STC) can save the storage space for interactive information, and can be successfully applied in networked control systems with limited communication resources. In this study, the set stabilization problem and STC design of Boolean control networks are investigated via the semi-tensor product technique. On the one hand, the largest control invariant subset is calculated in terms of the strongly connected components of the state transition graph, by which a graph-theoretical condition for set stabilization is derived. On the other hand, a characteristic function is exploited to determine the triggering mechanism and feasible controls. Based on this, the minimum-time and minimum-triggering open-loop, state-feedback and output-feedback STCs for set stabilization are designed,respectively. As classic applications of self-triggered set stabilization, self-triggered synchronization, self-triggered output tracking and self-triggered output regulation are discussed as well. Additionally, several practical examples are given to illustrate the effectiveness of theoretical results.

Optimal Cooperative Secondary Control for Islanded DC Microgrids via a Fully Actuated Approach

DC-DC converter-based multi-bus DC microgrids(MGs) in series have received much attention, where the conflict between voltage recovery and current balancing has been a hot topic. The lack of models that accurately portray the electrical characteristics of actual MGs while is controller design-friendly has kept the issue active. To this end, this paper establishes a large-signal model containing the comprehensive dynamical behavior of the DC MGs based on the theory of high-order fully actuated systems, and proposes distributed optimal control based on this. The proposed secondary control method can achieve the two goals of voltage recovery and current sharing for multi-bus DC MGs. Additionally, the simple structure of the proposed approach is similar to one based on droop control, which allows this control technique to be easily implemented in a variety of modern microgrids with different configurations. In contrast to existing studies, the process of controller design in this paper is closely tied to the actual dynamics of the MGs. It is a prominent feature that enables engineers to customize the performance metrics of the system. In addition, the analysis of the stability of the closed-loop DC microgrid system, as well as the optimality and consensus of current sharing are given. Finally, a scaled-down solar and battery-based microgrid prototype with maximum power point tracking controller is developed in the laboratory to experimentally test the efficacy of the proposed control method.

Wellbore-heat-transfer-model-based optimization and control for cooling downhole drilling fluid

To address the two critical issues of evaluating the necessity of implementing cooling techniques and achieving real-time temperature control of drilling fluids underground in the current drilling fluid cooling technology,we first established a temperature and pressure coupled downhole heat transfer model,which can be used in both water-based and oil-based drilling fluid.Then,fourteen factors,which could affect wellbore temperature,were analyzed.Based on the standard deviation of the downhole temperature corresponding to each influencing factor,the influence of each factor was quantified.The influencing factors that can be used to guide the drilling fluid's cooling technology were drilling fluid thermal conductivity,drilling fluid heat capacity,drilling fluid density,drill strings rotation speed,pump rate,viscosity,ROP,and injection temperature.The nondominated sorting genetic algorithm was used to optimize these six parameters,but the optimization process took 182 min.Combining these eight parameters'influence rules with the nondominated sorting genetic algorithm can reduce the optimization time to 108 s.Theoretically,the downhole temperature has been demonstrated to increase with the inlet temperature increasing linearly under quasi-steady states.Combining this law and PID,the downhole temperature can be controlled,which can reduce the energy for cooling the surface drilling fluid and can ensure the downhole temperature reaches the set value as soon as possible.

Sand control mechanism of radial well filled with phase change material in hydrate reservoir

Radial well filled with phase change material has been proposed as a novel sand control method for hydrate exploitation.In order to reveal the sand control mechanism,CFD-DEM coupling method is applied to simulate the migration,settlement,and blockage processes of sand particles in the radial well.The obtained results indicate that three scenarios have been recognized for sand particles passing through sand control medium,based on the diameter ratio of sand control medium to sand particle(D_(d)):fully passing(D_(d)=8.75-22.5),partially passing and partially blocked(D_(d)=3.18-5.63),and completely blocked(D_(d)=2.18-3.21).After being captured by the sand control medium,sand particles can block pores,which increases fluid flow resistance and causes a certain pressure difference in the radial well.The pressure in the radial well should be lower than the hydrate phase equilibrium pressure during sand control design,for the purpose of promoting hydrate decomposition,and sand capture.The length of the radial well should be optimized based on the reservoir pore pressure,production pressure difference,bottom hole pressure,and the pressure gradient in the radial well.It should be noticed that the sand control medium leads to a decrease in permeability after sand particles captured.Even the permeability is reduced to several hundred millidarcy,it is still sufficient to ensure the effective flow of gas and water after hydrate decomposition.Increasing fluid velocity reduces the blocking capacity of the sand control medium,mainly because of deterioration in bridging between sand particles.

Screening, Treatment and Control of High Blood Pressure on Five Sites in Mali

Introduction: High blood pressure is defined as blood pressure greater than or equal to 140 mm Hg for systolic and or 90 mm Hg for diastolic. It constitutes a major public health problem, the leading chronic disease in the world. The objective was to determine the prevalence, treatment rate and control of hypertension. Methods: This was a cross-sectional and descriptive study which took place over a period of 6 months in a hospital environment and in the general population. Results: Of the 1000 participants, 637 had hypertension, giving a prevalence of 63.70% with a female predominance. Thirty-three percent (33%) were unaware of their high blood pressure. The age group 60 and more was the most represented (44%). A proportion of 33 and 23.20 were overweight and obese participants, respectively. Male subjects were more overweight than female, unlike obesity which was more common among female subjects. Sixty-two percent (62%) of hypertensives were treated, of whom 44% were non-compliant. The excessively high cost and consumption of medications as needed were the main factors in therapeutic non-compliance. Twenty-two percent (22%) of all hypertensive patients and 35% of treated hypertensive patients were controlled. Women were more treated but less observant and less controlled than men. Therapeutic coverage and combination therapy rates were lower in rural areas. Hypertensives who had a high level of education were better treated and controlled than those who had no level. Conclusion: High blood pressure remains a real public health problem in Mali. It is more common in people aged 60 and over and in females. One in three hypertensives were unaware of their hypertension. The majority received antihypertensive treatments, but only a minority of them had their hypertension controlled.

Adaptable and Dynamic Access Control Decision-Enforcement Approach Based on Multilayer Hybrid Deep Learning Techniques in BYOD Environment

Organizations are adopting the Bring Your Own Device(BYOD)concept to enhance productivity and reduce expenses.However,this trend introduces security challenges,such as unauthorized access.Traditional access control systems,such as Attribute-Based Access Control(ABAC)and Role-Based Access Control(RBAC),are limited in their ability to enforce access decisions due to the variability and dynamism of attributes related to users and resources.This paper proposes a method for enforcing access decisions that is adaptable and dynamic,based on multilayer hybrid deep learning techniques,particularly the Tabular Deep Neural Network Tabular DNN method.This technique transforms all input attributes in an access request into a binary classification(allow or deny)using multiple layers,ensuring accurate and efficient access decision-making.The proposed solution was evaluated using the Kaggle Amazon access control policy dataset and demonstrated its effectiveness by achieving a 94%accuracy rate.Additionally,the proposed solution enhances the implementation of access decisions based on a variety of resource and user attributes while ensuring privacy through indirect communication with the Policy Administration Point(PAP).This solution significantly improves the flexibility of access control systems,making themmore dynamic and adaptable to the evolving needs ofmodern organizations.Furthermore,it offers a scalable approach to manage the complexities associated with the BYOD environment,providing a robust framework for secure and efficient access management.