Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Effects of Different Herbicides on Weed Control in Alfalfa Field

[Objectives]The paper was to systematically study the technology of weed control in alfalfa field.[Methods]Reviving alfalfa field and newly sown alfalfa field after emergence were selected,and the effects of different herbicides on weed control and alfalfa yield were discussed.[Results]The optimal herbicides after alfalfa reviving were 5%imazethapyr and 10%imazethapyr,and the optimal dosages were 1.5 and 1.05 L/hm 2,respectively.The optimal herbicides after emergence of newly born alfalfa were 5%imazethapyr and 10%imazethapyr,and the optimal dosages were 1.5 and 0.75 L/hm 2,respectively.[Conclusions]This study will provide a technical support for high quality production of alfalfa.

Cultivation system influenced the critical period for weed control in cotton field

Background The critical period of weed control(CPWC) refers to the period of time during the crop growth cycle when weeds must be controlled to prevent yield losses.Ultra-narrow row(UNR) is a method of planting of cotton in rows that are 25 cm or less apart.Amongst cultural techniques for weed control,the use of narrow row spacing is considered to be a most promising approach that can effectively suppress weed growth and provide greater yields in cotton.This cultivation system can shorten the length of the critical weed-crop interference duration and results in greater yield.The current research aimed to determination of critical time of weed control in cotton(Gossypium hirsutum L.) under conventional and ultra-narrow row spacing conditions.Field experiments were arranged as factorial experiment in a randomized complete block design with three replications.Factors were cultivation system(conventional(50 cm row spacing) and ultra narrow row(25 cm row spacing and weed treatment including 30,45,60,and 75 days weeding after emergence during the growing season(weed free),and 30,45,60,and 75 without weeding(weed infested) in the growing season along with weedy and weed-free from sowing to harvesting.A four-parameter loglogistic model was fit to the two sets of relating relative crop yield to data obtained from increasing durations of weed interference and lengths of weed-free period.Results In both years and cultivation systems,the relative yield of cotton decreased with the increasing duration of weed-interference but increased with the increasing duration of weed-free period.Ultra-narrow row cultivation delayed the beginning of the CPWC in cotton.Under ultra-narrow row condition,the CPWC ranged from 21 to 99 days after germination in 2021 and 23 to 91 days in 2022 based on the 5% acceptable yield loss.Under conventional cultivation CPWC ranged from 17 to 102 days after emergence in 2021 and 18 to 95 days after emergence in 2022.Conclusions Under both conventional and Ultra-narrow row conditions,weed interference reduces seed yield.Under ultra-narrow row condition,weed interference until 21.1–23.5 days after cotton emergence and under conventional condition,weed interference until 16.9–18.5 days after cotton emergence had not significant reduction on cotton yield.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

Vibration Control of A Flexible Marine Riser System Subject to Input Dead Zone and Extraneous Disturbances

An observer-based adaptive backstepping boundary control is proposed for vibration control of flexible offshore riser systems with unknown nonlinear input dead zone and uncertain environmental disturbances.The control algorithm can update the control law online through real-time data to make the controller adapt to the environment and improve the control precision.Specifically,based on the adaptive backstepping framework,virtual control laws and Lyapunov functions are designed for each subsystem.Three direction interference observers are designed to track the timevarying boundary disturbance.On this basis,the inverse of the dead zone and linear state transformation are used to compensate for the original system and eliminate the adverse effects of the dead zone.In addition,the stability of the closed-loop system is proven by Lyapunov stability theory.All the system states are bounded,and the vibration offset of the riser converges to a small area of the initial position.Finally,four examples of flexible marine risers are simulated in MATLAB to verify the effectiveness of the proposed controller.

Control of highly pathogenic avian influenza through vaccination

The stamping-out strategy has been used to control highly pathogenic avian influenza viruses in many countries,driven by the belief that vaccination would not be successful against such viruses and fears that avian influenza virus in vaccinated birds would evolve more rapidly and pose a greater risk to humans.In this review,we summarize the successes in controlling highly pathogenic avian influenza in China and make suggestions regarding the requirements for vaccine selection and effectiveness.In addition,we present evidence that vaccination of poultry not only eliminates human infection with avian influenza virus,but also significantly reduces and abolishes some harmful characteristics of avian influenza virus.

An Integrated Control Framework for Torque Vectoring and Active Suspension System

Four-wheel independently driven electric vehicles(FWID-EV)endow a flexible and scalable control framework to improve vehicle performance.This paper integrates the torque vectoring and active suspension system(ASS)to enhance the vehicle’s longitudinal and vertical motion control performance.While the nonlinear characteristic of the tire model leads to a relatively heavier computational burden.To facilitate the controller design and ease the load,a half-vehicle dynamics system is built and simplified to the linear-time-varying(LTV)model.Then a model predictive controller is developed by formulating the objective function by comprehensively considering the safety,energy-saving and comfort requirements.The in-wheel motor efficiency and the power loss of tire slip are treated as optimization indices in this work to reduce energy consumption.Finally,the effectiveness of the proposed controller is verified through the rapid-control-prototype(RCP)test.The results demonstrate the enhancement of the energy-saving as well as comfort on the basis of vehicle stability.

Mitochondrial dysfunction and quality control lie at the heart of subarachnoid hemorrhage

The dramatic increase in intracranial pressure after subarachnoid hemorrhage leads to a decrease in cerebral perfusion pressure and a reduction in cerebral blood flow.Mitochondria are directly affected by direct factors such as ischemia,hypoxia,excitotoxicity,and toxicity of free hemoglobin and its degradation products,which trigger mitochondrial dysfunction.Dysfunctional mitochondria release large amounts of reactive oxygen species,inflammatory mediators,and apoptotic proteins that activate apoptotic pathways,further damaging cells.In response to this array of damage,cells have adopted multiple mitochondrial quality control mechanisms through evolution,including mitochondrial protein quality control,mitochondrial dynamics,mitophagy,mitochondrial biogenesis,and intercellular mitochondrial transfer,to maintain mitochondrial homeostasis under pathological conditions.Specific interventions targeting mitochondrial quality control mechanisms have emerged as promising therapeutic strategies for subarachnoid hemorrhage.This review provides an overview of recent research advances in mitochondrial pathophysiological processes after subarachnoid hemorrhage,particularly mitochondrial quality control mechanisms.It also presents potential therapeutic strategies to target mitochondrial quality control in subarachnoid hemorrhage.

A Blockchain-Based Access Control Scheme for Reputation Value Attributes of the Internet of Things

The Internet of Things(IoT)access controlmechanism may encounter security issues such as single point of failure and data tampering.To address these issues,a blockchain-based IoT reputation value attribute access control scheme is proposed.Firstly,writing the reputation value as an attribute into the access control policy,and then deploying the access control policy in the smart contract of the blockchain system can enable the system to provide more fine-grained access control;Secondly,storing a large amount of resources fromthe Internet of Things in Inter Planetary File System(IPFS)to improve system throughput;Finally,map resource access operations to qualification tokens to improve the performance of the access control system.Complete simulation experiments based on the Hyperledger Fabric platform.Fromthe simulation experimental results,it can be seen that the access control system can achieve more fine-grained and dynamic access control while maintaining high throughput and low time delay,providing sufficient reliability and security for access control of IoT devices.

A Review on Vibration Control of Multiple Cylinders Subjected to FlowInduced Vibrations

The fatigue damage caused by flow-induced vibration(FIV)is one of the major concerns for multiple cylindrical structures in many engineering applications.The FIV suppression is of great importance for the security of many cylindrical structures.Many active and passive control methods have been employed for the vibration suppression of an isolated cylinder undergoing vortex-induced vibrations(VIV).The FIV suppression methods are mainly extended to the multiple cylinders from the vibration control of the isolated cylinder.Due to the mutual interference between the multiple cylinders,the FIV mechanism is more complex than the VIV mechanism,which makes a great challenge for the FIV suppression.Some efforts have been devoted to vibration suppression of multiple cylinder systems undergoing FIV over the past two decades.The control methods,such as helical strakes,splitter plates,control rods and flexible sheets,are not always effective,depending on many influence factors,such as the spacing ratio,the arrangement geometrical shape,the flow velocity and the parameters of the vibration control devices.The FIV response,hydrodynamic features and wake patterns of the multiple cylinders equipped with vibration control devices are reviewed and summarized.The FIV suppression efficiency of the vibration control methods are analyzed and compared considering different influence factors.Further research on the FIV suppression of multiple cylinders is suggested to provide insight for the development of FIV control methods and promote engineering applications of FIV control methods.

Communication Resource-Efficient Vehicle Platooning Control With Various Spacing Policies

Platooning represents one of the key features that connected automated vehicles may possess as it allows multiple automated vehicles to be maneuvered cooperatively with small headways on roads. However, a critical challenge in accomplishing automated vehicle platoons is to deal with the effects of intermittent and sporadic vehicle-to-vehicle data transmissions caused by limited wireless communication resources. This paper addresses the co-design problem of dynamic event-triggered communication scheduling and cooperative adaptive cruise control for a convoy of automated vehicles with diverse spacing policies. The central aim is to achieve automated vehicle platooning under various gap references with desired platoon stability and spacing performance requirements, while simultaneously improving communication efficiency. Toward this aim, a dynamic event-triggered scheduling mechanism is developed such that the intervehicle data transmissions are scheduled dynamically and efficiently over time. Then, a tractable co-design criterion on the existence of both the admissible event-driven cooperative adaptive cruise control law and the desired scheduling mechanism is derived. Finally, comparative simulation results are presented to substantiate the effectiveness and merits of the obtained results.

A 16-bit 18-MSPS flash-assisted SAR ADC with hybrid synchronous and asynchronous control logic

This paper presents a 16-bit,18-MSPS(million samples per second)flash-assisted successive-approximation-register(SAR)analog-to-digital converter(ADC)utilizing hybrid synchronous and asynchronous(HYSAS)timing control logic based on an on-chip delay-locked loop(DLL).The HYSAS scheme can provide a longer settling time for the capacitive digital-to-analog converter(CDAC)than the synchronous and asynchronous SAR ADC.Therefore,the issue of incomplete settling or ringing in the DAC voltage for cases of either on-chip or off-chip reference voltage can be solved to a large extent.In addition,the fore-ground calibration of the CDAC’s mismatch is performed with a finite-impulse-response bandpass filter(FIR-BPF)based least-mean-square(LMS)algorithm in an off-chip FPGA(field programmable gate array).Fabricated in 40-nm CMOS process,the proto-type ADC achieves 94.02-dB spurious-free dynamic range(SFDR),and 75.98-dB signal-to-noise-and-distortion ratio(SNDR)for a 2.88-MHz input under 18-MSPS sampling rate.

Distributed Platooning Control of Automated Vehicles Subject to Replay Attacks Based on Proportional Integral Observers

Secure platooning control plays an important role in enhancing the cooperative driving safety of automated vehicles subject to various security vulnerabilities.This paper focuses on the distributed secure control issue of automated vehicles affected by replay attacks.A proportional-integral-observer(PIO)with predetermined forgetting parameters is first constructed to acquire the dynamical information of vehicles.Then,a time-varying parameter and two positive scalars are employed to describe the temporal behavior of replay attacks.In light of such a scheme and the common properties of Laplace matrices,the closed-loop system with PIO-based controllers is transformed into a switched and time-delayed one.Furthermore,some sufficient conditions are derived to achieve the desired platooning performance by the view of the Lyapunov stability theory.The controller gains are analytically determined by resorting to the solution of certain matrix inequalities only dependent on maximum and minimum eigenvalues of communication topologies.Finally,a simulation example is provided to illustrate the effectiveness of the proposed control strategy.

A Proposed Method for Evaluating Management Feasibility When Determining Weed Control Priorities after Major Fires and Floods

Major fires and floods have enormous impacts on natural ecosystems and are predicted to increase in frequency with global warming.Land managers need to make decisions on the prioritisation of weeds for control in post-disturbance landscapes,but little is available in the way of guidance to support timely decision making.Semi-quantitative models(e.g.,scoring systems)have been employed routinely in weed risk assessment,which considers the potential impacts posed by weeds,as well as the likelihood of these impacts being realised.Some progress has been made in the development of similar models addressing the topic of weed risk management.Under conditions prevailing after major disturbances,changes(both positive and negative)can be expected in the multiple factors that determine weed management feasibility,relative to pre-disturbance conditions.A semi-quantitative model is proposed that is based on the key factors that contribute to weed management feasibility in post-disturbance environments,along with annotated modules that could be used by land managers in both post-fire and post-flood situations.The fundamental challenge for weed management in these scenarios lies in the identification of differences between weeds and native species in relation to(1)patterns of seedling emergence;and(2)detectability relative to the growth stage.These two factors will determine the timing of control actions that are designed to address the trade-off between weed control and off-target damage during the period when both types of plants are recovering from a major disturbance event.The model is intuitively sound,but field testing is required to determine both its practical value and any necessary improvement.

Path Planning and Tracking Control for Parking via Soft Actor-Critic Under Non-Ideal Scenarios

Parking in a small parking lot within limited space poses a difficult task. It often leads to deviations between the final parking posture and the target posture. These deviations can lead to partial occupancy of adjacent parking lots, which poses a safety threat to vehicles parked in these parking lots. However, previous studies have not addressed this issue. In this paper, we aim to evaluate the impact of parking deviation of existing vehicles next to the target parking lot(PDEVNTPL) on the automatic ego vehicle(AEV) parking, in terms of safety, comfort, accuracy, and efficiency of parking. A segmented parking training framework(SPTF) based on soft actor-critic(SAC) is proposed to improve parking performance. In the proposed method, the SAC algorithm incorporates strategy entropy into the objective function, to enable the AEV to learn parking strategies based on a more comprehensive understanding of the environment. Additionally, the SPTF simplifies complex parking tasks to maintain the high performance of deep reinforcement learning(DRL). The experimental results reveal that the PDEVNTPL has a detrimental influence on the AEV parking in terms of safety, accuracy, and comfort, leading to reductions of more than 27%, 54%, and 26%respectively. However, the SAC-based SPTF effectively mitigates this impact, resulting in a considerable increase in the parking success rate from 71% to 93%. Furthermore, the heading angle deviation is significantly reduced from 2.25 degrees to 0.43degrees.

Adaptive Trajectory Tracking Control for Nonholonomic Wheeled Mobile Robots:A Barrier Function Sliding Mode Approach

The trajectory tracking control performance of nonholonomic wheeled mobile robots(NWMRs)is subject to nonholonomic constraints,system uncertainties,and external disturbances.This paper proposes a barrier function-based adaptive sliding mode control(BFASMC)method to provide high-precision,fast-response performance and robustness for NWMRs.Compared with the conventional adaptive sliding mode control,the proposed control strategy can guarantee that the sliding mode variables converge to a predefined neighborhood of origin with a predefined reaching time independent of the prior knowledge of the uncertainties and disturbances bounds.Another advantage of the proposed algorithm is that the control gains can be adaptively adjusted to follow the disturbances amplitudes thanks to the barrier function.The benefit is that the overestimation of control gain can be eliminated,resulting in chattering reduction.Moreover,a modified barrier function-like control gain is employed to prevent the input saturation problem due to the physical limit of the actuator.The stability analysis and comparative experiments demonstrate that the proposed BFASMC can ensure the prespecified convergence performance of the NWMR system output variables and strong robustness against uncertainties/disturbances.

Technical Points of Green Prevention and Control Technology of Major Diseases and Pests in Lixian Rhubarb(Rheum palmatum L.)

Based on different types of diseases,pests and weeds in the whole growth period of rhubarb(sowing period-harvesting period),the corresponding green prevention and control technology is proposed,aiming to further reduce the application amount of pesticides and fertilizers in the production of medicinal sources of Lixian rhubarb during the"14 th Five-Year Plan"period.The results will provide a theoretical basis for increasing the promotion and application of agricultural prevention and control(including disease-resistant varieties,ecological regulation),physical prevention and control,biological prevention and control measures,thus ensuring effective protection of the ecological environment,green,healthy and sustainable development of traditional Chinese medicine agriculture in Longnan,and source quality of authentic medicinal materials.

A Tutorial on Quantized Feedback Control

In this tutorial paper, we explore the field of quantized feedback control, which has gained significant attention due to the growing prevalence of networked control systems. These systems require the transmission of feedback information, such as measurements and control signals, over digital networks, presenting novel challenges in estimation and control design. Our examination encompasses various topics, including the minimal information needed for effective feedback control, the design of quantizers, strategies for quantized control design and estimation,achieving consensus control with quantized data, and the pursuit of high-precision tracking using quantized measurements.

Enhanced Species Diversity Unlocked by Habitat Conservation in Agricultural Landscapes: New Perspectives on Ecological Weed Management and Sustainable Intensification—Species Diversity and Habitat Conservation

The impulse to remain profitable by increasing agricultural production levels in view of the greater demand for food, provided impetus to production intensification. The aim of this review is to summarise current literature, reporting specifically on the impact of production intensification on habitats and yield constraints caused by weeds. Secondly, in alleviating these effects over the short term, ecological measures that enhance species diversity in conserved habitats and promote semi-natural habitats in the agricultural landscape, are discussed. In large-scale intensive agriculture, weed control is predominantly rooted in agrochemical applications in the form of herbicides. Long lasting intensive agricultural practices show discord both with the promotion of the biodiversity of microbes belowground and aboveground and with organisms involved in the breaking down of plant material. The presence of native species in the surroundings, in combination with hedgerows and field margins, with a comparatively intricate and balanced variety of plants in a sheltered environment, are essential for settlement of benign insects, particularly in the face of intensive agricultural production. The promising tactic of advantageous seed predators enables decreased herbicide applications. Crop mosaics arranged to advance compatibility at the landscape scale are important to bolster pollination services and insect management, while ecological variety in the surroundings acts as a safety net for habitat diversity. Weed control in combination with different tactics of vegetation use, comprising cover cropping, hedgerows and field margins, sets up safe havens in the landscape, and improves the diffusion of complementary life forms. Field margins perform a meaningful natural function as point of provision for forage, safe havens and distribution passageways for pollinators and insect predators. Production practices that promote more heterogeneity and combine high density semi-natural safe havens and habitat conservation in agro ecosystems are beneficial to species diversity across trophic levels and contribute to agricultural production stability and food safety.

Risk-Informed Model-Free Safe Control of Linear Parameter-Varying Systems

This paper presents a risk-informed data-driven safe control design approach for a class of stochastic uncertain nonlinear discrete-time systems.The nonlinear system is modeled using linear parameter-varying(LPV)systems.A model-based probabilistic safe controller is first designed to guarantee probabilisticλ-contractivity(i.e.,stability and invariance)of the LPV system with respect to a given polyhedral safe set.To obviate the requirement of knowing the LPV system model and to bypass identifying its open-loop model,its closed-loop data-based representation is provided in terms of state and scheduling data as well as a decision variable.It is shown that the variance of the closedloop system,as well as the probability of safety satisfaction,depends on the decision variable and the noise covariance.A minimum-variance direct data-driven gain-scheduling safe control design approach is presented next by designing the decision variable such that all possible closed-loop system realizations satisfy safety with the highest confidence level.This minimum-variance approach is a control-oriented learning method since it minimizes the variance of the state of the closed-loop system with respect to the safe set,and thus minimizes the risk of safety violation.Unlike the certainty-equivalent approach that results in a risk-neutral control design,the minimum-variance method leads to a risk-averse control design.It is shown that the presented direct risk-averse learning approach requires weaker data richness conditions than existing indirect learning methods based on system identification and can lead to a lower risk of safety violation.Two simulation examples along with an experimental validation on an autonomous vehicle are provided to show the effectiveness of the presented approach.