Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Dynamic simulation analysis of molten salt reactor-coupled air-steam combined cycle power generation system

A nonlinear dynamic simulation model based on coordinated control of speed and flow rate for the molten salt reactor and combined cycle systems is proposed here to ensure the coordination and stability between the molten salt reactor and power system.This model considers the impact of thermal properties of fluid variation on accuracy and has been validated with Simulink.This study reveals the capability of the control system to compensate for anomalous situations and maintain shaft stability in the event of perturbations occurring in high-temperature molten salt tank outlet parameters.Meanwhile,the control system’s impact on the system’s dynamic characteristics under molten salt disturbance is also analyzed.The results reveal that after the disturbance occurs,the controlled system benefits from the action of the control,and the overshoot and disturbance amplitude are positively correlated,while the system power and frequency eventually return to the initial values.This simulation model provides a basis for utilizing molten salt reactors for power generation and maintaining grid stability.

Superiority of a Convolutional Neural Network Model over Dynamical Models in Predicting Central Pacific ENSO

The application of deep learning is fast developing in climate prediction,in which El Ni?o–Southern Oscillation(ENSO),as the most dominant disaster-causing climate event,is a key target.Previous studies have shown that deep learning methods possess a certain level of superiority in predicting ENSO indices.The present study develops a deep learning model for predicting the spatial pattern of sea surface temperature anomalies(SSTAs)in the equatorial Pacific by training a convolutional neural network(CNN)model with historical simulations from CMIP6 models.Compared with dynamical models,the CNN model has higher skill in predicting the SSTAs in the equatorial western-central Pacific,but not in the eastern Pacific.The CNN model can successfully capture the small-scale precursors in the initial SSTAs for the development of central Pacific ENSO to distinguish the spatial mode up to a lead time of seven months.A fusion model combining the predictions of the CNN model and the dynamical models achieves higher skill than each of them for both central and eastern Pacific ENSO.

The impact of demographic dynamics on food consumption and its environmental outcomes:Evidence from China

With increasing population and changing demographics,food consumption has experienced a significant transition in quantity and quality.However,a dearth of knowledge remains regarding its environmental impacts and how it responds to demographic dynamics,particularly in emerging economies like China.Using the two-stage Quadratic Almost Demand System(QUAIDS)model,this study empirically examines the impact of demographic dynamics on food consumption and its environmental outcomes based on the provincial data from 2000 to 2020 in China.Under various scenarios,according to changes in demographics,we extend our analysis to project the long-term trend of food consumption and its environmental impacts,including greenhouse gas(GHG)emissions,water footprint(WF),and land appropriation(LA).The results reveal that an increase in the proportion of senior people significantly decreases the consumption of grain and livestock meat and increases the consumption of poultry,egg,and aquatic products,particularly for urban residents.Moreover,an increase in the proportion of males in the population leads to higher consumption of poultry and aquatic products.Correspondingly,in the current scenario of an increased aging population and sex ratio,it is anticipated that GHG emissions,WF,and LA are likely to decrease by 1.37,2.52,and 3.56%,respectively.More importantly,in the scenario adhering to the standards of nutritional intake according to the Dietary Guidelines for Chinese Residents in 2022,GHG emissions,WF,and LA in urban areas would increase by 12.78,20.94,and 18.32%,respectively.Our findings suggest that changing demographics should be considered when designing policies to mitigate the diet-environment-health trilemma and achieve sustainable food consumption.

Time-domain dynamic constitutive model suitable for mucky soil site seismic response

Soil nonlinear behavior displays noticeable effects on the site seismic response.This study proposes a new functional expression of the skeleton curve to replace the hyperbolic skeleton curve.By integrating shear modulus and combining the dynamic skeleton curve and the damping degradation coefficient,the constitutive equation of the logarithmic dynamic skeleton can be obtained,which considers the damping effect in a soil dynamics problem.Based on the finite difference method and the multi-transmitting boundary condition,a 1D site seismic response analysis program called Soilresp1D has been developed herein and used to analyze the time-domain seismic response in three types of sites.At the same time,this study also provides numerical simulation results based on the hyperbolic constitutive model and the equivalent linear method.The results verify the rationality of the new soil dynamic constitutive model.It can analyze the mucky soil site nonlinear seismic response,reflecting the deformation characteristics and damping effect of the silty soil.The hysteresis loop area is more extensive,and the residual strain is evident.

Mechanical properties and acoustic emission characteristics of soft rock with different water contents under dynamic disturbance

Uniaxial compression tests and cyclic loading acoustic emission tests were conducted on 20%,40%,60%,80%,dry and saturated muddy sandstone by using a creep impact loading system to investigate the mechanical properties and acoustic emission characteristics of soft rocks with different water contents under dynamic disturbance.The mechanical properties and acoustic emission characteristics of muddy sandstones at different water contents were analysed.Results of experimental studies show that water is a key factor in the mechanical properties of rocks,softening them,increasing their porosity,reducing their brittleness and increasing their plasticity.Under uniaxial compression,the macroscopic damage characteristics of the muddy sandstone change from mono-bevel shear damage and‘X’type conjugate bevel shear damage to a roadway bottom-drum type damage as the water content increases.Dynamic perturbation has a strengthening effect on the mechanical properties of samples with 60%and less water content,and a weakening effect on samples with 80%and more water content,but the weakening effect is not obvious.Macroscopic damage characteristics of dry samples remain unchanged,water samples from shear damage and tensile–shear composite damage gradually transformed into cleavage damage,until saturation transformation monoclinic shear damage.The evolution of acoustic emission energy and event number is mainly divided into four stages:loading stage(Ⅰ),dynamic loading stage(Ⅱ),yield failure stage(Ⅲ),and post-peak stage(Ⅳ),the acoustic emission characteristics of the stages were different for different water contents.The characteristic value of acoustic emission key point frequency gradually decreases,and the damage degree of the specimen increases,corresponding to low water content—high main frequency—low damage and high water content—low main frequency—high damage.

Hybrid Dynamic Variables-Dependent Event-Triggered Fuzzy Model Predictive Control

This article focuses on dynamic event-triggered mechanism(DETM)-based model predictive control(MPC) for T-S fuzzy systems.A hybrid dynamic variables-dependent DETM is carefully devised,which includes a multiplicative dynamic variable and an additive dynamic variable.The addressed DETM-based fuzzy MPC issue is described as a “min-max” optimization problem(OP).To facilitate the co-design of the MPC controller and the weighting matrix of the DETM,an auxiliary OP is proposed based on a new Lyapunov function and a new robust positive invariant(RPI) set that contain the membership functions and the hybrid dynamic variables.A dynamic event-triggered fuzzy MPC algorithm is developed accordingly,whose recursive feasibility is analysed by employing the RPI set.With the designed controller,the involved fuzzy system is ensured to be asymptotically stable.Two examples show that the new DETM and DETM-based MPC algorithm have the advantages of reducing resource consumption while yielding the anticipated performance.

Dynamics and genetic regulation of macronutrient concentrations during grain development in maize

Nitrogen(N), phosphorus(P), and potassium(K) are essential macronutrients that are crucial not only for maize growth and development, but also for crop yield and quality. The genetic basis of macronutrient dynamics and accumulation during grain filling in maize remains largely unknown. In this study, we evaluated grain N, P, and K concentrations in 206 recombinant inbred lines generated from a cross of DH1M and T877 at six time points after pollination. We then calculated conditional phenotypic values at different time intervals to explore the dynamic characteristics of the N, P, and K concentrations. Abundant phenotypic variations were observed in the concentrations and net changes of these nutrients. Unconditional quantitative trait locus(QTL) mapping revealed 41 non-redundant QTLs, including 17, 16, and 14 for the N, P, and K concentrations, respectively. Conditional QTL mapping uncovered 39 non-redundant QTLs related to net changes in the N, P, and K concentrations. By combining QTL, gene expression, co-expression analysis, and comparative genomic data, we identified 44, 36, and 44 candidate genes for the N, P, and K concentrations, respectively, including GRMZM2G371058 encoding a Doftype zinc finger DNA-binding family protein, which was associated with the N concentration, and GRMZM2G113967encoding a CBL-interacting protein kinase, which was related to the K concentration. The results deepen our understanding of the genetic factors controlling N, P, and K accumulation during maize grain development and provide valuable genes for the genetic improvement of nutrient concentrations in maize.

Vibration Reduction by a Partitioned Dynamic Vibration Absorber with Acoustic Black Hole Features

Vibration quality is a vital indicator for assessing the progress of modern equipment.The dynamic vibration absorber(DVA)based on the acoustic black hole(ABH)feature is a new passive control method that manipulates waves.It offers efficient energy focalization and broad-spectrum vibration suppression,making it highly promising for applications in large equipment such as aircraft,trains,and ships.Despite previous advancements in ABH-DVA development,certain challenges remain,particularly in ensuring effective coupling with host structures during control.To address these issues,this study proposes a partitioned ABH-featured dynamic vibration absorber(PABH-DVA)with partitions in the radial direction of the disc.By employing a plate as the host structure,simulations and experiments were conducted,demonstrating that the PABH-DVA outperforms the original symmetric ABH-DVA in terms of damping performance.The study also calculated and compared the coupling coefficients of the two ABH-DVAs to uncover the mechanism behind the enhanced damping.Simulation results revealed that the PABH-DVA exhibits more coupled modes,occasionally with lower coupling coefficients than the symmetric ABH-DVA.The influence of frequency ratio and modal mass was further analyzed to explain the reasons behind the PABH-DVA's superior damping performance.Additionally,the study discussed the impact of the number of slits and their orientation.This research further explains the coupling mechanism between the ABH-DVA and the controlled structure,and provides new ideas for the further application of ABH in engineering.

Application of GNSS-PPP on Dynamic Deformation Monitoring of Offshore Platforms

The real-time dynamic deformation monitoring of offshore platforms under environmental excitation is crucial to their safe operation.Although Global Navigation Satellite System-Precise Point Positioning(GNSS-PPP)has been considered for this purpose,its monitoring accuracy is relatively low.Moreover,the influence of background noise on the dynamic monitoring accuracy of GNSS-PPP remains unclear.Hence,it is imperative to further validate the feasibility of GNSS-PPP for deformation monitoring of offshore platforms.To address these concerns,vibration table tests with different amplitudes and frequencies are conducted.The results demonstrate that GNSS-PPP can effectively monitor horizontal vibration displacement as low as±30 mm,which is consistent with GNSS-RTK.Furthermore,the spectral characteristic of background noise in GNSS-PPP is similar to that of GNSS-RTK(Real Time Kinematic).Building on this observation,an improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise(CEEMDAN)has been proposed to de-noise the data and enhance the dynamic monitoring accuracy of GNSS-PPP.Field monitoring application research is also undertaken,successfully extracting and analyzing the dynamic deformation of an offshore platform structure under environmental excitation using GNSS-PPP monitoring in conjunction with improved CEEMDAN de-noising.By comparing the de-noised dynamic deformation trajectories of the offshore platform during different periods,it is observed that the platform exhibits reversible alternating vibration responses under environmental excitation,with more pronounced displacement deformation in the direction of load action.The research results confirm the feasibility and potential of GNSS-PPP for dynamic deformation monitoring of offshore platforms.

Analysis of CH_(4) and H_(2) Adsorption on Heterogeneous Shale Surfaces Using aMolecular Dynamics Approach

Determining the adsorption of shale gas on complex surfaces remains a challenge in molecular simulation studies.Difficulties essentially stem from the need to create a realistic shale structure model in terms of mineral heterogeneityand multiplicity.Moreover,precise characterization of the competitive adsorption of hydrogen andmethane in shale generally requires the experimental determination of the related adsorptive capacity.In thisstudy,the adsorption of adsorbates,methane(CH_(4)),and hydrogen(H_(2))on heterogeneous shale surface modelsof Kaolinite,Orthoclase,Muscovite,Mica,C_(60),and Butane has been simulated in the frame of a moleculardynamic’s numerical technique.The results show that these behaviors are influenced by pressure and potentialenergy.On increasing the pressure from 500 to 2000 psi,the sorption effect for CH_(4)significantly increasesbut shows a decline at a certain stage(if compared to H_(2)).The research findings also indicate that raw shalehas a higher capacity to adsorb CH_(4)compared to hydrogen.However,in shale,this difference is negligible.

A dynamic database of solid-state electrolyte(DDSE)picturing all-solid-state batteries

All-solid-state batteries(ASSBs)are a class of safer and higher-energy-density materials compared to conventional devices,from which solid-state electrolytes(SSEs)are their essential components.To date,investigations to search for high ion-conducting solid-state electrolytes have attracted broad concern.However,obtaining SSEs with high ionic conductivity is challenging due to the complex structural information and the less-explored structure-performance relationship.To provide a solution to these challenges,developing a database containing typical SSEs from available experimental reports would be a new avenue to understand the structureperformance relationships and find out new design guidelines for reasonable SSEs.Herein,a dynamic experimental database containing>600 materials was developed in a wide range of temperatures(132.40–1261.60 K),including mono-and divalent cations(e.g.,Li^(+),Na^(+),K^(+),Ag^(+),Ca^(2+),Mg^(2+),and Zn^(2+))and various types of anions(e.g.,halide,hydride,sulfide,and oxide).Data-mining was conducted to explore the relationships among different variates(e.g.,transport ion,composition,activation energy,and conductivity).Overall,we expect that this database can provide essential guidelines for the design and development of high-performance SSEs in ASSB applications.This database is dynamically updated,which can be accessed via our open-source online system.

Image super‐resolution via dynamic network

Convolutional neural networks depend on deep network architectures to extract accurate information for image super‐resolution.However,obtained information of these con-volutional neural networks cannot completely express predicted high‐quality images for complex scenes.A dynamic network for image super‐resolution(DSRNet)is presented,which contains a residual enhancement block,wide enhancement block,feature refine-ment block and construction block.The residual enhancement block is composed of a residual enhanced architecture to facilitate hierarchical features for image super‐resolution.To enhance robustness of obtained super‐resolution model for complex scenes,a wide enhancement block achieves a dynamic architecture to learn more robust information to enhance applicability of an obtained super‐resolution model for varying scenes.To prevent interference of components in a wide enhancement block,a refine-ment block utilises a stacked architecture to accurately learn obtained features.Also,a residual learning operation is embedded in the refinement block to prevent long‐term dependency problem.Finally,a construction block is responsible for reconstructing high‐quality images.Designed heterogeneous architecture can not only facilitate richer structural information,but also be lightweight,which is suitable for mobile digital devices.Experimental results show that our method is more competitive in terms of performance,recovering time of image super‐resolution and complexity.The code of DSRNet can be obtained at https://github.com/hellloxiaotian/DSRNet.

Community structure and species diversity dynamics of a subtropical evergreen broad-leaved forest in China:2005 to 2020

Here,we characterize the temporal and spatial dynamics of forest community structure and species diversity in a subtropical evergreen broad-leaved forest in China.We found that community structure in this forest changed over a 15-year period.Specifically,renewal and death of common species was large,with the renewal of individuals mainly concentrated within a few populations,especially those of Aidia canthioides and Cryptocarya concinna.The numbers of individual deaths for common species were concentrated in the small and mid-diameter level.The spatial distribution of community species diversity fluctuated in each monitoring period,showing a more dispersed diversity after the 15-year study period,and the coefficient of variation on quadrats increased.In 2010,the death and renewal of the community and the spatial variation of species diversity were different compared to other survey years.Extreme weather may have affected species regeneration and community stability in our subtropical monsoon evergreen broad-leaved forests.Our findings suggest that strengthening the monitoring and management of the forest community will help better understand the long-and short-term causes of dynamic fluctuations of community structure and species diversity,and reveal the factors that drive changes in community structure.

Construction of Dynamic Alloy Interfaces for Uniform Li Deposition in Li-Metal Batteries

It is well accepted that a lithiophilic interface can effectively regulate Li deposition behaviors,but the influence of the lithiophilic interface is gradually diminished upon continuous Li deposition that completely isolates Li from the lithiophilic metals.Herein,we perform in-depth studies on the creation of dynamic alloy interfaces upon Li deposition,arising from the exceptionally high diffusion coefficient of Hg in the amalgam solid solution.As a comparison,other metals such as Au,Ag,and Zn have typical diffusion coefficients of 10-20 orders of magnitude lower than that of Hg in the similar solid solution phases.This difference induces compact Li deposition pattern with an amalgam substrate even with a high areal capacity of 55 mAh cm^(-2).This finding provides new insight into the rational design of Li anode substrate for the stable cycling of Li metal batteries.

UAV-assisted data collection for wireless sensor networks with dynamic working modes

Wireless Sensor Network(WSN)is a cornerstone of Internet of Things(IoT)and has rich application scenarios.In this work,we consider a heterogeneous WSN whose sensor nodes have a diversity in their Residual Energy(RE).In this work,to protect the sensor nodes with low RE,we investigate dynamic working modes for sensor nodes which are determined by their RE and an introduced energy threshold.Besides,we employ an Unmanned Aerial Vehicle(UAV)to collect the stored data from the heterogeneous WSN.We aim to jointly optimize the cluster head selection,energy threshold and sensor nodes’working mode to minimize the weighted sum of energy con-sumption from the WSN and UAV,subject to the data collection rate constraint.To this end,we propose an efficient search method to search for an optimal energy threshold,and develop a penalty-based successive convex approximation algorithm to select the cluster heads.Then we present a low-complexity iterative approach to solve the joint optimization problem and discuss the implementation procedure.Numerical results justify that our proposed approach is able to reduce the energy consumption of the sensor nodes with low RE significantly and also saves energy for the whole WSN.

A stable implicit nodal integration-based particle finite element method(N-PFEM)for modelling saturated soil dynamics

In this study,we present a novel nodal integration-based particle finite element method(N-PFEM)designed for the dynamic analysis of saturated soils.Our approach incorporates the nodal integration technique into a generalised Hellinger-Reissner(HR)variational principle,creating an implicit PFEM formulation.To mitigate the volumetric locking issue in low-order elements,we employ a node-based strain smoothing technique.By discretising field variables at the centre of smoothing cells,we achieve nodal integration over cells,eliminating the need for sophisticated mapping operations after re-meshing in the PFEM.We express the discretised governing equations as a min-max optimisation problem,which is further reformulated as a standard second-order cone programming(SOCP)problem.Stresses,pore water pressure,and displacements are simultaneously determined using the advanced primal-dual interior point method.Consequently,our numerical model offers improved accuracy for stresses and pore water pressure compared to the displacement-based PFEM formulation.Numerical experiments demonstrate that the N-PFEM efficiently captures both transient and long-term hydro-mechanical behaviour of saturated soils with high accuracy,obviating the need for stabilisation or regularisation techniques commonly employed in other nodal integration-based PFEM approaches.This work holds significant implications for the development of robust and accurate numerical tools for studying saturated soil dynamics.

An active high-static-low-dynamic-stiffness vibration isolator with adjustable buckling beams:theory and experiment

High-static-low-dynamic-stiffness(HSLDS)vibration isolators with buckling beams have been widely used to isolate external vibrations.An active adjustable device composed of proportion integration(PI)active controllers and piezoelectric actuators is proposed for improving the negative stiffness stroke of buckling beams.A nonlinear output frequency response function is used to analyze the effect of the vibration reduction.The prototype of the active HSLDS device is built,and the verification experiment is conducted.The results show that compared with the traditional HSLDS vibration isolator,the active HSLDS device can broaden the isolation frequency bandwidth,and effectively reduce the resonant amplitude by adjusting the active control parameters.The maximum vibration reduction rate of the active HSLDS vibration isolator can attain 89.9%,and the resonant frequency can be reduced from 31.08 Hz to 13.28 Hz.Therefore,this paper devotes to providing a new design scheme for enhanced HSLDS vibration isolators.

Millisecond dynamics of colloidal suspension studied by X-ray photon correlation spectroscopy at the Shanghai Synchrotron Radiation Facility

X-ray photon correlation spectroscopy(XPCS)has emerged as a powerful tool for probing the nanoscale dynamics of soft condensed matter and strongly correlated materials owing to its high spatial resolution and penetration capabilities.This technique requires high brilliance and beam coherence,which are not directly available at modern synchrotron beamlines in China.To facilitate future XPCS experiments,we modified the optical setup of the newly commissioned BL10U1 USAXS beamline at the Shanghai Synchrotron Radiation Facility(SSRF).Subsequently,we performed XPCS measurements on silica suspensions in glycerol,which were opaque owing to their high concentrations.Images were collected using a high frame rate area detector.A comprehensive analysis was performed,yielding correlation functions and several key dynamic parameters.All the results were consistent with the theory of Brownian motion and demonstrated the feasibility of XPCS at SSRF.Finally,by carefully optimizing the setup and analyzing the algorithms,we achieved a time resolution of 2 ms,which enabled the characterization of millisecond dynamics in opaque systems.

Dynamically triggered seismicity on a tectonic scale:A review

Earthquakes triggered by dynamic disturbances have been confirmed by numerous observations and experiments.In the past several decades,earthquake triggering has attracted increasing attention of scholars in relation to exploring the mechanism of earthquake triggering,earthquake prediction,and the desire to use the mechanism of earthquake triggering to reduce,prevent,or trigger earthquakes.Natural earthquakes and large‐scale explosions are the most common sources of dynamic disturbances that trigger earthquakes.In the past several decades,some models have been developed,including static,dynamic,quasi‐static,and other models.Some reviews have been published,but explosiontriggered seismicity was not included.In recent years,some new results on earthquake triggering have emerged.Therefore,this paper presents a new review to reflect the new results and include the content of explosion‐triggered earthquakes for the reference of scholars in this area.Instead of a complete review of the relevant literature,this paper primarily focuses on the main aspects of dynamic earthquake triggering on a tectonic scale and makes some suggestions on issues that need to be resolved in this area in the future.