Reset-Free Reinforcement Learning via Multi-State Recovery and Failure Prevention for Autonomous Robots

Reinforcement learning holds promise in enabling robotic tasks as it can learn optimal policies via trial and error.However,the practical deployment of reinforcement learning usually requires human intervention to provide episodic resets when a failure occurs.Since manual resets are generally unavailable in autonomous robots,we propose a reset-free reinforcement learning algorithm based on multi-state recovery and failure prevention to avoid failure-induced resets.The multi-state recovery provides robots with the capability of recovering from failures by self-correcting its behavior in the problematic state and,more importantly,deciding which previous state is the best to return to for efficient re-learning.The failure prevention reduces potential failures by predicting and excluding possible unsafe actions in specific states.Both simulations and real-world experiments are used to validate our algorithm with the results showing a significant reduction in the number of resets and failures during the learning.

Development of diabetic complications and influencing factors among 32653 type 2 diabetes patients:retrospective cohort study using a multi-state Markov

Background:Patients with type 2 diabetes are at high risk for developing multiple chronic complications.However,there is a lack of studies of the cumulative number of diabetic complications in China.Methods:A retrospective cohort study was performed from 2009 to 2021.Type 2 diabetes patients who were first diagnosed after the age of 35 years between January 1,2009,and December 31,2017,were included.Five states were defined according to the number of chronic complications:no(S0),one(S1),two(S2),three(S3),and four or more complications(S4).A multi-state Markov model was constructed to estimate transition probability,transition intensity,mean sojourn time,and the possible factors for each state.Results:The study included 32653 type 2 diabetes patients(mean age,59.59 years;15929(48.8%)male),and mean follow-up time of 7.75 years.In all,4375 transitions were observed.The 12-year transition probability of from state S0 to S1 was the lowest at 16.4%,while that from S2 to S3 was the highest,at 45.6%.Higher fasting blood glucose,lower high-density lipoprotein cholesterol,higher total cholesterol,and an unhealthy diet were associated with higher risk of progression from S0 to S1.Being female,less than 60 years old,weekly physical activity,and vegetarian diet decreased this risk.Being female and less than 60 years old reduced the likelihood of transition from S1 to S2,whereas lower high-density lipoprotein cholesterol increased this likelihood.Conclusions:Following the occurrence of two complications in type 2 diabetes patients,the risk for accumulating a third complication within a short time is significantly increased.It is important to take advantage of the stable window period when patients have fewer than two complications,strengthen the monitoring of blood glucose and blood lipids,and encourage patients to maintain good living habits to prevent further deterioration.

A Two-Step Algorithm to Estimate Variable Importance for Multi-State Data:An Application to COVID-19

Survival data with amulti-state structure are frequently observed in follow-up studies.An analytic approach based on a multi-state model(MSM)should be used in longitudinal health studies in which a patient experiences a sequence of clinical progression events.One main objective in the MSM framework is variable selection,where attempts are made to identify the risk factors associated with the transition hazard rates or probabilities of disease progression.The usual variable selection methods,including stepwise and penalized methods,do not provide information about the importance of variables.In this context,we present a two-step algorithm to evaluate the importance of variables formulti-state data.Three differentmachine learning approaches(randomforest,gradient boosting,and neural network)as themost widely usedmethods are considered to estimate the variable importance in order to identify the factors affecting disease progression and rank these factors according to their importance.The performance of our proposed methods is validated by simulation and applied to the COVID-19 data set.The results revealed that the proposed two-stage method has promising performance for estimating variable importance.

Morphology and valence state evolution of Cu:Unraveling the impact on nitric oxide electroreduction

Ammonia(NH3)serves as a critical component in the fertilizer industry and fume gas denitrification.However,the conventional NH3production process,namely the Haber-Bosch process,leads to considerable energy consumption and waste gas emissions.To address this,electrocatalytic nitric oxide reduction reaction(NORR)has emerged as a promising strategy to bridge NH3consumption to NH3production,harnessing renewable electricity for a sustainable future.Copper(Cu)stands out as a prominent electrocatalyst for NO reduction,given its exceptional NH3yield and selectivity.However,a crucial aspect that remains insufficiently explored is the effects of morphology and valence states of Cu on the NORR performance.In this investigation,we synthesized CuO nanowires(CuO-NF)and Cu nanocubes(Cu-NF)as cathodes through an in situ growth method.Remarkably,CuO-NF exhibited an impressive NH3yield of 0.50±0.02 mg cm^(-2)h^(-1)at-0.6 V vs.reversible hydrogen electrode(RHE)with faradaic efficiency of29,68%±1,35%,surpassing that of Cu-NF(0.17±0.01 mg cm^(-2)h^(-1),16.18%±1.40%).Throughout the electroreduction process,secondary cubes were generated on the CuO-NF surface,preserving their nanosheet cluster morphology,sustained by an abundant supply of subsurface oxygen(s-O)even after an extended duration of 10 h,until s-O depletion ensued.Conversely,Cu-NF exhibited inadequate s-O content,leading to rapid crystal collapse within the same timeframe.The distinctive current-potential relationship,akin to a volcano-type curve,was attributed to distinct NO hydrogenation mechanisms.Further Tafel analysis revealed the exchange current density(i0)and standard heterogeneous rate constant(k0)for CuO-NF,yielding 3.44×10^(-6)A cm^(-2)and 3.77×10^(-6)cm^(-2)s^(-1)when NORR was driven by overpotentials.These findings revealed the potential of CuO-NF for NO reduction and provided insights into the intricate interplay between crystal morphology,valence states,and electrochemical performance.

Assessment of Wet Season Precipitation in the Central United States by the Regional Climate Simulation of the WRFG Member in NARCCAP and Its Relationship with Large-Scale Circulation Biases

Assessment of past-climate simulations of regional climate models(RCMs)is important for understanding the reliability of RCMs when used to project future regional climate.Here,we assess the performance and discuss possible causes of biases in a WRF-based RCM with a grid spacing of 50 km,named WRFG,from the North American Regional Climate Change Assessment Program(NARCCAP)in simulating wet season precipitation over the Central United States for a period when observational data are available.The RCM reproduces key features of the precipitation distribution characteristics during late spring to early summer,although it tends to underestimate the magnitude of precipitation.This dry bias is partially due to the model’s lack of skill in simulating nocturnal precipitation related to the lack of eastward propagating convective systems in the simulation.Inaccuracy in reproducing large-scale circulation and environmental conditions is another contributing factor.The too weak simulated pressure gradient between the Rocky Mountains and the Gulf of Mexico results in weaker southerly winds in between,leading to a reduction of warm moist air transport from the Gulf to the Central Great Plains.The simulated low-level horizontal convergence fields are less favorable for upward motion than in the NARR and hence,for the development of moist convection as well.Therefore,a careful examination of an RCM’s deficiencies and the identification of the source of errors are important when using the RCM to project precipitation changes in future climate scenarios.

Thermal safety boundary of lithium-ion battery at different state of charge

Thermal runaway(TR)is a critical issue hindering the large-scale application of lithium-ion batteries(LIBs).Understanding the thermal safety behavior of LIBs at the cell and module level under different state of charges(SOCs)has significant implications for reinforcing the thermal safety design of the lithium-ion battery module.This study first investigates the thermal safety boundary(TSB)correspondence at the cells and modules level under the guidance of a newly proposed concept,safe electric quantity boundary(SEQB).A reasonable thermal runaway propagation(TRP)judgment indicator,peak heat transfer power(PHTP),is proposed to predict whether TRP occurs.Moreover,a validated 3D model is used to quantitatively clarify the TSB at different SOCs from the perspective of PHTP,TR trigger temperature,SOC,and the full cycle life.Besides,three different TRP transfer modes are discovered.The interconversion relationship of three different TRP modes is investigated from the perspective of PHTP.This paper explores the TSB of LIBs under different SOCs at both cell and module levels for the first time,which has great significance in guiding the thermal safety design of battery systems.

Pixelated non-volatile programmable photonic integrated circuits with 20-level intermediate states

Multi-level programmable photonic integrated circuits(PICs)and optical metasurfaces have gained widespread attention in many fields,such as neuromorphic photonics,opticalcommunications,and quantum information.In this paper,we propose pixelated programmable Si_(3)N_(4)PICs with record-high 20-level intermediate states at 785 nm wavelength.Such flexibility in phase or amplitude modulation is achieved by a programmable Sb_(2)S_(3)matrix,the footprint of whose elements can be as small as 1.2μm,limited only by the optical diffraction limit of anin-house developed pulsed laser writing system.We believe our work lays the foundation for laser-writing ultra-high-level(20 levels and even more)programmable photonic systems and metasurfaces based on phase change materials,which could catalyze diverse applications such as programmable neuromorphic photonics,biosensing,optical computing,photonic quantum computing,and reconfigurable metasurfaces.

Semi-analytical solution for drained expansion analysis of a hollow cylinder of critical state soils

The expansion of a thick-walled hollow cylinder in soil is of non-self-similar nature that the stress/deformation paths are not the same for different soil material points.As a result,this problem cannot be solved by the common self-similar-based similarity techniques.This paper proposes a novel,exact solution for rigorous drained expansion analysis of a hollow cylinder of critical state soils.Considering stress-dependent elastic moduli of soils,new analytical stress and displacement solutions for the nonself-similar problem are developed taking the small strain assumption in the elastic zone.In the plastic zone,the cavity expansion response is formulated into a set of first-order partial differential equations(PDEs)with the combination use of Eulerian and Lagrangian descriptions,and a novel solution algorithm is developed to efficiently solve this complex boundary value problem.The solution is presented in a general form and thus can be useful for a wide range of soils.With the new solution,the non-self-similar nature induced by the finite outer boundary is clearly demonstrated and highlighted,which is found to be greatly different to the behaviour of cavity expansion in infinite soil mass.The present solution may serve as a benchmark for verifying the performance of advanced numerical techniques with critical state soil models and be used to capture the finite boundary effect for pressuremeter tests in small-sized calibration chambers.

Association of DNA methylation/demethylation with the functional outcome of stroke in a hyperinflammatory state

Inflammation is closely related to stroke prognosis, and high inflammation status leads to poor functional outcome in stroke. DNA methylation is involved in the pathogenesis and prognosis of stroke. However, the effect of DNA methylation on stroke at high levels of inflammation is unclear. In this study, we constructed a hyperinflammatory cerebral ischemia mouse model and investigated the effect of hypomethylation and hypermethylation on the functional outcome. We constructed a mouse model of transient middle cerebral artery occlusion and treated the mice with lipopolysaccharide to induce a hyperinflammatory state. To investigate the effect of DNA methylation on stroke, we used small molecule inhibitors to restrain the function of key DNA methylation and demethylation enzymes. 2,3,5-Triphenyltetrazolium chloride staining, neurological function scores, neurobehavioral tests, enzyme-linked immunosorbent assay, quantitative reverse transcription PCR and western blot assay were used to evaluate the effects after stroke in mice. We assessed changes in the global methylation status by measuring DNA 5-mc and DNA 5-hmc levels in peripheral blood after the use of the inhibitor. In the group treated with the DNA methylation inhibitor, brain tissue 2,3,5-triphenyltetrazolium chloride staining showed an increase in infarct volume, which was accompanied by a decrease in neurological scores and worsening of neurobehavioral performance. The levels of inflammatory factors interleukin 6 and interleukin-1 beta in ischemic brain tissue and plasma were elevated, indicating increased inflammation. Related inflammatory pathway exploration showed significant overactivation of nuclear factor kappa B. These results suggested that inhibiting DNA methylation led to poor functional outcome in mice with high inflammation following stroke. Further, the effects were reversed by inhibition of DNA demethylation. Our findings suggest that DNA methylation regulates the inflammatory response in stroke and has an important role in the functional outcome of hyperinflammatory stroke.

Modeling time-dependent mechanical behavior of hard rock considering excavation-induced damage and complex 3D stress states

To investigate the long-term stability of deep rocks,a three-dimensional(3D)time-dependent model that accounts for excavation-induced damage and complex stress state is developed.This model comprises three main components:a 3D viscoplastic isotropic constitutive relation that considers excavation damage and complex stress state,a quantitative relationship between critical irreversible deformation and complex stress state,and evolution characteristics of strength parameters.The proposed model is implemented in a self-developed numerical code,i.e.CASRock.The reliability of the model is validated through experiments.It is indicated that the time-dependent fracturing potential index(xTFPI)at a given time during the attenuation creep stage shows a negative correlation with the extent of excavationinduced damage.The time-dependent fracturing process of rock demonstrates a distinct interval effect of the intermediate principal stress,thereby highlighting the 3D stress-dependent characteristic of the model.Finally,the influence of excavation-induced damage and intermediate principal stress on the time-dependent fracturing characteristics of the surrounding rocks around the tunnel is discussed.

Whole-process case management effects on mental state and selfcare ability in patients with liver cancer

BACKGROUND Regarding the incidence of malignant tumors in China,the incidence of liver cancer ranks fourth,second only to lung,gastric,and esophageal cancers.The case fatality rate ranks third after lung and cervical cancer.In a previous study,the whole-process management model was applied to patients with breast cancer,which effectively reduced their negative emotions and improved treatment adherence and nursing satisfaction.METHODS In this single-center,randomized,controlled study,60 randomly selected patients with liver cancer who had been admitted to our hospital from January 2021 to January 2022 were randomly divided into an observation group(n=30),who received whole-process case management on the basis of routine nursing mea-sures,and a control group(n=30),who were given routine nursing measures.We compared differences between the two groups in terms of anxiety,depression,the level of hope,self-care ability,symptom distress,sleep quality,and quality of life.RESULTS Post-intervention,Hamilton anxiety scale,Hamilton depression scale,memory symptom assessment scale,and Pittsburgh sleep quality index scores in both groups were lower than those pre-intervention,and the observation group had lower scores than the control group(P<0.05).Herth hope index,self-care ability assessment scale-revision in Chinese,and quality of life measurement scale for patients with liver cancer scores in both groups were higher than those pre-intervention,with higher scores in the observation group compared with the control group(P<0.05).CONCLUSION Whole-process case management can effectively reduce anxiety and depression in patients with liver cancer,alleviate symptoms and problems,and improve the level of hope,self-care ability,sleep quality,and quality of life,as well as provide feasible nursing alternatives for patients with liver cancer.

Geometric Error Identification of Gantry-Type CNC Machine Tool Based on Multi-Station Synchronization Laser Tracers

Laser tracers are a three-dimensional coordinate measurement system that are widely used in industrial measurement.We propose a geometric error identification method based on multi-station synchronization laser tracers to enable the rapid and high-precision measurement of geometric errors for gantry-type computer numerical control(CNC)machine tools.This method also improves on the existing measurement efficiency issues in the single-base station measurement method and multi-base station time-sharing measurement method.We consider a three-axis gantry-type CNC machine tool,and the geometric error mathematical model is derived and established based on the combination of screw theory and a topological analysis of the machine kinematic chain.The four-station laser tracers position and measurement points are realized based on the multi-point positioning principle.A self-calibration algorithm is proposed for the coordinate calibration process of a laser tracer using the Levenberg-Marquardt nonlinear least squares method,and the geometric error is solved using Taylor’s first-order linearization iteration.The experimental results show that the geometric error calculated based on this modeling method is comparable to the results from the Etalon laser tracer.For a volume of 800 mm×1000 mm×350 mm,the maximum differences of the linear,angular,and spatial position errors were 2.0μm,2.7μrad,and 12.0μm,respectively,which verifies the accuracy of the proposed algorithm.This research proposes a modeling method for the precise measurement of errors in machine tools,and the applied nature of this study also makes it relevant both to researchers and those in the industrial sector.

Preparation of entangledW states based on the cavity QED system

We present a qubit-loss-free(QLF)fusion scheme for generating large-scale atom W states in cavity quantum electrodynamics(QED)system.Compared to the most current fusion schemes which are conditioned on the case where one particle can be extracted from each initial W state to the fusion process,our scheme will access one or two particles from each W state.Based on the atom–cavity-field detuned interaction,three jWin+m+t states can be generated from the jWin,jWim,and jWit states with the help of two auxiliary atoms,and three jWin+m+t+q states can be generated from jWin,jWim,jWit,and a jWiq state with the help of three auxiliary atoms.Comparing the numerical simulations of the resource cost of fusing three small-size W states based on the previous schemes,our fusion scheme seems to be more efficient.This QLF fusion scheme can be generalized to the case of fusing k different or identical particle W states.Furthermore,with no qubit loss,it greatly reduces the number of fusion steps and prepares W states with larger particle numbers.

Entropy variances of pure coherent states in the diffusion channel

Using the operator correspondence of the real and fictious modes in the thermo entangled state representation, wesolve the quantum master equation describing the diffusion channel and obtain the Kraus operator-sum representation ofits analytical solution. we find that the pure coherent states evolve into the new mixed thermal superposed states in thediffusion channel. Also, we investigate the statistical properties of the initial coherent states and their entropy evolutions inthe diffusion channel, and find that the entropy evolutions are only related to the decay time and without the amplitudes ofthe initial coherent states.

Machine-learning-assisted efficient reconstruction of the quantum states generated from the Sagnac polarization-entangled photon source

Neural networks are becoming ubiquitous in various areas of physics as a successful machine learning(ML)technique for addressing different tasks.Based on ML technique,we propose and experimentally demonstrate an efficient method for state reconstruction of the widely used Sagnac polarization-entangled photon source.By properly modeling the target states,a multi-output fully connected neural network is well trained using only six of the sixteen measurement bases in standard tomography technique,and hence our method reduces the resource consumption without loss of accuracy.We demonstrate the ability of the neural network to predict state parameters with a high precision by using both simulated and experimental data.Explicitly,the mean absolute error for all the parameters is below 0.05 for the simulated data and a mean fidelity of 0.99 is achieved for experimentally generated states.Our method could be generalized to estimate other kinds of states,as well as other quantum information tasks.

Target layer state estimation in multi-layer complex dynamical networks considering nonlinear node dynamics

In many engineering networks, only a part of target state variables are required to be estimated.On the other hand,multi-layer complex network exists widely in practical situations.In this paper, the state estimation of target state variables in multi-layer complex dynamical networks with nonlinear node dynamics is studied.A suitable functional state observer is constructed with the limited measurement.The parameters of the designed functional observer are obtained from the algebraic method and the stability of the functional observer is proven by the Lyapunov theorem.Some necessary conditions that need to be satisfied for the design of the functional state observer are obtained.Different from previous studies, in the multi-layer complex dynamical network with nonlinear node dynamics, the proposed method can estimate the state of target variables on some layers directly instead of estimating all the individual states.Thus, it can greatly reduce the placement of observers and computational cost.Numerical simulations with the three-layer complex dynamical network composed of three-dimensional nonlinear dynamical nodes are developed to verify the effectiveness of the method.

Topological edge and corner states of valley photonic crystals with zipper-like boundary conditions

We present a stable valley photonic crystal(VPC)unit cell with C_(3v)symmetric quasi-ring-shaped dielectric columns and realize its topological phase transition by breaking mirror symmetry.Based on this unit cell structure,topological edge states(TESs)and topological corner states(TCSs)are realized.We obtain a new type of wave transmission mode based on photonic crystal zipper-like boundaries and apply it to a beam splitter assembled from rectangular photonic crystals(PCs).The constructed beam splitter structure is compact and possesses frequency separation functions.In addition,we construct a box-shaped triangular PC structures with zipper-like boundaries and discover phenomena of TCSs in the corners,comparing its corner states with those formed by other boundaries.Based on this,we explore the regularities of the electric field patterns of TESs and TCSs,explain the connection between the characteristic frequencies and locality of TCSs,which helps better control photons and ensures low power consumption of the system.

Predicted Critical State Based on Invariance of the Lyapunov Exponent in Dual Spaces

Critical states in disordered systems,fascinating and subtle eigenstates,have attracted a lot of research interests.However,the nature of critical states is difficult to describe quantitatively,and in general,it cannot predict a system that hosts the critical state.We propose an explicit criterion whereby the Lyapunov exponent of the critical state should be 0 simultaneously in dual spaces,namely the Lyapunov exponent remains invariant under the Fourier transform.With this criterion,we can exactly predict a one-dimensional quasiperiodic model which is not of self-duality,but hosts a large number of critical states.Then,we perform numerical verification of the theoretical prediction and display the self-similarity of the critical state.Due to computational complexity,calculations are not performed for higher dimensional models.However,since the description of extended and localized states by the Lyapunov exponent is universal and dimensionless,utilizing the Lyapunov exponent of dual spaces to describe critical states should also be universal.Finally,we conjecture that some kind of connection exists between the invariance of the Lyapunov exponent and conformal invariance,which can promote the research of critical phenomena.

Empowering neuromorphic computing with topological states

In 1980,scientist Klaus von Klitzing discovered the quantum Hall effect[1],a groundbreaking achievement that earned him the Nobel Prize in Physics in 1985.This discovery was a significant milestone in condensed matter physics,representing the first identification of topological quantum states.

Screening of Rodents from Rural Communities of Lagos and Ogun State Nigeria of Lassa Virus RNA

Background: In recent times, there has been an increase in the number of Lassa fever cases resulting from the several episodes of Lassa fever epidemics ravaging Nigeria and other West African countries. The presence of Lassa virus in rodents other than the major reservoir (Mastomys natalensis) has been a public health concern as to the actual burden of the disease. It is therefore of a public health necessity to explore the LASV RNA habouring potential of several species of rodents in endemic as well as non-endemic areas for proper prevention of emergence of outbreaks in non-endemic areas. Objectives: The aim of this study was to detect the presence of LASV RNA in different species of rodents in Ikorodu, Lagos state and Abeokuta, Ogun state. Methods: A total of ninety one (91) rodents were captured from Ikorodu, Lagos State (61 rodents) and Abeokuta, Ogun State (30 rodents), euthanized, bled, and plasma obtained for the detection of LASV RNA by Reverse Transcriptase Polymerase Chain Reaction. Results: A total of 91 rodents consisting of 77 Rattus rattus and 14 Crocidura spp. The S segment of LASV RNA was not in any of the 91 rodents’ plasma samples. Conclusion: The rodents captured within the rural communities of Ikorodu, Lagos State and Abeokuta, Ogun State were found not to habour the LASV RNA. This study is limited by the relatively small sample size. Similar studies should be encouraged both in endemic and non-endemic areas in order to understand the actual burden of Lassa fever as well as put into check future epidemics.