Mastering air combat game with deep reinforcement learning

Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem of plasticity loss in neural networks.Plasticity loss is the difficulty of learning new knowledge after the network has converged.To this end,we propose a motivational curriculum learning distributed proximal policy optimization(MCLDPPO)algorithm,through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods.The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide.Furthermore,a complete tactical maneuver is encapsulated based on the existing air combat knowledge,and through the flexible use of these maneuvers,some tactics beyond human knowledge can be realized.In addition,we designed an interruption mechanism for the agent to increase the frequency of decisionmaking when the agent faces an emergency.When the number of threats received by the agent changes,the current action is interrupted in order to reacquire observations and make decisions again.Using the interruption mechanism can significantly improve the performance of the agent.To simulate actual air combat better,we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously,effectively improving data throughput.The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat,verifying the effectiveness of the algorithmic framework proposed in this paper.

Investigation of Nonlinear PI Multi-loop Control Strategy for Aircraft HVDC Generator System with Wound Rotor Synchronous Machine

In order to enhance the transient performance of aircraft high voltage DC(HVDC)generation system with wound rotor synchronous machine(WRSM)under a wide speed range,the nonlinear PI multi-loop control strategy is proposed in this paper.Traditional voltage control method is hard to achieve the dynamic performance requirements of the HVDC generation system under a wide speed range,so the nonlinear PI parameter adjustment,load current feedback and speed feedback are added to the voltage and excitation current double loop control.The transfer function of the HVDC generation system is derived,and the relationship between speed,load current and PI parameters is obtained.The PI parameters corresponding to the load at certain speed are used to shorten the adjusting time when the load suddenly changes.The dynamic responses in transient processes are analyzed by experiment.The results illustrate that the WRSM HVDC generator system with this method has better dynamic performance.

Green tea polyphenols alleviate di-(2-ethylhexyl)phthalate-induced liver injury in mice

BACKGROUND Di(2-ethylhexyl)phthalate(DEHP)is a common plasticizer known to cause liver injury.Green tea is reported to exert therapeutic effects on heavy metal exposureinduced organ damage.However,limited studies have examined the therapeutic effects of green tea polyphenols(GTPs)on DEHP-induced liver damage.AIM To evaluate the molecular mechanism underlying the therapeutic effects of GTPs on DEHP-induced liver damage.METHODS C57BL/6J mice were divided into the following five groups:Control,model[DEHP(1500 mg/kg bodyweight)],treatment[DEHP(1500 mg/kg bodyweight)+GTP(70 mg/kg bodyweight),oil,and GTP(70 mg/kg bodyweight)]groups.After 8 wk,the liver function,blood lipid profile,and liver histopathology were examined.Differentially expressed micro RNAs(miRNAs)and mRNAs in the liver tissues were examined using high-throughput sequencing.Additionally,functional enrichment analysis and immune infiltration prediction were performed.The miRNA-mRNA regulatory axis was elucidated using the starBase database.Protein expression was evaluated using immunohistochemistry.RESULTS GTPs alleviated DHEP-induced liver dysfunction,blood lipid dysregulation,fatty liver disease,liver fibrosis,and mitochondrial and endoplasmic reticulum lesions in mice.The infiltration of macrophages,mast cells,and natural killer cells varied between the model and treatment groups.mmu-miR-141-3p(a differentially expressed miRNA),Zcchc24(a differentially expressed mRNA),and Zcchc24(a differentially expressed protein)constituted the miRNA-mRNA-protein regulatory axis involved in mediating the therapeutic effects of GTPs on DEHP-induced liver damage in mice.CONCLUSION This study demonstrated that GTPs mitigate DEHP-induced liver dysfunction,blood lipid dysregulation,fatty liver disease,and partial liver fibrosis,and regulate immune cell infiltration.Additionally,an important miRNAmRNA-protein molecular regulatory axis involved in mediating the therapeutic effects of GTPs on DEHP-induced liver damage was elucidated.

Passive leg raising as an indicator of fluid responsiveness in patients with severe sepsis

In the management of critically ill patients, the assessment of volume responsiveness and the decision to administer a fluid bolus constitute a common dilemma for physicians. Static indices of cardiac preload are poor predictors of volume responsiveness. Passive leg raising （PLR） mimics an endogenous volume expansion （VE） that can be used to predict fluid responsiveness. This study was to assess the changes in stroke volume index （SVI） induced by PLR as an indicator of fluid responsiveness in mechanically ventilated patients with severe sepsis. This was a prospective study. Thirty-two mechanically ventilated patients with severe sepsis were admitted for VE in ICU of the First Affiliated Hospital, Zhejiang University School of Medicine and Ningbo Medical Treatment Center Lihuili Hospital from May 2010 to December 2011. Patients with non-sinus rhythm or arrhythmia, parturients, and amputation of the lower limbs were excluded. Measurements of SVI were obtained in a semi-recumbent position （baseline） and during PLR by the technique of pulse indicator continuous cardiac output （PiCCO） system prior to VE. Measurements were repeated after VE （500 mL 6% hydroxyethyl starch infusion within 30 minutes） to classify patients as either volume responders or non-responders based on their changes in stroke volume index （ASVI） over 15%. Heart rate （HR）, systolic artery blood pressure （ABPs）, diastolic artery blood pressure （ABPd）, mean arterial blood pressure （ABPm）, mean central venous pressure （CVPm） and cardiac index （CI） were compared between the two groups. The changes ofABPs, ABPm, CVPm, and SVI after PLR and VE were compared with the indices at the baseline. The ROC curve was drawn to evaluate the value of ASVI and the change of CVPm （ACVPm） in predicting volume responsiveness. SPSS 17.0 software was used for statistical analysis. Among the 32 patients, 22 were responders and 10 were non-responders. After PLR among the responders, some hemodynamic variables （including ABPs, ABPd, ABPm and CVPm） were significantly elevated （101.2±17.6 vs. 118.6±23.7, P=0.03; 52.8±10.7 vs. 64.8±10.7, P=0.006; 68.3±11.7 vs. 81.9±14.4, P=0.008; 6.8±3.2 vs. 11.9±4.0, P=0.001）. After PLR, the area under curve （AUC） and the ROC curve of ASVI and ACVPm for predicting the responsiveness after VE were 0.882±0.061 （95%CI 0.759-1.000） and 0.805±0.079 （95%CI 0.650-0.959） when the cut-off levels of ASVI and ACVPm were 8.8% and 12.7%, the sensitivities were 72.7% and 72.7%, and the specificities were 80% and 80%. Changes in ASVI and ACVPm induced by PLR are accurate indices for predicting fluid responsiveness in mechanically ventilated patients with severe sepsis.