Efficient Penetration Testing Path Planning Based on Reinforcement Learning with Episodic Memory

Intelligent penetration testing is of great significance for the improvement of the security of information systems,and the critical issue is the planning of penetration test paths.In view of the difficulty for attackers to obtain complete network information in realistic network scenarios,Reinforcement Learning(RL)is a promising solution to discover the optimal penetration path under incomplete information about the target network.Existing RL-based methods are challenged by the sizeable discrete action space,which leads to difficulties in the convergence.Moreover,most methods still rely on experts’knowledge.To address these issues,this paper proposes a penetration path planning method based on reinforcement learning with episodic memory.First,the penetration testing problem is formally described in terms of reinforcement learning.To speed up the training process without specific prior knowledge,the proposed algorithm introduces episodic memory to store experienced advantageous strategies for the first time.Furthermore,the method offers an exploration strategy based on episodic memory to guide the agents in learning.The design makes full use of historical experience to achieve the purpose of reducing blind exploration and improving planning efficiency.Ultimately,comparison experiments are carried out with the existing RL-based methods.The results reveal that the proposed method has better convergence performance.The running time is reduced by more than 20%.

Linked Data-based Slide Repository： The Episodic Slide Retrieval Using the Episodic Keyword Networks

This paper focuses on developing a system that allows presentation authors to effectively retrieve presentation slides for reuse from a large volume of existing presentation materials. We assume episodic memories of the authors can be used as contextual keywords in query expressions to efficiently dig out the expected slides for reuse rather than using only the part-of-slide-descriptions-based keyword queries. As a system, a new slide repository is proposed, composed of slide material collections, slide content data and pieces of information from authors＇ episodic memories related to each slide and presentation together with a slide retrieval application enabling authors to use the episodic memories as part of queries. The result of our experiment shows that the episodic memory-used queries can give more discoverability than the keyword-based queries. Additionally, an improvement model is discussed on the slide retrieval for further slide-finding efficiency by expanding the episodic memories model in the repository taking in the links with the author-and-slide-related data and events having been post on the private and social media sites.

Sequence Thickness and its Response to Episodic Tectonic Evolution in Paleogene Qikou Sag, Bohaiwan Basin

The sequence stratigraphy framework of Paleogene of Qikou Sag has been built in this paper. The evolution law of episodic subsidence in Qikou Sag has been expounded, with the analysis of the characteristic of sequence thickness, as well as the study of consedimental tectonic activity, subsidence history, paleogeomorphology and the coupling relationship among them. The faults controlling action was the dominant mechanism during rifting stage I , as the depression was the dominant mechanism during rifting stage III, which embodies a process from intense disparity fault depression to integral subsidence. Meanwhile, under the control of regional faulting and later stage depression, subsidence rate centers and paleogeomorphic concave centers are always corresponding to the sequence thickness centers, and the changes of subsidence centers and paleogeomorphology concave centers are consistent with the migration of thickness centers, which presents the restriction, coupling and response among them and clarifies the law that tectonic activity controls the basin fill by means of controlling the subsidence rate and paleogeomorphology from contributing factor angle. On this basis, the dynamic evolution of the composite pattern of structure-subsidence-deposition in the fault controlling actic region during the three rifting stages and their petroleum geological significance has been discussed.

On the Geodynamic Mechanism of Episodic Uplift of the Tibetan Plateau during the Cenozoic Era

Multi-stage uplift of the Tibetan Plateau during the Cenozoic implies a complex geodynamic process.In this paper,we review main geodynamic models for the uplift of the plateau,and,in particular,analyze the spatio-temporal framework of the Cenozoic deformation structures,which are closely related to the deep geodynamic mechanism for the plateau uplift.From this perspective,significant change of the deformation regime over the Tibetan Plateau occurred by the middle-late Miocene,while thrust and thrust-folding system under NS compression was succeded by extension or stress-relaxation.Meanwhile,a series of large-scale strike-slip faults commenced or was kinemtically reversed.Based on a systematic synthesis of the structure deformation,magmatism,geomorphological process and geophysical exploration,we propose a periodical model of alternating crustal compression and extension for episodic uplift of the Tibetan Plateau.

Episodic Carbonate Deposits on the Triassic Continental Slope in Southern China

Abstract Episodic carbonate deposits on the Triassic continental slope in southern China are mainly composed of gravity-flow limestones and contourite limestones. Gravity-flow limestones were well developed in the lower and middle Yangtze area in the Early Triassic and in the Yunnan-Guizhou-Guangxi area in the Early and Middle Triassic. Five fundamental types of gravity-flow limestones are recognized: slide limestone, debris-flow limestone, grain-flow limestone, turbidite limestone and rockfall limestone. They form six types of assemblage beds: slide-debris-flow limestones, slide-debris-flow-turbidite limestone, slide-debris-flow-grain-flow-turbidite limestone, rockfall-debris-flow limestone, debris-flow-turbidite limestone, and debris-flow-grain-flow-turbidite limestone. The first two were formed mainly in the Early Triassic slopes. The Middle Triassic slopes were characterized by widespread rockfall limestone. Growth faults, storms, earthquakes and oversteepened slopes are considered to be the probable triggers of the gravity flows.

Base-Level Cycles and Episodic Coal Accumulation——Case Study of Dongsheng Coalfield in Ordos Basin

The advantage of high-resolution sequence stratigraphy, which takes base-levels as reference, is that it can be applied to continental depositional basins controlled by multiple factors and can effectively improve the accuracy and resolution of sequential stratigraphic analysis. Moreover, the principles of base-level cycles are also suitable for analyzing sequential stratigraphy in continental coal-bearing basins because of their accuracy in forecasting distribution of coal measures. By taking the Dongsheng coalfield in the Ordos basin as an example, the extensive application of base-level cycles in exploration and exploitation of coal is analyzed. The result shows that the Yan’an formation in the Dongsheng area is a long-term base-level cycle which is bordered by nonconformities and made up of five mid-term cycles and 13 short-term cycles. The long-term cycle and the mid-term cycles are obvious in comparison with a transverse profile. The episodic coal accumulation in the Mesozoic Ordos basin means that the deposition of primary matter (peat bogs) of coalification is discontinuous, periodical and cyclical in the evolution of the basin. The episodic accumulation of coal measures in the Yan’an stage is controlled by ascending-descending changes of a long-term cycle and middle-term cycles. Coal measures formed during the early and late periods of the long-term cycle are characterized by multiple layers, big cumulative thickness and poor continuity. Coal measures formed in the mid-term of the long cycle are dominated by good continuity, fewer layers and a small additive thickness, which is favorable for the accumulation of thick and continuous coal measures in the transition stage of mid term base-level cycles.

Measuring episodic abdominal pain and disability in suspected sphincter of Oddi dysfunction

AIM:To evaluate the reliability of an instrument that measures disability arising from episodic abdominal pain in patients with suspected sphincter of Oddi dysfunction(SOD).METHODS:Although several treatments have been utilized to reduce pain and associated disability,measurement tools have not been developed to reliably track outcomes.Two pilot studies were conducted to assess test-retest reliability of a newly developed instrument,the recurrent abdominal pain intensity and disability(RAPID) instrument.The RAPID score is a 90-d summation of days where productivity for various daily activities is reduced as a result of abdominal pain episodes,and is modeled after the migraine disability assessment instrument used to measure headache-related disability.RAPID was administered by telephone on 2 consecutive occasions in 2 consenting populations with suspected SOD:a pre-sphincterotomy population(Pilot Ⅰ,n = 55) and a post-sphincterotomy population(Pilot Ⅱ,n = 70).RESULTS:The average RAPID scores for Pilots Ⅰ and Ⅱ were:82 d(median:81.5 d,SD:64 d) and 48 d(median:0 d,SD:91 d),respectively.The concordance between the 2 assessments for both populations was very good:0.81 for the pre-sphincterotomy population and 0.95 for the post-sphincterotomy population.CONCLUSION:The described pilot studies suggest that RAPID is a reliable instrument for measuring disability resulting from abdominal pain in suspected SOD patients.

Potential Effects of Episodic Deposition on Nutrients and Heavy Metals in Decomposing Litters of Suaeda glauca in Salt Marsh of the Yellow River Estuary, China

Episodic deposition has been recognized as a major factor affecting the decomposition rate of detrital material in salt marshes. In this paper, three one-off burial treatments, no burial treatment(0 cm, NBT), current burial treatment(10 cm, CBT) and strong burial treatment(20 cm, SBT), were designed in intertidal zone of the Yellow River Estuary to determine the potential influences of episodic deposition on nutrient(C, N) and heavy metal(Pb, Cr, Cu, Zn, Ni, Mn, Cd, V and Co) variations in decomposing litters of Suaeda glauca. Results showed that although various burial treatments showed no statistical difference in decomposition rate of S. glauca, the values generally followed the sequence of CBT(0.002 403/d) > SBT(0.002 195/d) > NBT(0.002 060/d). The nutrients and heavy metals in decomposing litters of the three burial treatments exhibited different variations except for N, Cu, Cr, Ni and Co. Except for Mn, no significant differences in C, N, Pb, Cr, Cu, Zn, Ni, V and Co concentrations occurred among the three treatments(P > 0.05). With increasing burial depth, Cr and Cd levels generally increased while Cu, Ni and Mn concentrations decreased. Although episodic deposition was generally favorable for C and N release from S. glauca, its influence on release was insignificant. In the three burial treatments, Pb, Cr, Zn, Ni, Mn, V and Co stocks in S. glauca generally evidenced the export of metals from litter to environment, and, with increasing burial depth, the export amounts increased greatly. The S. glauca were particular efficient in binding Cd and releasing Pb, Cr, Zn, Ni, Mn, V and Co, and, with increasing burial depth, stocks of Cu in decomposing litters generally shifted from release to accumulation. The experiment indicated that the potential eco-toxic risk of Pb, Cr, Zn, Ni, Mn, V and Co exposure would be serious as the strong burial episodes occurred in S. glauca marsh.

Fracturing and Episodic Fluid Expulsion in Pressure Compartments

The fracturing of the pressure compartments in sedimentary basins may not be caused completely by natural hydraulic fracturing. On the bases of fracture mechanics and numerical simulation, we consider that there are two mechanisms concerning the fracturing of pressure compartments, which are as follows: (1) natural hydraulic fracturing of the sediments within pressure compartments, and (2) tensile fracturing due to tangential traction. When the fracture, formed in the sediments within pressure compartments due to hydraulic fracturing, arrives at the base of the seal, the intensely tangential tensile stresses are created and lead to the fracturing of the seal. After the seal fractured, the fluid escaped from the pressure compartments. In a shallow pressure compartment, the duration of fracture opening and fluid expulsion is about 10-20 years in a cycle.

Precambrian Mantle Characteristics as Recorded by the Episodic Mafic Magmatism in the Liaodong Peninsula

1 Introduction Mafic rocks are widespread throughout the Liaodong Peninsula and vicinity,northeastern North China Craton,providing important constraints on their mantle source characteristics of individual episodes,as they have

Numerical Modeling of Episodic Compaction and Its Efecting Parameters in Geopresured Shales

ＮｕｍｅｒｉｃａｌＭｏｄｅｌｉｎｇｏｆＥｐｉｓｏｄｉｃＣｏｍｐａｃｔｉｏｎａｎｄＩｔｓＥｆｅｃｔｉｎｇＰａｒａｍｅｔｅｒｓｉｎＧｅｏｐｒｅｓｕｒｅｄＳｈａｌｅｓ＊ＸｉｅＸｉｎｏｎｇＦａｃｕｌｔｙｏｆＥａｒｔｈＲｅｓｏｕｒｃｅｓ，ＣｈｉｎａＵｎｉｖｅｒ...

Frequent Episodic Vertigo is an Unexpected Side Effect of Flutamide

An 82-year-old male suffered from prostatic cancer five years ago. Since then, he has taken flutamide and was bothered with episodic vertigo (EV) every morning. In order to treat prostatic cancer, flutamide was not discontinued, but conservative treatment and life-style change were recommended. Finally, EV actually subsided. Herein, we report the rare case, in which EV was an unexpected side effect of flutamide. Herein we review his whole history, physical examination, vestibular function test, electronystagmogram, caloric test, awake encephalogram, blood examinations, color-coded duplex ultrasonogram and magnetic resonance imaging/angiogram to suggest a mechanism of flutamide responsible for EV.

Episodic Learning and Memory in Prodromal Huntington’s Disease: The Role of Multimodal Encoding and Selective Reminding

This study investigated episodic memory in prodromal HD. Three groups were compared (N = 70): mutation carriers with less than 12.5 years to disease onset (n = 16), mutation carriers with 12.5 or more years to disease onset (n = 16), and noncarriers (n = 38). Episodic memory was assessed using the Fuld Object Memory Evaluation, which included multimodal presentation and selective reminding, and the Claeson-Dahl Learning Test which included verbal repeated presentation and recall trials. Both carrier groups demonstrated deficient episodic memory compared to noncarriers. The results suggest deficient episodic memory in prodromal HD, and that inconsistent retrieval contributes to these deficits. Multimodal presentation attenuates the deficits.

Promotion of structural plasticity in area V2 of visual cortex prevents against object recognition memory deficits in aging and Alzheimer's disease rodents

Memory deficit,which is often associated with aging and many psychiatric,neurological,and neurodegenerative diseases,has been a challenging issue for treatment.Up till now,all potential drug candidates have failed to produce satisfa ctory effects.Therefore,in the search for a solution,we found that a treatment with the gene corresponding to the RGS14414protein in visual area V2,a brain area connected with brain circuits of the ventral stream and the medial temporal lobe,which is crucial for object recognition memory(ORM),can induce enhancement of ORM.In this study,we demonstrated that the same treatment with RGS14414in visual area V2,which is relatively unaffected in neurodegenerative diseases such as Alzheimer s disease,produced longlasting enhancement of ORM in young animals and prevent ORM deficits in rodent models of aging and Alzheimer’s disease.Furthermore,we found that the prevention of memory deficits was mediated through the upregulation of neuronal arbo rization and spine density,as well as an increase in brain-derived neurotrophic factor(BDNF).A knockdown of BDNF gene in RGS14414-treated aging rats and Alzheimer s disease model mice caused complete loss in the upregulation of neuronal structural plasticity and in the prevention of ORM deficits.These findings suggest that BDNF-mediated neuronal structural plasticity in area V2 is crucial in the prevention of memory deficits in RGS14414-treated rodent models of aging and Alzheimer’s disease.Therefore,our findings of RGS14414gene-mediated activation of neuronal circuits in visual area V2 have therapeutic relevance in the treatment of memory deficits.

Episodic rifting of continental marginal basins and tectonic events since 10 Ma in the South China Sea

Integrated study on the tectonic, basin filling and thermal evolution of the Tertiary basins on the northern margin of the South China Sea has indicated that the rift continental margin on the northern South China Sea is not a passive margin and that the western margin is characteristic of the transformextension. Episodic rifting and the thermal events since 10 Ma in the area have been documented by the integrated analysis of dynamic process. It has been clarified that the tectono-thermal events have exerted a significant influence on basin features and hydrocarbon accumulation, and particularly, the westward collision of the Lusong Island Arc, the generation of densely spaced faults in the Pearl River Mouth Basin, the rapid subsidence, high heat flow and large-scale over pressure and thermal-fluid breakthrough in the Yinggehai and the Qiongdongnan basins have proved to be the important factors determining the formation of oil and gas in these basins.

Mechanism of diapirism and episodic fluid injections in the Yinggehai Basin

The diapirism in the Yinggehai Basin is a combined result of strong overpressure caused by rapid sedimentation of fine-grain sediments and the tensile stress field resulting from right-lateral slip of the boundary-fault. The diapirism showed multiple-stage, episodic nature, and caused intermittent counter-direction onlaps and changes in the thickness of strata. The shallow gas reservoirs in the diapir structural zone displayed obvious inter-reservoir compositional heterogeneities, and their filling history could be divided into 4 stages, with gases injected during different stages having different hydrocarbon gas, CO2 and N2 contents and different stable isotope compositions. The multiple-episode, intermittent activities of the diapirs, multiple-stage, non-continuous injections of fluids, and the transient thermal effect of fluid flow as well as the strong migration fractionation reflected episodic fluid injection and natural gas accumulation.

Two episodic changes of trace fossils through the Permian-Triassic transition in the Meishan cores,Zhejiang Province

Six ichnogenera,Chondrites,Palaeophycus,Planolites,Rhizocorallium,Skolithos,and Thalassinoides,were identified from the drilling cores near the Global Stratotype Section and Point(GSSP) of Permian-Triassic boundary at Meishan,Zhejiang Province.The ichnological indices,including abundance,bioturbation index,and disturbed depth of trace fossils,show two episodes of intense changes near the Permian-Triassic Boundary(PTB).Episode I occurred in Beds 25-27b when the ecologically complicate forms such as Chondrites,Skolithos,Rhizocorallium,and Thalassinoides disappeared hereafter,the bioturbation index reduced from 1-5 to 1-3,and the disturbed depth declined from 5-66 to 2-5 cm.Episode II took place at the base of Bed 33 with the disappearance of Palaeophycus and Planolites,and subsequent absence of trace fossils and bioturbation till the middle-upper part of Bed 41 when the disturbed structures reoccurred,but they are only tiny Planolites and the bioturbation index was never higher than 3 and the disturbed depth less than 4 mm.Episode I shows an intense change,corresponding to the main stage of the end-Permian mass extinction,whereas Episode II is relatively weak,corresponding to the epilogue of the mass extinction of trace makers in the Early Triassic.Subsequently,ichnofossils were dominated by surface tracks in simple ecological habit and structures.This phenomenon indicates that the Early Triassic benthonic fauna is changed from sessile benthic system to mobile benthic system after the end-Permian mass extinction.In other words,the evolution of the trace fossils across the Permian-Triassic transition had an episodic process similar to the body fossils.In addition,the change of ichnofabrics is well coincided with the negative excursion of carbon isotopes and the expansion of cyanobacteria.As the results of physical and biogenic processes,trace fossils provided unique materials for the study of the biotic and environmental events,as well as their coupling evolution through the great Permian-Triassic transition.

Episodic fluid movements in superimposed basin:Combined evidence from outcrop and fluid inclusions of the Majiang ancient oil reservoir,Guizhou Province

Combined evidence from the outcrop and the fluid inclusion assemblage (FIA) analysis indicates that there exist two episodes of fluid flow controlled by the tectonic activity. The first episode was recorded mainly in the reservoir rock of the Honghuayuan Formation,representing the fluid flow of hydrocarbon charging. The second episode occurred mainly along the fault systems,representing the fluid move-ment when the ancient oil reservoir was destroyed. The host mineral morphology,homogeneous tem-perature,and salinity of the FIAs record an episodic fluid movement. Characters of high homogenous temperature,low salinity and a quick temperature variation of the first episode fluid flow may indicate an early-stage fluid eruption,and correspondingly,fine-grained calcite was formed. Temperature of the erupted fluid tended to decrease during its mixing with the upper formation fluid and finally had the same temperature as the upper formation. From then on,the temperature was rather steady and fa-vored the growth of the coarse calcite. Due to this character of the temperature variation of the episodic fluid flow,we can use the homogenous temperature of the FIA of the coarse calcite to date the forma-tion and the destruction time of the Majiang ancient oil reservoir. Episodic fluid flow was known for its inhomogeneous trapping,which resulted in the failure of dating according to the burial history. But taking a close look at its temperature variation,we think that the latest stage of fluid flow,characterized by steady state temperature and grow of the coarser crystals,can be used for dating. It will be of great value if this method is proved to be effective. The formation and the destruction time of the Majiang ancient oil reservoir were dated to be in the Indosinian Period and the late Yanshan-early Hymalayan Period respectively. This conclusion is in great discrepancy with the common accepted idea that the Majiang ancient oil reservoir was formed and destroyed during the Caledonian tectonic movement. Even so,this paper further discussed its reliability from the view of the source rock evolution and also the local tectonic evolution.

Porphyry Cu deposits linked to episodic growth of an underlying parental magma chamber

Saindak is one of the typical porphyry Cu deposits(PCDs)in the Chagai magmatic arc in Pakistan.Ore-forming porphyries at Saindak PCD are mainly composed of tonalite.Here,we use geochemistry of apatite enclosed in plagioclase phenocrysts from the ore-forming tonalite to constrain the releasing and recharging processes of S and Cl in the underlying parental magma chamber during PCD mineralization.Although apatite inclusions have homogeneous intra-grain S and Cl compositions,there is significant inter-grain S and Cl variations in apatite inclusions located from core to rim in the hosting plagioclase.Such inter-grain S and Cl variation in apatites are coupled with the core-to-rim trends of An,FeO and Mg contents of the hosting plagioclase phenocryst.It indicates that the Saindak PCD likely formed by episodic injection of primitive magmas during the growth of an underlying magma chamber,rather than by one major injection or by addition of mafic melt derived from different source region.Each primitive melt injection introduced essential ore-forming materials such as S and Cl,which were rapidly and effectively released to the coexisting fluids,causing mineralization.Once primitive melt injection stops,signaling the end of growth of underlying magma chamber,mineralization will cease quickly although the hydrothermal system can still survive for a long time.However,the later released fluids are relatively depleted in ore-forming materials,and thus have lower capability to generate mineralization.Accordingly,predominant porphyry-type mineralizations occurred during the growth rather than waning stage of a magmatic system.

Motilin and ghrelin gene experienced episodic evolution during primitive placental mammal evolution

Motilin and ghrelin,members of a structure-function-related hormone family,play important roles in gastrointestinal function,regulation of energy homeostasis and growth hormone secretion.We observed episodic evolution in both of their prehormone gene sequences during primitive placental mammal evolution,during which most of the nonsynonymous changes result in radical substitution.Of note,a functional obestatin hormone might have only originated after this episodic evolution event.Early in placental mammal evolution,a series of biology complexities evolved.At the same time the motilin and ghrelin prehormone genes,which play important roles in several of these processes,experienced episodic evolution with dramatic changes in their coding sequences.These observations suggest that some of the lineage-specific physiological adaptations are due to episodic evolution of the motilin and ghrelin genes.