Efficient Penetration Testing Path Planning Based on Reinforcement Learning with Episodic Memory

Intelligent penetration testing is of great significance for the improvement of the security of information systems,and the critical issue is the planning of penetration test paths.In view of the difficulty for attackers to obtain complete network information in realistic network scenarios,Reinforcement Learning(RL)is a promising solution to discover the optimal penetration path under incomplete information about the target network.Existing RL-based methods are challenged by the sizeable discrete action space,which leads to difficulties in the convergence.Moreover,most methods still rely on experts’knowledge.To address these issues,this paper proposes a penetration path planning method based on reinforcement learning with episodic memory.First,the penetration testing problem is formally described in terms of reinforcement learning.To speed up the training process without specific prior knowledge,the proposed algorithm introduces episodic memory to store experienced advantageous strategies for the first time.Furthermore,the method offers an exploration strategy based on episodic memory to guide the agents in learning.The design makes full use of historical experience to achieve the purpose of reducing blind exploration and improving planning efficiency.Ultimately,comparison experiments are carried out with the existing RL-based methods.The results reveal that the proposed method has better convergence performance.The running time is reduced by more than 20%.

Sequence Thickness and its Response to Episodic Tectonic Evolution in Paleogene Qikou Sag, Bohaiwan Basin

The sequence stratigraphy framework of Paleogene of Qikou Sag has been built in this paper. The evolution law of episodic subsidence in Qikou Sag has been expounded, with the analysis of the characteristic of sequence thickness, as well as the study of consedimental tectonic activity, subsidence history, paleogeomorphology and the coupling relationship among them. The faults controlling action was the dominant mechanism during rifting stage I , as the depression was the dominant mechanism during rifting stage III, which embodies a process from intense disparity fault depression to integral subsidence. Meanwhile, under the control of regional faulting and later stage depression, subsidence rate centers and paleogeomorphic concave centers are always corresponding to the sequence thickness centers, and the changes of subsidence centers and paleogeomorphology concave centers are consistent with the migration of thickness centers, which presents the restriction, coupling and response among them and clarifies the law that tectonic activity controls the basin fill by means of controlling the subsidence rate and paleogeomorphology from contributing factor angle. On this basis, the dynamic evolution of the composite pattern of structure-subsidence-deposition in the fault controlling actic region during the three rifting stages and their petroleum geological significance has been discussed.

On the Geodynamic Mechanism of Episodic Uplift of the Tibetan Plateau during the Cenozoic Era

Multi-stage uplift of the Tibetan Plateau during the Cenozoic implies a complex geodynamic process.In this paper,we review main geodynamic models for the uplift of the plateau,and,in particular,analyze the spatio-temporal framework of the Cenozoic deformation structures,which are closely related to the deep geodynamic mechanism for the plateau uplift.From this perspective,significant change of the deformation regime over the Tibetan Plateau occurred by the middle-late Miocene,while thrust and thrust-folding system under NS compression was succeded by extension or stress-relaxation.Meanwhile,a series of large-scale strike-slip faults commenced or was kinemtically reversed.Based on a systematic synthesis of the structure deformation,magmatism,geomorphological process and geophysical exploration,we propose a periodical model of alternating crustal compression and extension for episodic uplift of the Tibetan Plateau.

Episodic Carbonate Deposits on the Triassic Continental Slope in Southern China

Abstract Episodic carbonate deposits on the Triassic continental slope in southern China are mainly composed of gravity-flow limestones and contourite limestones. Gravity-flow limestones were well developed in the lower and middle Yangtze area in the Early Triassic and in the Yunnan-Guizhou-Guangxi area in the Early and Middle Triassic. Five fundamental types of gravity-flow limestones are recognized: slide limestone, debris-flow limestone, grain-flow limestone, turbidite limestone and rockfall limestone. They form six types of assemblage beds: slide-debris-flow limestones, slide-debris-flow-turbidite limestone, slide-debris-flow-grain-flow-turbidite limestone, rockfall-debris-flow limestone, debris-flow-turbidite limestone, and debris-flow-grain-flow-turbidite limestone. The first two were formed mainly in the Early Triassic slopes. The Middle Triassic slopes were characterized by widespread rockfall limestone. Growth faults, storms, earthquakes and oversteepened slopes are considered to be the probable triggers of the gravity flows.

Base-Level Cycles and Episodic Coal Accumulation——Case Study of Dongsheng Coalfield in Ordos Basin

The advantage of high-resolution sequence stratigraphy, which takes base-levels as reference, is that it can be applied to continental depositional basins controlled by multiple factors and can effectively improve the accuracy and resolution of sequential stratigraphic analysis. Moreover, the principles of base-level cycles are also suitable for analyzing sequential stratigraphy in continental coal-bearing basins because of their accuracy in forecasting distribution of coal measures. By taking the Dongsheng coalfield in the Ordos basin as an example, the extensive application of base-level cycles in exploration and exploitation of coal is analyzed. The result shows that the Yan’an formation in the Dongsheng area is a long-term base-level cycle which is bordered by nonconformities and made up of five mid-term cycles and 13 short-term cycles. The long-term cycle and the mid-term cycles are obvious in comparison with a transverse profile. The episodic coal accumulation in the Mesozoic Ordos basin means that the deposition of primary matter (peat bogs) of coalification is discontinuous, periodical and cyclical in the evolution of the basin. The episodic accumulation of coal measures in the Yan’an stage is controlled by ascending-descending changes of a long-term cycle and middle-term cycles. Coal measures formed during the early and late periods of the long-term cycle are characterized by multiple layers, big cumulative thickness and poor continuity. Coal measures formed in the mid-term of the long cycle are dominated by good continuity, fewer layers and a small additive thickness, which is favorable for the accumulation of thick and continuous coal measures in the transition stage of mid term base-level cycles.

Measuring episodic abdominal pain and disability in suspected sphincter of Oddi dysfunction

AIM:To evaluate the reliability of an instrument that measures disability arising from episodic abdominal pain in patients with suspected sphincter of Oddi dysfunction(SOD).METHODS:Although several treatments have been utilized to reduce pain and associated disability,measurement tools have not been developed to reliably track outcomes.Two pilot studies were conducted to assess test-retest reliability of a newly developed instrument,the recurrent abdominal pain intensity and disability(RAPID) instrument.The RAPID score is a 90-d summation of days where productivity for various daily activities is reduced as a result of abdominal pain episodes,and is modeled after the migraine disability assessment instrument used to measure headache-related disability.RAPID was administered by telephone on 2 consecutive occasions in 2 consenting populations with suspected SOD:a pre-sphincterotomy population(Pilot Ⅰ,n = 55) and a post-sphincterotomy population(Pilot Ⅱ,n = 70).RESULTS:The average RAPID scores for Pilots Ⅰ and Ⅱ were:82 d(median:81.5 d,SD:64 d) and 48 d(median:0 d,SD:91 d),respectively.The concordance between the 2 assessments for both populations was very good:0.81 for the pre-sphincterotomy population and 0.95 for the post-sphincterotomy population.CONCLUSION:The described pilot studies suggest that RAPID is a reliable instrument for measuring disability resulting from abdominal pain in suspected SOD patients.

Potential Effects of Episodic Deposition on Nutrients and Heavy Metals in Decomposing Litters of Suaeda glauca in Salt Marsh of the Yellow River Estuary, China

Episodic deposition has been recognized as a major factor affecting the decomposition rate of detrital material in salt marshes. In this paper, three one-off burial treatments, no burial treatment(0 cm, NBT), current burial treatment(10 cm, CBT) and strong burial treatment(20 cm, SBT), were designed in intertidal zone of the Yellow River Estuary to determine the potential influences of episodic deposition on nutrient(C, N) and heavy metal(Pb, Cr, Cu, Zn, Ni, Mn, Cd, V and Co) variations in decomposing litters of Suaeda glauca. Results showed that although various burial treatments showed no statistical difference in decomposition rate of S. glauca, the values generally followed the sequence of CBT(0.002 403/d) > SBT(0.002 195/d) > NBT(0.002 060/d). The nutrients and heavy metals in decomposing litters of the three burial treatments exhibited different variations except for N, Cu, Cr, Ni and Co. Except for Mn, no significant differences in C, N, Pb, Cr, Cu, Zn, Ni, V and Co concentrations occurred among the three treatments(P > 0.05). With increasing burial depth, Cr and Cd levels generally increased while Cu, Ni and Mn concentrations decreased. Although episodic deposition was generally favorable for C and N release from S. glauca, its influence on release was insignificant. In the three burial treatments, Pb, Cr, Zn, Ni, Mn, V and Co stocks in S. glauca generally evidenced the export of metals from litter to environment, and, with increasing burial depth, the export amounts increased greatly. The S. glauca were particular efficient in binding Cd and releasing Pb, Cr, Zn, Ni, Mn, V and Co, and, with increasing burial depth, stocks of Cu in decomposing litters generally shifted from release to accumulation. The experiment indicated that the potential eco-toxic risk of Pb, Cr, Zn, Ni, Mn, V and Co exposure would be serious as the strong burial episodes occurred in S. glauca marsh.

Fracturing and Episodic Fluid Expulsion in Pressure Compartments

The fracturing of the pressure compartments in sedimentary basins may not be caused completely by natural hydraulic fracturing. On the bases of fracture mechanics and numerical simulation, we consider that there are two mechanisms concerning the fracturing of pressure compartments, which are as follows: (1) natural hydraulic fracturing of the sediments within pressure compartments, and (2) tensile fracturing due to tangential traction. When the fracture, formed in the sediments within pressure compartments due to hydraulic fracturing, arrives at the base of the seal, the intensely tangential tensile stresses are created and lead to the fracturing of the seal. After the seal fractured, the fluid escaped from the pressure compartments. In a shallow pressure compartment, the duration of fracture opening and fluid expulsion is about 10-20 years in a cycle.

A Data-Based Feedback Relearning Algorithm for Uncertain Nonlinear Systems

In this paper,a data-based feedback relearning algorithm is proposed for the robust control problem of uncertain nonlinear systems.Motivated by the classical on-policy and off-policy algorithms of reinforcement learning,the online feedback relearning(FR)algorithm is developed where the collected data includes the influence of disturbance signals.The FR algorithm has better adaptability to environmental changes(such as the control channel disturbances)compared with the off-policy algorithm,and has higher computational efficiency and better convergence performance compared with the on-policy algorithm.Data processing based on experience replay technology is used for great data efficiency and convergence stability.Simulation experiments are presented to illustrate convergence stability,optimality and algorithmic performance of FR algorithm by comparison.

Precambrian Mantle Characteristics as Recorded by the Episodic Mafic Magmatism in the Liaodong Peninsula

1 Introduction Mafic rocks are widespread throughout the Liaodong Peninsula and vicinity,northeastern North China Craton,providing important constraints on their mantle source characteristics of individual episodes,as they have

Numerical Modeling of Episodic Compaction and Its Efecting Parameters in Geopresured Shales

ＮｕｍｅｒｉｃａｌＭｏｄｅｌｉｎｇｏｆＥｐｉｓｏｄｉｃＣｏｍｐａｃｔｉｏｎａｎｄＩｔｓＥｆｅｃｔｉｎｇＰａｒａｍｅｔｅｒｓｉｎＧｅｏｐｒｅｓｕｒｅｄＳｈａｌｅｓ＊ＸｉｅＸｉｎｏｎｇＦａｃｕｌｔｙｏｆＥａｒｔｈＲｅｓｏｕｒｃｅｓ，ＣｈｉｎａＵｎｉｖｅｒ...

Frequent Episodic Vertigo is an Unexpected Side Effect of Flutamide

An 82-year-old male suffered from prostatic cancer five years ago. Since then, he has taken flutamide and was bothered with episodic vertigo (EV) every morning. In order to treat prostatic cancer, flutamide was not discontinued, but conservative treatment and life-style change were recommended. Finally, EV actually subsided. Herein, we report the rare case, in which EV was an unexpected side effect of flutamide. Herein we review his whole history, physical examination, vestibular function test, electronystagmogram, caloric test, awake encephalogram, blood examinations, color-coded duplex ultrasonogram and magnetic resonance imaging/angiogram to suggest a mechanism of flutamide responsible for EV.

Linked Data-based Slide Repository： The Episodic Slide Retrieval Using the Episodic Keyword Networks

This paper focuses on developing a system that allows presentation authors to effectively retrieve presentation slides for reuse from a large volume of existing presentation materials. We assume episodic memories of the authors can be used as contextual keywords in query expressions to efficiently dig out the expected slides for reuse rather than using only the part-of-slide-descriptions-based keyword queries. As a system, a new slide repository is proposed, composed of slide material collections, slide content data and pieces of information from authors＇ episodic memories related to each slide and presentation together with a slide retrieval application enabling authors to use the episodic memories as part of queries. The result of our experiment shows that the episodic memory-used queries can give more discoverability than the keyword-based queries. Additionally, an improvement model is discussed on the slide retrieval for further slide-finding efficiency by expanding the episodic memories model in the repository taking in the links with the author-and-slide-related data and events having been post on the private and social media sites.

Episodic Learning and Memory in Prodromal Huntington’s Disease: The Role of Multimodal Encoding and Selective Reminding

This study investigated episodic memory in prodromal HD. Three groups were compared (N = 70): mutation carriers with less than 12.5 years to disease onset (n = 16), mutation carriers with 12.5 or more years to disease onset (n = 16), and noncarriers (n = 38). Episodic memory was assessed using the Fuld Object Memory Evaluation, which included multimodal presentation and selective reminding, and the Claeson-Dahl Learning Test which included verbal repeated presentation and recall trials. Both carrier groups demonstrated deficient episodic memory compared to noncarriers. The results suggest deficient episodic memory in prodromal HD, and that inconsistent retrieval contributes to these deficits. Multimodal presentation attenuates the deficits.

Promotion of structural plasticity in area V2 of visual cortex prevents against object recognition memory deficits in aging and Alzheimer's disease rodents

Memory deficit,which is often associated with aging and many psychiatric,neurological,and neurodegenerative diseases,has been a challenging issue for treatment.Up till now,all potential drug candidates have failed to produce satisfa ctory effects.Therefore,in the search for a solution,we found that a treatment with the gene corresponding to the RGS14414protein in visual area V2,a brain area connected with brain circuits of the ventral stream and the medial temporal lobe,which is crucial for object recognition memory(ORM),can induce enhancement of ORM.In this study,we demonstrated that the same treatment with RGS14414in visual area V2,which is relatively unaffected in neurodegenerative diseases such as Alzheimer s disease,produced longlasting enhancement of ORM in young animals and prevent ORM deficits in rodent models of aging and Alzheimer’s disease.Furthermore,we found that the prevention of memory deficits was mediated through the upregulation of neuronal arbo rization and spine density,as well as an increase in brain-derived neurotrophic factor(BDNF).A knockdown of BDNF gene in RGS14414-treated aging rats and Alzheimer s disease model mice caused complete loss in the upregulation of neuronal structural plasticity and in the prevention of ORM deficits.These findings suggest that BDNF-mediated neuronal structural plasticity in area V2 is crucial in the prevention of memory deficits in RGS14414-treated rodent models of aging and Alzheimer’s disease.Therefore,our findings of RGS14414gene-mediated activation of neuronal circuits in visual area V2 have therapeutic relevance in the treatment of memory deficits.

Volcanically Driven Terrestrial Environmental Perturbations during the Carnian Pluvial Episode in the Eastern Tethys

The Carnian Pluvial Episode(CPE)fingerprints global environmental perturbations and biological extinction on land and oceans and is potentially linked to the Wrangellia Large Igneous Province(LIP).However,the correlation between terrestrial environmental changes and Wrangellia volcanism in the Ordos Basin during the CPE remains poorly understood.Records of negative carbon isotopic excursions(NCIEs),mercury(Hg),Hg/TOC,and Hg enrichment factor(HgEF)from oil shales in a large-scale terrestrial Ordos Basin in the Eastern Tethys were correlated with marine and other terrestrial successions.The three significant NCIEs in the study section were consistently correlated with those in the CPE successions of Europe,the UK,and South and North China.The U-Pb geochronology indicates a Ladinian-Carnian age for the Chang 7 Member.A comprehensive overview of the geochronology,NCIE correlation,and previous bio-and chronostratigraphic frameworks shows that the Ladinian-Carnian boundary is located in the lower part of Chang 7 in the Yishicun section.HgEF may be a more reliable proxy for tracing volcanic eruptions than the Hg/TOC ratio because the accumulation rates of TOC content largely vary in terrestrial and marine successions.The records of Hg,Hg/TOC,HgEF,and NCIEs in the Ordos Basin aligned with Carnian successions worldwide and were marked by similar anomalies,indicating a global response to the Wrangellia LIP during the CPE.Anoxia,a warm-humid climate,enhancement of detrital input,and NCIEs are synchronous with the CPE interval in the Ordos Basin,which suggests that the CPE combined with the regional Qinling Orogeny should dominate the enhanced rate of terrigenous input and paleoenvironmental evolution in the Ordos Basin.

基于搜索引擎的知识发现

数据挖掘一般用于高度结构化的大型数据库,以发现其中所蕴含的知识。随着在线文本的增多,其中所蕴含的知识也越来越丰富,但是,它们却难以被分析利用。因而,研究一套行之有效的方案发现文本中所蕴含的知识是非常重要的,也是当前重要的研究课题。该文利用搜索引擎Google获取相关Web页面,进行过滤和清洗后得到相关文本,然后,进行文本聚类,利用Episode进行事件识别和信息抽取,数据集成及数据挖掘,从而实现知识发现。最后给出了原型系统,对知识发现进行实践检验,收到了很好的效果。

Episode:互动小说移动应用市场的先驱

艾匹·德互动公司(Episode Interactive)开发的移动应用程序Episode是全球最早开发的互动小说应用之一,也是目前世界上最大的移动叙事网络和平台之一。凭借对移动娱乐市场的敏锐嗅觉,它在欧美移动互动小说市场取得了先发优势,构建了以用户生成内容(UGC)、自主研发、IP改编三端驱动内容生产,以年轻女性群体为主要受众,以付费道具模式和广告为主要收入来源的成熟出版生态和商业模式,并拥有世界上最大的互动叙事创作者社区之一。虽然由于受受众群体、内容主题和资本博弈等因素的影响,移动互动小说市场的竞争异常激烈,但Episode凭借着良好的用户基础和在UGC领域的多年耕耘,有应对挑战的资本和信心。

三维动画Episode的设计与制作

随着计算机处理图形图像的技术日益完善,CG制作越来越频繁地出现在生活中,其中的三维动画尤为常见。Maya是现在国际最为流行的顶级三维动画制作软件。通过Episode动画短片的制作,介绍运用Maya制作三维动画的步骤和过程。

Chemical characteristics of PM_(2.5) during a typical haze episode in Guangzhou

The chemical characteristics（water-soluble ions and carbonaceous species） of PM2.5 in Guangzhou were measured during a typical haze episode.Most of the chemical species in PM2.5 showed significant difference between normal and haze days.The highest contributors to PM2.5 were organic carbon（OC）,nitrate,and sulfate in haze days and were OC,sulfate,and elemental carbon（EC） in normal days.The concentrations of secondary species such as,NO3^-,SO4^2-,and NH4^＋ in haze days were 6.5,3.9,and 5.3 times higher than those in normal days,respectively,while primary species（EC,Ca^2＋,K^＋） show similar increase from normal to haze days by a factor about 2.2-2.4.OC/EC ratio ranged from 2.8 to 6.2 with an average of 4.7 and the estimation on a minimum OC/EC ratio showed that SOC（secondary organic carbon） accounted more than 36.6% for the total organic carbon in haze days.The significantly increase in the secondary species（SOC,NO3^-,SO4^2-,and NH4^＋）,especially in NO3^-,caused the worst air quality in this region.Simultaneously,the result illustrated that the serious air pollution in haze episodes was strongly correlated with the meteorological conditions.During the sampling periods,air pollution and visibility had a good relationship with the air mass transport distance;the shorter air masses transport distance,the worse air quality and visibility in Guangzhou,indicating the strong domination of local sources contributing to haze formation.High concentration of the secondary aerosol in haze episodes was likely due to the higher oxidation rates of sulfur and nitrogen species.