Efficient Penetration Testing Path Planning Based on Reinforcement Learning with Episodic Memory

Intelligent penetration testing is of great significance for the improvement of the security of information systems,and the critical issue is the planning of penetration test paths.In view of the difficulty for attackers to obtain complete network information in realistic network scenarios,Reinforcement Learning(RL)is a promising solution to discover the optimal penetration path under incomplete information about the target network.Existing RL-based methods are challenged by the sizeable discrete action space,which leads to difficulties in the convergence.Moreover,most methods still rely on experts’knowledge.To address these issues,this paper proposes a penetration path planning method based on reinforcement learning with episodic memory.First,the penetration testing problem is formally described in terms of reinforcement learning.To speed up the training process without specific prior knowledge,the proposed algorithm introduces episodic memory to store experienced advantageous strategies for the first time.Furthermore,the method offers an exploration strategy based on episodic memory to guide the agents in learning.The design makes full use of historical experience to achieve the purpose of reducing blind exploration and improving planning efficiency.Ultimately,comparison experiments are carried out with the existing RL-based methods.The results reveal that the proposed method has better convergence performance.The running time is reduced by more than 20%.

Promotion of structural plasticity in area V2 of visual cortex prevents against object recognition memory deficits in aging and Alzheimer's disease rodents

Memory deficit,which is often associated with aging and many psychiatric,neurological,and neurodegenerative diseases,has been a challenging issue for treatment.Up till now,all potential drug candidates have failed to produce satisfa ctory effects.Therefore,in the search for a solution,we found that a treatment with the gene corresponding to the RGS14414protein in visual area V2,a brain area connected with brain circuits of the ventral stream and the medial temporal lobe,which is crucial for object recognition memory(ORM),can induce enhancement of ORM.In this study,we demonstrated that the same treatment with RGS14414in visual area V2,which is relatively unaffected in neurodegenerative diseases such as Alzheimer s disease,produced longlasting enhancement of ORM in young animals and prevent ORM deficits in rodent models of aging and Alzheimer’s disease.Furthermore,we found that the prevention of memory deficits was mediated through the upregulation of neuronal arbo rization and spine density,as well as an increase in brain-derived neurotrophic factor(BDNF).A knockdown of BDNF gene in RGS14414-treated aging rats and Alzheimer s disease model mice caused complete loss in the upregulation of neuronal structural plasticity and in the prevention of ORM deficits.These findings suggest that BDNF-mediated neuronal structural plasticity in area V2 is crucial in the prevention of memory deficits in RGS14414-treated rodent models of aging and Alzheimer’s disease.Therefore,our findings of RGS14414gene-mediated activation of neuronal circuits in visual area V2 have therapeutic relevance in the treatment of memory deficits.

Episodic Learning and Memory in Prodromal Huntington’s Disease: The Role of Multimodal Encoding and Selective Reminding

This study investigated episodic memory in prodromal HD. Three groups were compared (N = 70): mutation carriers with less than 12.5 years to disease onset (n = 16), mutation carriers with 12.5 or more years to disease onset (n = 16), and noncarriers (n = 38). Episodic memory was assessed using the Fuld Object Memory Evaluation, which included multimodal presentation and selective reminding, and the Claeson-Dahl Learning Test which included verbal repeated presentation and recall trials. Both carrier groups demonstrated deficient episodic memory compared to noncarriers. The results suggest deficient episodic memory in prodromal HD, and that inconsistent retrieval contributes to these deficits. Multimodal presentation attenuates the deficits.

Linked Data-based Slide Repository： The Episodic Slide Retrieval Using the Episodic Keyword Networks

This paper focuses on developing a system that allows presentation authors to effectively retrieve presentation slides for reuse from a large volume of existing presentation materials. We assume episodic memories of the authors can be used as contextual keywords in query expressions to efficiently dig out the expected slides for reuse rather than using only the part-of-slide-descriptions-based keyword queries. As a system, a new slide repository is proposed, composed of slide material collections, slide content data and pieces of information from authors＇ episodic memories related to each slide and presentation together with a slide retrieval application enabling authors to use the episodic memories as part of queries. The result of our experiment shows that the episodic memory-used queries can give more discoverability than the keyword-based queries. Additionally, an improvement model is discussed on the slide retrieval for further slide-finding efficiency by expanding the episodic memories model in the repository taking in the links with the author-and-slide-related data and events having been post on the private and social media sites.

A Map Construction Method Based on the Cognitive Mechanism of Rat Brain Hippocampus

The entorhinal-hippocampus structure in the mammalian brain is the core area for realizing spatial cognition.However,the visual perception and loop detection methods in the current biomimetic robot navigation model still rely on traditional visual SLAM schemes and lack the process of exploring and applying biological visual methods.Based on this,we propose amap constructionmethod thatmimics the entorhinal-hippocampal cognitive mechanismof the rat brain according to the response of entorhinal cortex neurons to eye saccades in recent related studies.That is,when mammals are free to watch the scene,the entorhinal cortex neurons will encode the saccade position of the eyeball to realize the episodicmemory function.The characteristics of thismodel are as follows:1)A scenememory algorithmthat relies on visual saccade vectors is constructed to imitate the biological brain’s memory of environmental situation information matches the current scene information with the memory;2)According to the information transmission mechanism formed by the hippocampus and the activation theory of spatial cells,a localization model based on the grid cells of the entorhinal cortex and the place cells of the hippocampus was constructed;3)Finally,the scene memory algorithm is used to correct the errors of the positioning model and complete the process of constructing the cognitive map.The model was subjected to simulation experiments on publicly available datasets and physical experiments using a mobile robot platform to verify the environmental adaptability and robustness of the algorithm.The algorithm will provide a basis for further research into bionic robot navigation.

On brain activity mapping: Insights and lessons from Brain Decoding Project to map memory patterns in the hippocampus

The BRAIN project recently announced by the president Obama is the reflection of unrelenting human quest for cracking the brain code, the patterns of neuronal activity that define who we are and what we are. While the Brain Activity Mapping proposal has rightly emphasized on the need to develop new technologies for measuring every spike from every neuron, it might be helpful to consider both the theoretical and experimental aspects that would accelerate our search for the organizing principles of the brain code. Here we share several insights and lessons from the similar proposal, namely, Brain Decoding Project that we initiated since 2007. We provide a specific example in our initial mapping of real-time memory traces from one part of the memory circuit, namely, the CA1 region of the mouse hippocampus. We show how innovative behavioral tasks and appropriate mathematical analyses of large datasets can play equally, if not more, important roles in uncovering the specific-to-general feature-coding cell assembly mechanism by which episodic memory, semantic knowledge, and imagination are generated and organized. Our own experiences suggest that the bottleneck of the Brain Project is not only at merely developing additional new technologies, but also the lack of efficient avenues to disseminate cutting edge platforms and decoding expertise to neuroscience community. Therefore, we propose that in order to harness unique insights and extensive knowledge from various investigators working in diverse neuroscience subfields, ranging from perception and emotion to memory and social behaviors, the BRAIN project should create a set of International and National Brain Decoding Centers at which cutting-edge recording technologies and expertise on analyzing large datasets analyses can be made readily available to the entire community of neuroscientists who can apply and schedule to perform cutting-edge research.

Does Engaging in Acute Exercise Prior to Memory Encoding and During Memory Consolidation have an Additive Effect on Long-Term Memory Function?

Previous work has demonstrated that acute exercise prior to memory encoding may enhance long-term memory.Similarly,other work demonstrates that acute exercise during the memory consolidation period may also enhance long-term memory function.However,no study has evaluated whether long-term memory is enhanced when an acute bout of exercise occurs during both of these time periods,when compared to just prior to memory encoding.A within-subject randomized con-trolled intervention was employed.On separate laboratory visits,participants completed two main protocols,including(1)exercise before memory encoding and(2)exercise before and after memory encoding.Long-term memory was assessed,via a word-list task,from a 20-min delay period and a 24-h delay period.We observed a significant main effect for time,F(8,176)=529.5,P<0.001,ηp^(2)=0.96,but no significant main effect for condition,F(l,22)=0.08,P=0.77,ηp^(2)=0.004,or time by condition interaction,F(8,176)=0.19,P=0.99,ηp^(2)=0.009.In conclusion,there was no difference in long-term memory function when comparing acute exercise only prior to memory encoding vs.acute exercise both before and immediately after memory encoding.

Agents with four categories of understanding abilities and their role in two-stage(deep)emotional intelligence simulation

Understanding is the essence of any intelligent system.We revise our four machine understanding paradigms which are:(i)basic understanding,(ii)rich understanding,(iii)exploratory understanding,and(iv)theory-based understanding;and we elaborate on the first two of them.We then introduce the concept of two-stage(or deep)machine understanding which provides descriptive understandings,as well as evaluations of these understandings.After a brief systematization of emotions,we cover the following paradigms for agents with two-stage(deep)understanding abilities for emotional intelligence simulation:(i)basic understanding,(ii)rich-understanding,and(iii)switchable understanding.

Can multi-modal neuroimaging evidence from hippocampus provide biomarkers for the progression of amnestic mild cognitive impairment?

Impaired structure and function of the hippocampus is a valuable predictor of progression from amnestic mild cognitive impairment（a MCI） to Alzheimer＇s disease（AD）. As a part of the medial temporal lobe memory system,the hippocampus is one of the brain regions affected earliest by AD neuropathology,and shows progressive degeneration as a MCI progresses to AD. Currently,no validated biomarkers can precisely predict the conversion from a MCI to AD. Therefore,there is a great need of sensitive tools for the early detection of AD progression. In this review,we summarize the specifi c structural and functional changes in the hippocampus from recent a MCI studies using neurophysiological and neuroimaging data. We suggest that a combination of advanced multi-modal neuroimaging measures in discovering biomarkers will provide more precise and sensitive measures of hippocampal changes than using only one of them. These will potentially affect early diagnosis and disease-modifying treatments. We propose a new sequential and progressive framework in which the impairment spreads from the integrity of fibers to volume and then to function in hippocampal subregions. Meanwhile,this is likely to be accompanied by progressive impairment of behavioral and neuropsychological performance in the progression of a MCI to AD.