Efficient Penetration Testing Path Planning Based on Reinforcement Learning with Episodic Memory

Intelligent penetration testing is of great significance for the improvement of the security of information systems,and the critical issue is the planning of penetration test paths.In view of the difficulty for attackers to obtain complete network information in realistic network scenarios,Reinforcement Learning(RL)is a promising solution to discover the optimal penetration path under incomplete information about the target network.Existing RL-based methods are challenged by the sizeable discrete action space,which leads to difficulties in the convergence.Moreover,most methods still rely on experts’knowledge.To address these issues,this paper proposes a penetration path planning method based on reinforcement learning with episodic memory.First,the penetration testing problem is formally described in terms of reinforcement learning.To speed up the training process without specific prior knowledge,the proposed algorithm introduces episodic memory to store experienced advantageous strategies for the first time.Furthermore,the method offers an exploration strategy based on episodic memory to guide the agents in learning.The design makes full use of historical experience to achieve the purpose of reducing blind exploration and improving planning efficiency.Ultimately,comparison experiments are carried out with the existing RL-based methods.The results reveal that the proposed method has better convergence performance.The running time is reduced by more than 20%.

Exploration Potential of Marine Source Rocks Oil-Gas Reservoirs in China

So far, more than 150 marine oil-gas fields have been found onshore and offshore about 350. The marine source rocks are mainly Paleozoic and Mesozoic onshore whereas Tertiary offshore. Three genetic categories of oil-gas reservoirs have been defined for the marine reservoirs in China： primary reservoirs, secondary reservoirs and hydrocarbon-regeneration reservoirs. And three exploration prospects have also been suggested： （1） Primary reservoirs prospects, which are chiefly distributed in many Tertiary basins of the South China Sea （SCS）, the Tertiary shelf basins of the East China Sea （ECS） and the Paleozoic of Tarim basin, Sichuan basin and Ordos basin. To explore large-middle-scale even giant oil-gas fields should chiefly be considered in this category reservoirs. These basins are the most hopeful areas to explore marine oil-gas fields in China, among which especially many Tertiary basins of the SCS should be strengthened to explore. （2） Secondary reservoirs prospects, which are mainly distributed in the Paleozoic and Mesozoic of the Tarim basin, Sichuan basin, Qiangtang basin and Chuxiong basin in western China, of which exploration potential is less than that of the primary reservoirs. （3） Hydrocarbon-regeneration reservoirs prospects, which are chiefly distributed in the Bohai Bay basin, North Jiangsu-South Yellow Sea basin, southern North China basin, Jianghan basin, South Poyang basin in eastern China and the Tarim basin in western China, of which source rocks are generally the Paleozoic. And the reservoirs formed by late-stage （always Cenozoic） secondary hydrocarbon generation of the Paleozoic source rocks should mainly be considered to explore, among which middle-small and small oil-gas fields are the chief exploration targets. As a result of higher thermal evolution of Paleozoic and Mesozoic source rocks, the marine reservoirs onshore are mainly gas fields, and so far marine oil fields have only been found in the Tarim basin. No other than establishing corresponding marine oil-gas exploration and development strategy and policy, sufficiently enhancing cognition to the particularity and complexity of China＇s marine petroleum geology, and applying new thoughts, new theories and new technologies, at the same time tackling some key technologies, it is possible to fast and effectually exploit and utilize the potential huge marine oil-gas resources of China.

Key strategies for predictive exploration in mature environment: model innovation, exploration technology optimization and information integration

Prediction has become more and more difficult in mineral exploration, especially in the mature exploration environment such as Tongling copper district. For enhancing predictive discovery of hidden ore deposits in such mature environment, the key strategies which should be adopted include the innovation of the exploration models, application of the advanced exploration techniques and integration of multiple sets of information. The innovation of the exploration models should incorporate the new metallogenic concepts that are based on the geodynamic anatomization. The advanced techniques applied in the mature exploration environment should aim at the speciality and complexity of the geological setting and working environments. The information synthesis is to integrate multiple sets of data for giving a more credible and visual prospectivity map by using the geographic imformation system(GIS) and several mathematical methods, such as weight of evidence and fuzzy logic, which can extract useful information from every set of data as much as possible. Guided by these strategies, a predictive exploration in Fenghuangshan ore field of Tongling copper district was implemented, and a hidden ore deposit was discovered.

Facilitating knowledge management strategies through IT and HRM

In an increasingly competitive environment, where new business practices are regularly introduced, organizations have to be innovative to survive. In the present competitive climate, knowledge is considered as the main distinguishing factor of business success, and it is seen as the foundation of organization＇s innovation. The emergence of knowledge-intensive society has changed the nature of business competition. Hence knowledge needs to be appropriately managed. Knowledge Management （KM） focuses on managing different knowledge processes such as acquiring, creating, storing, sharing, transferring and applying implicit and explicit knowledge with objective of product and process innovation, performance development and sustainable competitive advantage. This paper tries to demonstrate KM lead to promotion of innovation and performance when it is correctly supported by human resource management （HRM） and information technology （IT）. The questions we will try to investigate in this paper are： How knowledge in organizations can be managed？ What is the connection between HRM, IT and effective implementation knowledge management strategies and how these relationships affect on organizational goals？ For this means, this study ftrst states the importance of knowledge, KM process and introduces two strategies for managing knowledge （exploitative and explorative strategy）. Next, it demonstrates each strategy requires to desired facilitator to support them in action. Finally, with presenting of model, this study concludes that each one of strategy can cover some KM process. Hence for the effective implementation of knowledge management process, organizations have to utilize both strategies.

Risk Management and Operation Strategies of Chinese Overseas Risk Exploration Projects

Currently,the investment of oil and gas industry is still facing an unfavorable environment,in which,instable factors,such as financial crisis,terrorist,religious conflicts and rigorous environmental regulations,keep mucking up the business all around the world.Meanwhile,China’s rapid energy consumption growth boosted by a booming economy has put the country to rely heavily on exported oil.It is therefore extremely urgent to expand and diversify petroleum supply channel in consideration of the country’s energy security.As the world’s economy has been slowly recovering from the slump and

Skill Learning for Robotic Insertion Based on One-shot Demonstration and Reinforcement Learning

In this paper,an efficient skill learning framework is proposed for robotic insertion,based on one-shot demonstration and reinforcement learning.First,the robot action is composed of two parts:expert action and refinement action.A force Jacobian matrix is calibrated with only one demonstration,based on which stable and safe expert action can be generated.The deep deterministic policy gradients(DDPG)method is employed to learn the refinement action,which aims to improve the assembly efficiency.Second,an episode-step exploration strategy is developed,which uses the expert action as a benchmark and adjusts the exploration intensity dynamically.A safety-efficiency reward function is designed for the compliant insertion.Third,to improve the adaptability with different components,a skill saving and selection mechanism is proposed.Several typical components are used to train the skill models.And the trained models and force Jacobian matrices are saved in a skill pool.Given a new component,the most appropriate model is selected from the skill pool according to the force Jacobian matrix and directly used to accomplish insertion tasks.Fourth,a simulation environment is established under the guidance of the force Jacobian matrix,which avoids tedious training process on real robotic systems.Simulation and experiments are conducted to validate the effectiveness of the proposed methods.

Meta-synthesis management framework of a complex project： A case study of the deck pavement project of the Hong Kong-Zhuhai-Macao Bridge

Innovative technology and deep uncertainty during the design and construction process of complex projects introduce great challenges to their organization and management.The traditional methods,represented in the project management body of knowledge(PMBOK)guide,can solve systematic problems;however,they cannot solve complex problems.Based on the management practice implemented in the deck pavement project of the Hong Kong-Zhuhai-Macao Bridge(HZMB),in this work,we propose a meta-synthesis management framework for a complex project from the perspective of the science of complexity.The method deems that the complexity of the project has the characteristic of being multi-scale both in the design phase and the construction phase.These problems can be classified into different categories,each of which requires a different strategy.As a result,it is first necessary to adopt the"exploration"strategy to reduce project complexity and to transform the deep uncertainty problems into systematic problems.Then,the"exploitation"strategy should be used to apply the PMBOK and other traditional methods to achieve the design and construction goals of the project and to improve its efficiency.More specifically,in the design phase of a complex project,the"innovative integration"process is used for the exploration of the new engineering technology and knowledge;then,the"functional integration"process is employed to define the system architecture,the interface relationship,the technical index,and other functions.In the construction phase,the"adaptive integration"process is used for the construction of the engineering organization system;next,the"efficient integration"process is employed to improve the actual construction performance.The meta-synthesis management framework proposed in this work reveals the multi-scale principle of solving complex problems in the management practice of a complex project,and develops the methodology of metasynthesis.

Strategies of Efficient Exploration and Development of Deepwater Oil and Gas Overseas

Deepwater offshore area,with rich oil and gas resources,has currently become the hot area for oil and gas exploration and the main battlefield where global oil majors compete with each other.This article summarizes the current situation of deepwater oil and gas exploration and development overseas from the perspectives of exploration history,exploration achievements and technical progress,and analyzes the difficulties and challenges faced by the industry,such as low level understanding of hydrocarbon accumulation conditions,global security and environmental challenges.And then,several feasible implementation strategies have been proposed on how to actively deal with the challenges,promote Chinese oil companies' overseas exploration,and realize efficient exploration and development of deepwater oil and gas fields overseas.These strategies consist of three aspects:profit-oriented operation principle,exploration ideas supported by theory and technology,and exploration strategy that puts safety first.In the era of low oil prices,Chinese oil companies must build up confidence,seize the opportunity and face the challenge while taking efforts to innovate technologies and theories,updates strategic ideas,and cultivate deepwater talents so as to be in an active position in global deepwater exploration and development industry in the future.