Network Defense Decision-Making Based on Deep Reinforcement Learning and Dynamic Game Theory

Existing researches on cyber attackdefense analysis have typically adopted stochastic game theory to model the problem for solutions,but the assumption of complete rationality is used in modeling,ignoring the information opacity in practical attack and defense scenarios,and the model and method lack accuracy.To such problem,we investigate network defense policy methods under finite rationality constraints and propose network defense policy selection algorithm based on deep reinforcement learning.Based on graph theoretical methods,we transform the decision-making problem into a path optimization problem,and use a compression method based on service node to map the network state.On this basis,we improve the A3C algorithm and design the DefenseA3C defense policy selection algorithm with online learning capability.The experimental results show that the model and method proposed in this paper can stably converge to a better network state after training,which is faster and more stable than the original A3C algorithm.Compared with the existing typical approaches,Defense-A3C is verified its advancement.

Arrhythmia Detection by Using Chaos Theory with Machine Learning Algorithms

Heart monitoring improves life quality.Electrocardiograms(ECGs or EKGs)detect heart irregularities.Machine learning algorithms can create a few ECG diagnosis processing methods.The first method uses raw ECG and time-series data.The second method classifies the ECG by patient experience.The third technique translates ECG impulses into Q waves,R waves and S waves(QRS)features using richer information.Because ECG signals vary naturally between humans and activities,we will combine the three feature selection methods to improve classification accuracy and diagnosis.Classifications using all three approaches have not been examined till now.Several researchers found that Machine Learning(ML)techniques can improve ECG classification.This study will compare popular machine learning techniques to evaluate ECG features.Four algorithms—Support Vector Machine(SVM),Decision Tree,Naive Bayes,and Neural Network—compare categorization results.SVM plus prior knowledge has the highest accuracy(99%)of the four ML methods.QRS characteristics failed to identify signals without chaos theory.With 99.8%classification accuracy,the Decision Tree technique outperformed all previous experiments.

The Application of Reinforcement Theory in the Review Stage of English Teaching and Learning in Chinese Higher Vocational and Technical Colleges

Reinforcement theory is a behavioral psychology theory proposed by Skinner,which has been widely applied in various fields such as management and education.Positive reinforcement and negative reinforcement are the two types of reinforcement.By adopting these two different reinforcement methods appropriately,human behavior can develop in a positive direction.In the review stage of English teaching and learning in Chinese higher vocational and technical colleges,the use of different reinforcement methods based on various classes,individuals,conditions,and environments can effectively promote or change the behavior of teachers and students,thereby improving the effectiveness of the review.

High-throughput calculations combining machine learning to investigate the corrosion properties of binary Mg alloys

Magnesium(Mg)alloys have shown great prospects as both structural and biomedical materials,while poor corrosion resistance limits their further application.In this work,to avoid the time-consuming and laborious experiment trial,a high-throughput computational strategy based on first-principles calculations is designed for screening corrosion-resistant binary Mg alloy with intermetallics,from both the thermodynamic and kinetic perspectives.The stable binary Mg intermetallics with low equilibrium potential difference with respect to the Mg matrix are firstly identified.Then,the hydrogen adsorption energies on the surfaces of these Mg intermetallics are calculated,and the corrosion exchange current density is further calculated by a hydrogen evolution reaction(HER)kinetic model.Several intermetallics,e.g.Y_(3)Mg,Y_(2)Mg and La_(5)Mg,are identified to be promising intermetallics which might effectively hinder the cathodic HER.Furthermore,machine learning(ML)models are developed to predict Mg intermetallics with proper hydrogen adsorption energy employing work function(W_(f))and weighted first ionization energy(WFIE).The generalization of the ML models is tested on five new binary Mg intermetallics with the average root mean square error(RMSE)of 0.11 eV.This study not only predicts some promising binary Mg intermetallics which may suppress the galvanic corrosion,but also provides a high-throughput screening strategy and ML models for the design of corrosion-resistant alloy,which can be extended to ternary Mg alloys or other alloy systems.

Machine learning with active pharmaceutical ingredient/polymer interaction mechanism:Prediction for complex phase behaviors of pharmaceuticals and formulations

The high throughput prediction of the thermodynamic phase behavior of active pharmaceutical ingredients(APIs)with pharmaceutically relevant excipients remains a major scientific challenge in the screening of pharmaceutical formulations.In this work,a developed machine-learning model efficiently predicts the solubility of APIs in polymers by learning the phase equilibrium principle and using a few molecular descriptors.Under the few-shot learning framework,thermodynamic theory(perturbed-chain statistical associating fluid theory)was used for data augmentation,and computational chemistry was applied for molecular descriptors'screening.The results showed that the developed machine-learning model can predict the API-polymer phase diagram accurately,broaden the solubility data of APIs in polymers,and reproduce the relationship between API solubility and the interaction mechanisms between API and polymer successfully,which provided efficient guidance for the development of pharmaceutical formulations.

A game-theoretic approach for federated learning:A trade-off among privacy,accuracy and energy

Benefiting from the development of Federated Learning(FL)and distributed communication systems,large-scale intelligent applications become possible.Distributed devices not only provide adequate training data,but also cause privacy leakage and energy consumption.How to optimize the energy consumption in distributed communication systems,while ensuring the privacy of users and model accuracy,has become an urgent challenge.In this paper,we define the FL as a 3-layer architecture including users,agents and server.In order to find a balance among model training accuracy,privacy-preserving effect,and energy consumption,we design the training process of FL as game models.We use an extensive game tree to analyze the key elements that influence the players’decisions in the single game,and then find the incentive mechanism that meet the social norms through the repeated game.The experimental results show that the Nash equilibrium we obtained satisfies the laws of reality,and the proposed incentive mechanism can also promote users to submit high-quality data in FL.Following the multiple rounds of play,the incentive mechanism can help all players find the optimal strategies for energy,privacy,and accuracy of FL in distributed communication systems.

An Incentive Mechanism for Federated Learning:A Continuous Zero-Determinant Strategy Approach

As a representative emerging machine learning technique, federated learning(FL) has gained considerable popularity for its special feature of “making data available but not visible”. However, potential problems remain, including privacy breaches, imbalances in payment, and inequitable distribution.These shortcomings let devices reluctantly contribute relevant data to, or even refuse to participate in FL. Therefore, in the application of FL, an important but also challenging issue is to motivate as many participants as possible to provide high-quality data to FL. In this paper, we propose an incentive mechanism for FL based on the continuous zero-determinant(CZD) strategies from the perspective of game theory. We first model the interaction between the server and the devices during the FL process as a continuous iterative game. We then apply the CZD strategies for two players and then multiple players to optimize the social welfare of FL, for which we prove that the server can keep social welfare at a high and stable level. Subsequently, we design an incentive mechanism based on the CZD strategies to attract devices to contribute all of their high-accuracy data to FL.Finally, we perform simulations to demonstrate that our proposed CZD-based incentive mechanism can indeed generate high and stable social welfare in FL.

Hybrid data-driven framework for shale gas production performance analysis via game theory, machine learning, and optimization approaches

A comprehensive and precise analysis of shale gas production performance is crucial for evaluating resource potential,designing a field development plan,and making investment decisions.However,quantitative analysis can be challenging because production performance is dominated by the complex interaction among a series of geological and engineering factors.In fact,each factor can be viewed as a player who makes cooperative contributions to the production payoff within the constraints of physical laws and models.Inspired by the idea,we propose a hybrid data-driven analysis framework in this study,where the contributions of dominant factors are quantitatively evaluated,the productions are precisely forecasted,and the development optimization suggestions are comprehensively generated.More specifically,game theory and machine learning models are coupled to determine the dominating geological and engineering factors.The Shapley value with definite physical meaning is employed to quantitatively measure the effects of individual factors.A multi-model-fused stacked model is trained for production forecast,which provides the basis for derivative-free optimization algorithms to optimize the development plan.The complete workflow is validated with actual production data collected from the Fuling shale gas field,Sichuan Basin,China.The validation results show that the proposed procedure can draw rigorous conclusions with quantified evidence and thereby provide specific and reliable suggestions for development plan optimization.Comparing with traditional and experience-based approaches,the hybrid data-driven procedure is advanced in terms of both efficiency and accuracy.

A survey of multi-modal learning theory

Deep multi-modal learning,a rapidly growing field with a wide range of practical applications,aims to effectively utilize and integrate information from multiple sources,known as modalities.Despite its impressive empirical performance,the theoretical foundations of deep multi-modal learning have yet to be fully explored.In this paper,we will undertake a comprehensive survey of recent developments in multi-modal learning theories,focusing on the fundamental properties that govern this field.Our goal is to provide a thorough collection of current theoretical tools for analyzing multi-modal learning,to clarify their implications for practitioners,and to suggest future directions for the establishment of a solid theoretical foundation for deep multi-modal learning.

Recognition and interfere deceptive behavior based on inverse reinforcement learning and game theory

In real-time strategy(RTS)games,the ability of recognizing other players’goals is important for creating artifical intelligence(AI)players.However,most current goal recognition methods do not take the player’s deceptive behavior into account which often occurs in RTS game scenarios,resulting in poor recognition results.In order to solve this problem,this paper proposes goal recognition for deceptive agent,which is an extended goal recognition method applying the deductive reason method(from general to special)to model the deceptive agent’s behavioral strategy.First of all,the general deceptive behavior model is proposed to abstract features of deception,and then these features are applied to construct a behavior strategy that best matches the deceiver’s historical behavior data by the inverse reinforcement learning(IRL)method.Final,to interfere with the deceptive behavior implementation,we construct a game model to describe the confrontation scenario and the most effective interference measures.

A Graph Theory Based Self-Learning Honeypot to Detect Persistent Threats

Attacks on the cyber space is getting exponential in recent times.Illegal penetrations and breaches are real threats to the individuals and organizations.Conventional security systems are good enough to detect the known threats but when it comes to Advanced Persistent Threats(APTs)they fails.These APTs are targeted,more sophisticated and very persistent and incorporates lot of evasive techniques to bypass the existing defenses.Hence,there is a need for an effective defense system that can achieve a complete reliance of security.To address the above-mentioned issues,this paper proposes a novel honeypot system that tracks the anonymous behavior of the APT threats.The key idea of honeypot leverages the concepts of graph theory to detect such targeted attacks.The proposed honey-pot is self-realizing,strategic assisted which withholds the APTs actionable tech-niques and observes the behavior for analysis and modelling.The proposed graph theory based self learning honeypot using the resultsγ(C(n,1)),γc(C(n,1)),γsc(C(n,1))outperforms traditional techniques by detecting APTs behavioral with detection rate of 96%.

Project-based Language Learning: an Activity Theory Analysis in SOE Language Learning

This study focuses on the effectiveness of the project-based language learning(PBLL) in a college Secretarial Oral English(SOE) Module. Student reflections of the language project work have been analyzed through Activity Theory. Moreover,Data has been collected and categorized based on the components of complex human activity: the subject, object, tools(signs,symbols, and language), the community in which the activity take place, division of labor, and rules. The findings theoretically support the outcome of project-based language learning which align with the object of the activity.

College English Teaching Study Conducted by Discovery Learning Theory and Action Research——Take College English Ⅱ for Instance

The unceasing revolution of the global economy and culture boosts the revolutionary step of the educational circle.Combining the creed of The Guide of College English Teaching in 2016 with the results of investigation and survey in colleges, a research group in the Institute of Foreign Languages of Hankou University comes up with a revolutionary trial scheme on College English teaching conducted by discovery learning theory, as well as a research method of action research, which is in hope of mending the problems and shortcomings of current College English teaching.

Applying Learning Community Theory to Oral English Teaching in Sport Institutes

Since traditional English teaching method, which merely focuses on language teaching but ignores communicative competence, severely impedes the development of students' oral ability. It is high time that English teachers took measures to find a workable and valuable teaching method which can improve students' speaking proficiency effectively. Learning community theory provides a broad space for this, for it regards learning as a process which takes place in a community where the learners are sharing their experience towards knowledge building in an interactive and cooperative way.

The Application of Constructivism Theory to the Student-determined College English Learning

According to the further exploration into constructivism theory, the author illustrates the application of this theory to China＇s college English teaching, especially in the new perspective of student-determined learning.

A Summary of Some Teaching Methodologies as well as Some Linguistic and Learning Theories

This paper tries to summarize some main schools 0f teaching methodologies abroad and some main learning theories abroad. From this paper, we can know the main learning theories, the basic theories of them and the leading figures. It can help us understand the characteristics of each school of the teaching methodologies and learning theories.

Atomistic Modeling of Lithium Materials from Deep Learning Potential with Ab Initio Accuracy

Lithium has been paid great attention in recent years thanks to its significant appli-cations for battery and lightweight alloy.Developing a potential model with high ac-curacy and efficiency is impor-tant for theoretical simulation of lithium materials.Here,we build a deep learning potential(DP)for elemental lithium based on a concurrent-learning scheme and DP representation of the density-functional theory(DFT)potential energy surface(PES),the DP model enables material simulations with close-to DFT accuracy but at much lower computational cost.The simulations show that basic parameters,equation of states,elasticity,defects and surface are consistent with the first principles results.More notably,the liquid radial distribution func-tion based on our DP model is found to match well with experiment data.Our results demon-strate that the developed DP model can be used for the simulation of lithium materials.

Social Engineering Attack-Defense Strategies Based on Reinforcement Learning

Social engineering attacks are considered one of the most hazardous cyberattacks in cybersecurity,as human vulnerabilities are often the weakest link in the entire network.Such vulnerabilities are becoming increasingly susceptible to network security risks.Addressing the social engineering attack defense problem has been the focus of many studies.However,two main challenges hinder its successful resolution.Firstly,the vulnerabilities in social engineering attacks are unique due to multistage attacks,leading to incorrect social engineering defense strategies.Secondly,social engineering attacks are real-time,and the defense strategy algorithms based on gaming or reinforcement learning are too complex to make rapid decisions.This paper proposes a multiattribute quantitative incentive method based on human vulnerability and an improved Q-learning(IQL)reinforcement learning method on human vulnerability attributes.The proposed algorithm aims to address the two main challenges in social engineering attack defense by using a multiattribute incentive method based on human vulnerability to determine the optimal defense strategy.Furthermore,the IQL reinforcement learning method facilitates rapid decision-making during real-time attacks.The experimental results demonstrate that the proposed algorithm outperforms the traditional Qlearning(QL)and deep Q-network(DQN)approaches in terms of time efficiency,taking 9.1%and 19.4%less time,respectively.Moreover,the proposed algorithm effectively addresses the non-uniformity of vulnerabilities in social engineering attacks and provides a reliable defense strategy based on human vulnerability attributes.This study contributes to advancing social engineering attack defense by introducing an effective and efficient method for addressing the vulnerabilities of human factors in the cybersecurity domain.

Anderson's Cognitive Theory and Learning Strategy Studies in Second Language Acquisition

Second language acquisition can not be understood without addressing the interaction between language and cognition. Cognitive theory can extend to describe learning strategies as complex cognitive skills. Theoretical developments in Anderson’s production systems cover a broader range of behavior than other theories, including comprehension and production of oral and written texts as well as comprehension, problem solving, and verbal learning.Thus Anderson’s cognitive theory can be served as a rationale for learning strategy studies in second language acquisition.

Research on College Studentst'Autonomous EFL Learning Affer Course Exemption

By analyzing the English learning logs of 12 students in a provincial university in south-west China after they had been exempted from taking college English courses,this study investigated college students’autonomous EFL(English as a foreign language)learning after course exemption,including the use of mediational means in EFL learning,EFL learning hours,and other factors affecting EFL learning,in the hope of giving new perspectives on college ELF curriculum design,teaching,and education management.