Efficient Penetration Testing Path Planning Based on Reinforcement Learning with Episodic Memory

Intelligent penetration testing is of great significance for the improvement of the security of information systems,and the critical issue is the planning of penetration test paths.In view of the difficulty for attackers to obtain complete network information in realistic network scenarios,Reinforcement Learning(RL)is a promising solution to discover the optimal penetration path under incomplete information about the target network.Existing RL-based methods are challenged by the sizeable discrete action space,which leads to difficulties in the convergence.Moreover,most methods still rely on experts’knowledge.To address these issues,this paper proposes a penetration path planning method based on reinforcement learning with episodic memory.First,the penetration testing problem is formally described in terms of reinforcement learning.To speed up the training process without specific prior knowledge,the proposed algorithm introduces episodic memory to store experienced advantageous strategies for the first time.Furthermore,the method offers an exploration strategy based on episodic memory to guide the agents in learning.The design makes full use of historical experience to achieve the purpose of reducing blind exploration and improving planning efficiency.Ultimately,comparison experiments are carried out with the existing RL-based methods.The results reveal that the proposed method has better convergence performance.The running time is reduced by more than 20%.

Test Case Generation Evaluator for the Implementation of Test Case Generation Algorithms Based on Learning to Rank

In software testing,the quality of test cases is crucial,but manual generation is time-consuming.Various automatic test case generation methods exist,requiring careful selection based on program features.Current evaluation methods compare a limited set of metrics,which does not support a larger number of metrics or consider the relative importance of each metric to the final assessment.To address this,we propose an evaluation tool,the Test Case Generation Evaluator(TCGE),based on the learning to rank(L2R)algorithm.Unlike previous approaches,our method comprehensively evaluates algorithms by considering multiple metrics,resulting in a more reasoned assessment.The main principle of the TCGE is the formation of feature vectors that are of concern by the tester.Through training,the feature vectors are sorted to generate a list,with the order of the methods on the list determined according to their effectiveness on the tested assembly.We implement TCGE using three L2R algorithms:Listnet,LambdaMART,and RFLambdaMART.Evaluation employs a dataset with features of classical test case generation algorithms and three metrics—Normalized Discounted Cumulative Gain(NDCG),Mean Average Precision(MAP),and Mean Reciprocal Rank(MRR).Results demonstrate the TCGE’s superior effectiveness in evaluating test case generation algorithms compared to other methods.Among the three L2R algorithms,RFLambdaMART proves the most effective,achieving an accuracy above 96.5%,surpassing LambdaMART by 2%and Listnet by 1.5%.Consequently,the TCGE framework exhibits significant application value in the evaluation of test case generation algorithms.

A Study on Washback Effect of CET-6 Translation Test Towards College English Teaching and Learning

CET-6 is a nationwide and standardized test to evaluate college students' English levels. Now more than 10 million college students take part in the exam every year. On August 14, 2013, the CET-4 and CET-6 examination committee reformed the structure of the CET-6 papers. After that the sentence translation was changed into paragraph translation,the scores of translation section also increased to 15 points which is more difficult and more important than before. There is no doubt that CET-6will exert a great impact on the teaching and learning of College English in China. This thesis focuses on the washback effect of CET-6 translation test towards college English teaching and learning.First of all, this paper introduces the CET-6 translation test requirements, and the score requirements of translation part. The author also illustrates some significant language testing research of both domestic and foreign scholars, and explains the concept of reliability and validity. Secondly, through the survey method of interviews and questionnaire the author study the characteristics of teaching in college classroom, the effect of translation teaching, and how do students to study translation. The author makes a further research on the interaction between the translation test and the teaching and learning based on the relevant theories. At last, the author finds that the negative washback effect of the CET-6 translation test is greater than the positive one. The author reflects on the current translation teaching and learning in China, and hopes to minimize the negative effects of CET-6 translation test.

Predicting major adverse cardiovascular events after orthotopic liver transplantation using a supervised machine learning model:A cohort study

BACKGROUND Liver transplant(LT)patients have become older and sicker.The rate of post-LT major adverse cardiovascular events(MACE)has increased,and this in turn raises 30-d post-LT mortality.Noninvasive cardiac stress testing loses accuracy when applied to pre-LT cirrhotic patients.AIM To assess the feasibility and accuracy of a machine learning model used to predict post-LT MACE in a regional cohort.METHODS This retrospective cohort study involved 575 LT patients from a Southern Brazilian academic center.We developed a predictive model for post-LT MACE(defined as a composite outcome of stroke,new-onset heart failure,severe arrhythmia,and myocardial infarction)using the extreme gradient boosting(XGBoost)machine learning model.We addressed missing data(below 20%)for relevant variables using the k-nearest neighbor imputation method,calculating the mean from the ten nearest neighbors for each case.The modeling dataset included 83 features,encompassing patient and laboratory data,cirrhosis complications,and pre-LT cardiac assessments.Model performance was assessed using the area under the receiver operating characteristic curve(AUROC).We also employed Shapley additive explanations(SHAP)to interpret feature impacts.The dataset was split into training(75%)and testing(25%)sets.Calibration was evaluated using the Brier score.We followed Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guidelines for reporting.Scikit-learn and SHAP in Python 3 were used for all analyses.The supplementary material includes code for model development and a user-friendly online MACE prediction calculator.RESULTS Of the 537 included patients,23(4.46%)developed in-hospital MACE,with a mean age at transplantation of 52.9 years.The majority,66.1%,were male.The XGBoost model achieved an impressive AUROC of 0.89 during the training stage.This model exhibited accuracy,precision,recall,and F1-score values of 0.84,0.85,0.80,and 0.79,respectively.Calibration,as assessed by the Brier score,indicated excellent model calibration with a score of 0.07.Furthermore,SHAP values highlighted the significance of certain variables in predicting postoperative MACE,with negative noninvasive cardiac stress testing,use of nonselective beta-blockers,direct bilirubin levels,blood type O,and dynamic alterations on myocardial perfusion scintigraphy being the most influential factors at the cohort-wide level.These results highlight the predictive capability of our XGBoost model in assessing the risk of post-LT MACE,making it a valuable tool for clinical practice.CONCLUSION Our study successfully assessed the feasibility and accuracy of the XGBoost machine learning model in predicting post-LT MACE,using both cardiovascular and hepatic variables.The model demonstrated impressive performance,aligning with literature findings,and exhibited excellent calibration.Notably,our cautious approach to prevent overfitting and data leakage suggests the stability of results when applied to prospective data,reinforcing the model’s value as a reliable tool for predicting post-LT MACE in clinical practice.

A deep reinforcement learning(DRL)based approach for well-testing interpretation to evaluate reservoir parameters

Parameter inversions in oil/gas reservoirs based on well test interpretations are of great significance in oil/gas industry.Automatic well test interpretations based on artificial intelligence are the most promising to solve the problem of non-unique solution.In this work,a new deep reinforcement learning(DRL)based approach is proposed for automatic curve matching for well test interpretation,by using the double deep Q-network(DDQN).The DDQN algorithms are applied to train agents for automatic parameter tuning in three conventional well-testing models.In addition,to alleviate the dimensional disaster problem of parameter space,an asynchronous parameter adjustment strategy is used to train the agent.Finally,field applications are carried out by using the new DRL approaches.Results show that step number required for the DDQN to complete the curve matching is the least among,when comparing the naive deep Q-network(naive DQN)and deep Q-network(DQN).We also show that DDQN can improve the robustness of curve matching in comparison with supervised machine learning algorithms.Using DDQN algorithm to perform 100 curve matching tests on three traditional well test models,the results show that the mean relative error of the parameters is 7.58%for the homogeneous model,10.66%for the radial composite model,and 12.79%for the dual porosity model.In the actual field application,it is found that a good curve fitting can be obtained with only 30 steps of parameter adjustment.

AWeb Application Fingerprint Recognition Method Based on Machine Learning

Web application fingerprint recognition is an effective security technology designed to identify and classify web applications,thereby enhancing the detection of potential threats and attacks.Traditional fingerprint recognition methods,which rely on preannotated feature matching,face inherent limitations due to the ever-evolving nature and diverse landscape of web applications.In response to these challenges,this work proposes an innovative web application fingerprint recognition method founded on clustering techniques.The method involves extensive data collection from the Tranco List,employing adjusted feature selection built upon Wappalyzer and noise reduction through truncated SVD dimensionality reduction.The core of the methodology lies in the application of the unsupervised OPTICS clustering algorithm,eliminating the need for preannotated labels.By transforming web applications into feature vectors and leveraging clustering algorithms,our approach accurately categorizes diverse web applications,providing comprehensive and precise fingerprint recognition.The experimental results,which are obtained on a dataset featuring various web application types,affirm the efficacy of the method,demonstrating its ability to achieve high accuracy and broad coverage.This novel approach not only distinguishes between different web application types effectively but also demonstrates superiority in terms of classification accuracy and coverage,offering a robust solution to the challenges of web application fingerprint recognition.

Improving Prediction Efficiency of Machine Learning Models for Cardiovascular Disease in IoST-Based Systems through Hyperparameter Optimization

This study explores the impact of hyperparameter optimization on machine learning models for predicting cardiovascular disease using data from an IoST(Internet of Sensing Things)device.Ten distinct machine learning approaches were implemented and systematically evaluated before and after hyperparameter tuning.Significant improvements were observed across various models,with SVM and Neural Networks consistently showing enhanced performance metrics such as F1-Score,recall,and precision.The study underscores the critical role of tailored hyperparameter tuning in optimizing these models,revealing diverse outcomes among algorithms.Decision Trees and Random Forests exhibited stable performance throughout the evaluation.While enhancing accuracy,hyperparameter optimization also led to increased execution time.Visual representations and comprehensive results support the findings,confirming the hypothesis that optimizing parameters can effectively enhance predictive capabilities in cardiovascular disease.This research contributes to advancing the understanding and application of machine learning in healthcare,particularly in improving predictive accuracy for cardiovascular disease management and intervention strategies.

一种基于Q-learning强化学习的导向性处理器安全性模糊测试方案

针对处理器安全性模糊测试在进行细粒度变异时遗传算法存在一定的盲目性,易使生成的测试用例触发相同类型漏洞的问题,提出了一种基于Q-learning强化学习的导向性处理器安全性模糊测试方案。通过测试用例的状态值和所触发的漏洞类型对应的权值构造奖励函数,使用强化学习指导生成具有针对性和导向性的测试用例,快速地触发不同类型的漏洞。在Hikey970平台上的实验验证了基于ARMv8的测试用例生成框架的有效性,并且相较于传统使用遗传算法作为反馈的策略,本文方案在相同时间内生成有效测试用例的的数量多19.15%,发现漏洞类型的数量多80.00%。

Research and Practice of Hybrid Teaching for Software Testing based on Interactive Learning

To address the problems of insufficient number of personalized exercises and cases and teachers’lack of grasp of students’weak knowledge points in the current software testing online courses,we study the strategy of establishing and updating intelligent exercise sets and case libraries and analyze the answers and dig out the weak points of knowledge through group intelligence reasoning and interactive machine learning methods.This will help teachers to make uniform and targeted explanations,reduce manual judgment,and achieve intelligent teaching quality reform,and implement the educational concepts of“keeping up with the times”and“teaching according to students’abilities”.

Improving Ultrasonic Testing by Using Machine Learning Framework Based on Model Interpretation Strategy

Ultrasonic testing(UT)is increasingly combined with machine learning(ML)techniques for intelligently identifying damage.Extracting signifcant features from UT data is essential for efcient defect characterization.Moreover,the hidden physics behind ML is unexplained,reducing the generalization capability and versatility of ML methods in UT.In this paper,a generally applicable ML framework based on the model interpretation strategy is proposed to improve the detection accuracy and computational efciency of UT.Firstly,multi-domain features are extracted from the UT signals with signal processing techniques to construct an initial feature space.Subsequently,a feature selection method based on model interpretable strategy(FS-MIS)is innovatively developed by integrating Shapley additive explanation(SHAP),flter method,embedded method and wrapper method.The most efective ML model and the optimal feature subset with better correlation to the target defects are determined self-adaptively.The proposed framework is validated by identifying and locating side-drilled holes(SDHs)with 0.5λcentral distance and different depths.An ultrasonic array probe is adopted to acquire FMC datasets from several aluminum alloy specimens containing two SDHs by experiments.The optimal feature subset selected by FS-MIS is set as the input of the chosen ML model to train and predict the times of arrival(ToAs)of the scattered waves emitted by adjacent SDHs.The experimental results demonstrate that the relative errors of the predicted ToAs are all below 3.67%with an average error of 0.25%,signifcantly improving the time resolution of UT signals.On this basis,the predicted ToAs are assigned to the corresponding original signals for decoupling overlapped pulse-echoes and reconstructing high-resolution FMC datasets.The imaging resolution is enhanced to 0.5λby implementing the total focusing method(TFM).The relative errors of hole depths and central distance are no more than 0.51%and 3.57%,respectively.Finally,the superior performance of the proposed FS-MIS is validated by comparing it with initial feature space and conventional dimensionality reduction techniques.

Personalized Learning Path Recommendations for Software Testing Courses Based on Knowledge Graphs

Software testing courses are characterized by strong practicality,comprehensiveness,and diversity.Due to the differences among students and the needs to design personalized solutions for their specific requirements,the design of the existing software testing courses fails to meet the demands for personalized learning.Knowledge graphs,with their rich semantics and good visualization effects,have a wide range of applications in the field of education.In response to the current problem of software testing courses which fails to meet the needs for personalized learning,this paper offers a learning path recommendation based on knowledge graphs to provide personalized learning paths for students.

A Test Items Knowledge Library System for Adaptive Learning

A test items knowledge library system of for adaptive learning is proposed in this paper. The first step is to carry out the quantity and quality analysis of the test items by using the Bloom＇s revised taxonomy and scale anchoring respectively to produce the characteristics for test items. A smoothing method of arbitrary anchoring revised from scale anchoring is first proposed to make tests more accurate in distinguishing test levels. In addition, raised three dimensional indicators based on the Bloom＇s revised taxonomy are adopted to validate test contents and therefore it concretely describes the examining function of items. The items obtained have the precise and concrete properties; an item knowledge library is therefore constructed combining teaching materials and items using the technologies of ontology and knowledge management. Finally, a knowledge library system of test items is established to achieve the purpose of adaptive learning for learners.

Correlation and Regression Analyses on the Washback effects of Achievement Oral Test on College English Learning

The purpose of this study was to investigate how washback effect of a reformed test influenced students'perspectives in their spoken English learning. This study was expected to enrich the existing literature in testing washback in English as a foreign language context.Findings indicate that the adding of oral part in achievement test does have a positive washback effect on students'learning. However, such a washback effect on students'learning is quite superficial and limited.

Vocabulary Test for Vocabulary Learning in Middle School

As one of the core aspects and basic unit in language,vocabulary plays a salient role in improving student's language ability.Besides,vocabulary learning strategies play an obviously important part in the whole process of language learning.By examining the theories and approach to the vocabulary test,found by linguists locally and abroad,as well as vocabulary learning strategies,this research tries to systematically explicate several vocabulary tests in order to help English teachers design more creditable vocabulary tests to evaluate studenfs vocabulary knowledge.At the same time,in hope that through this research,middle school students would be guided to use English vocabulary learning strategies more efficiently,thus enhancing their self-learning ability.

Towards Integrated Testing Approach: An Application of Cognitive Science and Deep Learning Principle

The use of multiple-choice(MC)question types has been one of the most contentious issues in language testing.Much has been said and written about the use of MC over the years.However,no attempt has ever been made to introduce any innovation in test item types.The researchers proposed a jumbled words test item(JW)based on cognitive science and deep learning principles,and addressed the feasibility of replacing the type of multiple-choice(MC)question with JW to meet the ongoing rapid development of language testing practice.Two research questions were proposed ad hoc,focusing on the co-relationship between JW and MC scores.RASCH-GZ was used to perform item analyses(Rasch,1960).The item difficulty parameters thus obtained were used to compare the two different test items.The sample data metric includes 40 Chinese participants.The findings revealed that correlation analysis revealed that the performance of the same group of subjects taking both JW and MC was not relevant(Pearson Corr=0).This is primarily due to the total elimination of guessing factors inherent in test-takers during JW test performance.Three factors were specified for the design of the JW test:compute program,test difficulty,and score acceptability.These all have three dimensions.Data collected through questionnaires were analyzed using EFA in SPSS V.24.0.KMOs(=0.867)were found to be approximately one and significance at 0.000(0.05),indicating that the construct of theuestionnaire thus designed has better validity for factor analysis.Three important conclusions were obtained,the implications of which could provide impetus for our testing counterparts to practice more precisely and correctly,potentially reshaping our overall language testing practice.Limitations and recommendations for future research were also discussed.

A Study on English Vocabulary Learning Strategies of College Students

The present study attempts to investigate Chinese non-English majors' vocabulary learning strategies.The descriptive statistics is carried out on the learners' vocabulary learning strategies;a correlational study is conducted between the students' vocabulary learning strategies and their vocabulary test scores.The result shows that students still prefer to learn vocabulary through the traditional way and that the dictionary strategies and rehearsal strategies are negatively related to vocabulary test scores,while all the other five categories are positively related to them.The pedagogical implications of this study are suggested for college teachers' instructions of vocabulary learning strategies.

Machine Learning Approach to Enhance the Performance of MNP?Labeled Lateral Flow Immunoassay

The use of magnetic nanoparticle(MNP)-labeled immunochromatography test strips(ICTSs) is very important for point-ofcare testing(POCT). However, common diagnostic methods cannot accurately analyze the weak magnetic signal from ICTSs, limiting the applications of POCT. In this study, an ultrasensitive multiplex biosensor was designed to overcome the limitations of capturing and normalization of the weak magnetic signal from MNPs on ICTSs. A machine learning model for sandwich assays was constructed and used to classify weakly positive and negative samples, which significantly enhanced the specificity and sensitivity. The potential clinical application was evaluated by detecting 50 human chorionic gonadotropin(HCG) samples and 59 myocardial infarction serum samples. The quantitative range for HCG was 1–1000 mIU mL^(-1) and the ideal detection limit was 0.014 mIU mL^(-1), which was well below the clinical threshold. Quantitative detection results of multiplex cardiac markers showed good linear correlations with standard values. The proposed multiplex assay can be readily adapted for identifying other biomolecules and also be used in other applications such as environmental monitoring, food analysis, and national security.

Lithium chloride ameliorates learning and memory ability and inhibits glycogen synthase kinase-3 beta activity in a mouse model of fragile X syndrome

In the present study,Fmr1 knockout mice （KO mice） were used as the model for fragile X syndrome.The results of step-through and step-down tests demonstrated that Fmr1 KO mice had shorter latencies and more error counts,indicating a learning and memory disorder.After treatment with 30,60,90,120,or 200 mg/kg lithium chloride,the learning and memory abilities of the Fmr1 KO mice were significantly ameliorated,in particular,the 200 mg/kg lithium chloride treatment had the most significant effect.Western blot analysis showed that lithium chloride significantly enhanced the expression of phosphorylated glycogen synthase kinase 3 beta,an inactive form of glycogen synthase kinase 3 beta,in the cerebral cortex and hippocampus of the Fmr1 KO mice.These results indicated that lithium chloride improved learning and memory in the Fmr1 KO mice,possibly by inhibiting glycogen synthase kinase 3 beta activity.

Development of a depression in Parkinson's disease prediction model using machine learning

BACKGROUND It is important to diagnose depression in Parkinson’s disease(DPD)as soon as possible and identify the predictors of depression to improve quality of life in Parkinson’s disease(PD)patients.AIM To develop a model for predicting DPD based on the support vector machine,while considering sociodemographic factors,health habits,Parkinson's symptoms,sleep behavior disorders,and neuropsychiatric indicators as predictors and provide baseline data for identifying DPD.METHODS This study analyzed 223 of 335 patients who were 60 years or older with PD.Depression was measured using the 30 items of the Geriatric Depression Scale,and the explanatory variables included PD-related motor signs,rapid eye movement sleep behavior disorders,and neuropsychological tests.The support vector machine was used to develop a DPD prediction model.RESULTS When the effects of PD motor symptoms were compared using“functional weight”,late motor complications(occurrence of levodopa-induced dyskinesia)were the most influential risk factors for Parkinson's symptoms.CONCLUSION It is necessary to develop customized screening tests that can detect DPD in the early stage and continuously monitor high-risk groups based on the factors related to DPD derived from this predictive model in order to maintain the emotional health of PD patients.

Risk stratification of patients who present with chest pain and have normal troponins using a machine learning model

BACKGROUND Risk stratification tools exist for patients presenting with chest pain to the emergency room and have achieved the recommended negative predictive value(NPV)of 99%.However,due to low positive predictive value(PPV),current stratification tools result in unwarranted investigations such as serial laboratory tests and cardiac stress tests(CSTs).AIM To create a machine learning model(MLM)for risk stratification of chest pain with a better PPV.METHODS This retrospective cohort study used de-identified hospital data from January 2016 until November 2021.Inclusion criteria were patients aged>21 years who presented to the ER,had at least two serum troponins measured,were subsequently admitted to the hospital,and had a CST within 4 d of presentation.Exclusion criteria were elevated troponin value(>0.05 ng/mL)and missing values for body mass index.The primary outcome was abnormal CST.Demographics,coronary artery disease(CAD)history,hypertension,hyperlipidemia,diabetes mellitus,chronic kidney disease,obesity,and smoking were evaluated as potential risk factors for abnormal CST.Patients were also categorized into a high-risk group(CAD history or more than two risk factors)and a low-risk group(all other patients)for comparison.Bivariate analysis was performed using a χ^(2) test or Fisher’s exact test.Age was compared by t test.Binomial regression(BR),random forest,and XGBoost MLMs were used for prediction.Bootstrapping was used for the internal validation of prediction models.BR was also used for inference.Alpha criterion was set at 0.05 for all statistical tests.R software was used for statistical analysis.RESULTS The final cohort of the study included 2328 patients,of which 245(10.52%)patients had abnormal CST.When adjusted for covariates in the BR model,male sex[risk ratio(RR)=1.52,95%confidence interval(CI):1.2-1.94,P<0.001],CAD history(RR=4.46,95%CI:3.08-6.72,P<0.001),and hyperlipidemia(RR=3.87,95%CI:2.12-8.12,P<0.001)remained statistically significant.Incidence of abnormal CST was 12.2%in the high-risk group and 2.3%in the low-risk group(RR=5.31,95%CI:2.75-10.24,P<0.001).The XGBoost model had the best PPV of 24.33%,with an NPV of 91.34%for abnormal CST.CONCLUSION The XGBoost MLM achieved a PPV of 24.33%for an abnormal CST,which is better than current stratification tools(13.00%-17.50%).This highlights the beneficial potential of MLMs in clinical decision-making.