Vascular etiology is the second most prevalent cause of cognitive impairment globally.Endothelin-1,which is produced and secreted by endothelial cells and astrocytes,is implicated in the pathogenesis of stroke.However...Vascular etiology is the second most prevalent cause of cognitive impairment globally.Endothelin-1,which is produced and secreted by endothelial cells and astrocytes,is implicated in the pathogenesis of stroke.However,the way in which changes in astrocytic endothelin-1 lead to poststroke cognitive deficits following transient middle cerebral artery occlusion is not well understood.Here,using mice in which astrocytic endothelin-1 was overexpressed,we found that the selective overexpression of endothelin-1 by astrocytic cells led to ischemic stroke-related dementia(1 hour of ischemia;7 days,28 days,or 3 months of reperfusion).We also revealed that astrocytic endothelin-1 overexpression contributed to the role of neural stem cell proliferation but impaired neurogenesis in the dentate gyrus of the hippocampus after middle cerebral artery occlusion.Comprehensive proteome profiles and western blot analysis confirmed that levels of glial fibrillary acidic protein and peroxiredoxin 6,which were differentially expressed in the brain,were significantly increased in mice with astrocytic endothelin-1 overexpression in comparison with wild-type mice 28 days after ischemic stroke.Moreover,the levels of the enriched differentially expressed proteins were closely related to lipid metabolism,as indicated by Kyoto Encyclopedia of Genes and Genomes pathway analysis.Liquid chromatography-mass spectrometry nontargeted metabolite profiling of brain tissues showed that astrocytic endothelin-1 overexpression altered lipid metabolism products such as glycerol phosphatidylcholine,sphingomyelin,and phosphatidic acid.Overall,this study demonstrates that astrocytic endothelin-1 overexpression can impair hippocampal neurogenesis and that it is correlated with lipid metabolism in poststroke cognitive dysfunction.展开更多
The increasing prevalence of Internet of Things(IoT)devices has introduced a new phase of connectivity in recent years and,concurrently,has opened the floodgates for growing cyber threats.Among the myriad of potential...The increasing prevalence of Internet of Things(IoT)devices has introduced a new phase of connectivity in recent years and,concurrently,has opened the floodgates for growing cyber threats.Among the myriad of potential attacks,Denial of Service(DoS)attacks and Distributed Denial of Service(DDoS)attacks remain a dominant concern due to their capability to render services inoperable by overwhelming systems with an influx of traffic.As IoT devices often lack the inherent security measures found in more mature computing platforms,the need for robust DoS/DDoS detection systems tailored to IoT is paramount for the sustainable development of every domain that IoT serves.In this study,we investigate the effectiveness of three machine learning(ML)algorithms:extreme gradient boosting(XGB),multilayer perceptron(MLP)and random forest(RF),for the detection of IoTtargeted DoS/DDoS attacks and three feature engineering methods that have not been used in the existing stateof-the-art,and then employed the best performing algorithm to design a prototype of a novel real-time system towards detection of such DoS/DDoS attacks.The CICIoT2023 dataset was derived from the latest real-world IoT traffic,incorporates both benign and malicious network traffic patterns and after data preprocessing and feature engineering,the data was fed into our models for both training and validation,where findings suggest that while all threemodels exhibit commendable accuracy in detectingDoS/DDoS attacks,the use of particle swarmoptimization(PSO)for feature selection has made great improvements in the performance(accuracy,precsion recall and F1-score of 99.93%for XGB)of the ML models and their execution time(491.023 sceonds for XGB)compared to recursive feature elimination(RFE)and randomforest feature importance(RFI)methods.The proposed real-time system for DoS/DDoS attack detection entails the implementation of an platform capable of effectively processing and analyzing network traffic in real-time.This involvesemploying the best-performing ML algorithmfor detection and the integration of warning mechanisms.We believe this approach will significantly enhance the field of security research and continue to refine it based on future insights and developments.展开更多
Wi Fi and fingerprinting localization method have been a hot topic in indoor positioning because of their universality and location-related features.The basic assumption of fingerprinting localization is that the rece...Wi Fi and fingerprinting localization method have been a hot topic in indoor positioning because of their universality and location-related features.The basic assumption of fingerprinting localization is that the received signal strength indication(RSSI)distance is accord with the location distance.Therefore,how to efficiently match the current RSSI of the user with the RSSI in the fingerprint database is the key to achieve high-accuracy localization.In this paper,a particle swarm optimization-extreme learning machine(PSO-ELM)algorithm is proposed on the basis of the original fingerprinting localization.Firstly,we collect the RSSI of the experimental area to construct the fingerprint database,and the ELM algorithm is applied to the online stages to determine the corresponding relation between the location of the terminal and the RSSI it receives.Secondly,PSO algorithm is used to improve the bias and weight of ELM neural network,and the global optimal results are obtained.Finally,extensive simulation results are presented.It is shown that the proposed algorithm can effectively reduce mean error of localization and improve positioning accuracy when compared with K-Nearest Neighbor(KNN),Kmeans and Back-propagation(BP)algorithms.展开更多
This study comprehensively examines the current state of deep learning (DL) usage in indoor positioning.It emphasizes the significance and efficiency of convolutional neural networks (CNNs) and recurrent neuralnetwork...This study comprehensively examines the current state of deep learning (DL) usage in indoor positioning.It emphasizes the significance and efficiency of convolutional neural networks (CNNs) and recurrent neuralnetworks (RNNs). Unlike prior studies focused on single sensor modalities like Wi-Fi or Bluetooth, this researchexplores the integration of multiple sensor modalities (e.g.,Wi-Fi, Bluetooth, Ultra-Wideband, ZigBee) to expandindoor localization methods, particularly in obstructed environments. It addresses the challenge of precise objectlocalization, introducing a novel hybrid DL approach using received signal information (RSI), Received SignalStrength (RSS), and Channel State Information (CSI) data to enhance accuracy and stability. Moreover, thestudy introduces a device-free indoor localization algorithm, offering a significant advancement with potentialobject or individual tracking applications. It recognizes the increasing importance of indoor positioning forlocation-based services. It anticipates future developments while acknowledging challenges such as multipathinterference, noise, data standardization, and scarcity of labeled data. This research contributes significantly toindoor localization technology, offering adaptability, device independence, and multifaceted DL-based solutionsfor real-world challenges and future advancements. Thus, the proposed work addresses challenges in objectlocalization precision and introduces a novel hybrid deep learning approach, contributing to advancing locationcentricservices.While deep learning-based indoor localization techniques have improved accuracy, challenges likedata noise, standardization, and availability of training data persist. However, ongoing developments are expectedto enhance indoor positioning systems to meet real-world demands.展开更多
Federated learning is widely used to solve the problem of data decentralization and can provide privacy protectionfor data owners. However, since multiple participants are required in federated learning, this allows a...Federated learning is widely used to solve the problem of data decentralization and can provide privacy protectionfor data owners. However, since multiple participants are required in federated learning, this allows attackers tocompromise. Byzantine attacks pose great threats to federated learning. Byzantine attackers upload maliciouslycreated local models to the server to affect the prediction performance and training speed of the global model. Todefend against Byzantine attacks, we propose a Byzantine robust federated learning scheme based on backdoortriggers. In our scheme, backdoor triggers are embedded into benign data samples, and then malicious localmodels can be identified by the server according to its validation dataset. Furthermore, we calculate the adjustmentfactors of local models according to the parameters of their final layers, which are used to defend against datapoisoning-based Byzantine attacks. To further enhance the robustness of our scheme, each localmodel is weightedand aggregated according to the number of times it is identified as malicious. Relevant experimental data showthat our scheme is effective against Byzantine attacks in both independent identically distributed (IID) and nonindependentidentically distributed (non-IID) scenarios.展开更多
Cyberspace is extremely dynamic,with new attacks arising daily.Protecting cybersecurity controls is vital for network security.Deep Learning(DL)models find widespread use across various fields,with cybersecurity being...Cyberspace is extremely dynamic,with new attacks arising daily.Protecting cybersecurity controls is vital for network security.Deep Learning(DL)models find widespread use across various fields,with cybersecurity being one of the most crucial due to their rapid cyberattack detection capabilities on networks and hosts.The capabilities of DL in feature learning and analyzing extensive data volumes lead to the recognition of network traffic patterns.This study presents novel lightweight DL models,known as Cybernet models,for the detection and recognition of various cyber Distributed Denial of Service(DDoS)attacks.These models were constructed to have a reasonable number of learnable parameters,i.e.,less than 225,000,hence the name“lightweight.”This not only helps reduce the number of computations required but also results in faster training and inference times.Additionally,these models were designed to extract features in parallel from 1D Convolutional Neural Networks(CNN)and Long Short-Term Memory(LSTM),which makes them unique compared to earlier existing architectures and results in better performance measures.To validate their robustness and effectiveness,they were tested on the CIC-DDoS2019 dataset,which is an imbalanced and large dataset that contains different types of DDoS attacks.Experimental results revealed that bothmodels yielded promising results,with 99.99% for the detectionmodel and 99.76% for the recognition model in terms of accuracy,precision,recall,and F1 score.Furthermore,they outperformed the existing state-of-the-art models proposed for the same task.Thus,the proposed models can be used in cyber security research domains to successfully identify different types of attacks with a high detection and recognition rate.展开更多
In the era of the Internet of Things(IoT),the proliferation of connected devices has raised security concerns,increasing the risk of intrusions into diverse systems.Despite the convenience and efficiency offered by Io...In the era of the Internet of Things(IoT),the proliferation of connected devices has raised security concerns,increasing the risk of intrusions into diverse systems.Despite the convenience and efficiency offered by IoT technology,the growing number of IoT devices escalates the likelihood of attacks,emphasizing the need for robust security tools to automatically detect and explain threats.This paper introduces a deep learning methodology for detecting and classifying distributed denial of service(DDoS)attacks,addressing a significant security concern within IoT environments.An effective procedure of deep transfer learning is applied to utilize deep learning backbones,which is then evaluated on two benchmarking datasets of DDoS attacks in terms of accuracy and time complexity.By leveraging several deep architectures,the study conducts thorough binary and multiclass experiments,each varying in the complexity of classifying attack types and demonstrating real-world scenarios.Additionally,this study employs an explainable artificial intelligence(XAI)AI technique to elucidate the contribution of extracted features in the process of attack detection.The experimental results demonstrate the effectiveness of the proposed method,achieving a recall of 99.39%by the XAI bidirectional long short-term memory(XAI-BiLSTM)model.展开更多
Random noise attenuation is significant in seismic data processing.Supervised deep learning-based denoising methods have been widely developed and applied in recent years.In practice,it is often time-consuming and lab...Random noise attenuation is significant in seismic data processing.Supervised deep learning-based denoising methods have been widely developed and applied in recent years.In practice,it is often time-consuming and laborious to obtain noise-free data for supervised learning.Therefore,we propose a novel deep learning framework to denoise prestack seismic data without clean labels,which trains a high-resolution residual neural network(SRResnet)with noisy data for input and the same valid data with different noise for output.Since valid signals in noisy sample pairs are spatially correlated and random noise is spatially independent and unpredictable,the model can learn the features of valid data while suppressing random noise.Noisy data targets are generated by a simple conventional method without fine-tuning parameters.The initial estimates allow signal or noise leakage as the network does not require clean labels.The Monte Carlo strategy is applied to select training patches for increasing valid patches and expanding training datasets.Transfer learning is used to improve the generalization of real data processing.The synthetic and real data tests perform better than the commonly used state-of-the-art denoising methods.展开更多
In situations when the precise position of a machine is unknown,localization becomes crucial.This research focuses on improving the position prediction accuracy over long-range(LoRa)network using an optimized machine ...In situations when the precise position of a machine is unknown,localization becomes crucial.This research focuses on improving the position prediction accuracy over long-range(LoRa)network using an optimized machine learning-based technique.In order to increase the prediction accuracy of the reference point position on the data collected using the fingerprinting method over LoRa technology,this study proposed an optimized machine learning(ML)based algorithm.Received signal strength indicator(RSSI)data from the sensors at different positions was first gathered via an experiment through the LoRa network in a multistory round layout building.The noise factor is also taken into account,and the signal-to-noise ratio(SNR)value is recorded for every RSSI measurement.This study concludes the examination of reference point accuracy with the modified KNN method(MKNN).MKNN was created to more precisely anticipate the position of the reference point.The findings showed that MKNN outperformed other algorithms in terms of accuracy and complexity.展开更多
Objective:In the realm of Class I pathogens,Burkholderia pseudomallei(BP)stands out for its propensity to induce severe pathogenicity.Investigating the intricate interactions between BP and host cells is imperative fo...Objective:In the realm of Class I pathogens,Burkholderia pseudomallei(BP)stands out for its propensity to induce severe pathogenicity.Investigating the intricate interactions between BP and host cells is imperative for comprehending the dynamics of BP infection and discerning biomarkers indicative of the host cell response process.Methods:mRNA extraction from BP-infected mouse macrophages constituted the initial step of our study.Employing gene expression arrays,the extracted RNA underwent conversion into digital signals.The percentile shift method facilitated data processing,with the identification of genes manifesting significant differences accomplished through the application of the t-test.Subsequently,a comprehensive analysis involving Gene Ontology enrichment and Kyoto Encyclopedia of Genes and Genomes pathway was conducted on the differentially expressed genes(DEGs).Leveraging the ESTIMATE algorithm,gene signatures were utilized to compute risk scores for gene expression data.Support vector machine analysis and gene enrichment scores were instrumental in establishing correlations between biomarkers and macrophages,followed by an evaluation of the predictive power of the identified biomarkers.Results:The functional and pathway associations of the DEGs predominantly centered around G protein-coupled receptors.A noteworthy positive correlation emerged between the blue module,consisting of 416 genes,and the StromaScore.FZD4,identified through support vector machine analysis among intersecting genes,indicated a robust interaction with macrophages,suggesting its potential as a robust biomarker.FZD4 exhibited commendable predictive efficacy,with BP infection inducing its expression in both macrophages and mouse lung tissue.Western blotting in macrophages confirmed a significant upregulation of FZD4 expression from 0.5 to 24 h post-infection.In mouse lung tissue,FZD4 manifested higher expression in the cytoplasm of pulmonary epithelial cells in BP-infected lungs than in the control group.Conclusion:Thesefindings underscore the upregulation of FZD4 expression by BP in both macrophages and lung tissue,pointing to its prospective role as a biomarker in the pathogenesis of BP infection.展开更多
Recent work has validated a new method for estimating the grain size of microgranular materials in the range of tens to hundreds of micrometers using laser-induced breakdown spectroscopy(LIBS).In this situation,a piec...Recent work has validated a new method for estimating the grain size of microgranular materials in the range of tens to hundreds of micrometers using laser-induced breakdown spectroscopy(LIBS).In this situation,a piecewise univariate model must be constructed to estimate grain size due to the complex dependence of the plasma formation environment on grain size.In the present work,we tentatively construct a unified calibration model suitable for LIBS-based estimation of those grain sizes.Specifically,two unified multivariate calibration models are constructed based on back-propagation neural network(BPNN)algorithms using feature selection strategies with and without considering prior information.By detailed analysis of the performances of the two multivariate models,it was found that a unified calibration model can be successfully constructed based on BPNN algorithms for estimating the grain size in the range of tens to hundreds of micrometers.It was also found that the model constructed with a priorguided feature selection strategy had better prediction performance.This study has practical significance in developing the technology for material analysis using LIBS,especially when the LIBS signal exhibits a complex dependence on the material parameter to be estimated.展开更多
This work proposes a recorded recurrent twin delayed deep deterministic(RRTD3)policy gradient algorithm to solve the challenge of constructing guidance laws for intercepting endoatmospheric maneuvering missiles with u...This work proposes a recorded recurrent twin delayed deep deterministic(RRTD3)policy gradient algorithm to solve the challenge of constructing guidance laws for intercepting endoatmospheric maneuvering missiles with uncertainties and observation noise.The attack-defense engagement scenario is modeled as a partially observable Markov decision process(POMDP).Given the benefits of recurrent neural networks(RNNs)in processing sequence information,an RNN layer is incorporated into the agent’s policy network to alleviate the bottleneck of traditional deep reinforcement learning methods while dealing with POMDPs.The measurements from the interceptor’s seeker during each guidance cycle are combined into one sequence as the input to the policy network since the detection frequency of an interceptor is usually higher than its guidance frequency.During training,the hidden states of the RNN layer in the policy network are recorded to overcome the partially observable problem that this RNN layer causes inside the agent.The training curves show that the proposed RRTD3 successfully enhances data efficiency,training speed,and training stability.The test results confirm the advantages of the RRTD3-based guidance laws over some conventional guidance laws.展开更多
Determining homogeneous domains statistically is helpful for engineering geological modeling and rock mass stability evaluation.In this text,a technique that can integrate lithology,geotechnical and structural informa...Determining homogeneous domains statistically is helpful for engineering geological modeling and rock mass stability evaluation.In this text,a technique that can integrate lithology,geotechnical and structural information is proposed to delineate homogeneous domains.This technique is then applied to a high and steep slope along a road.First,geological and geotechnical domains were described based on lithology,faults,and shear zones.Next,topological manifolds were used to eliminate the incompatibility between orientations and other parameters(i.e.trace length and roughness)so that the data concerning various properties of each discontinuity can be matched and characterized in the same Euclidean space.Thus,the influence of implicit combined effect in between parameter sequences on the homogeneous domains could be considered.Deep learning technique was employed to quantify abstract features of the characterization images of discontinuity properties,and to assess the similarity of rock mass structures.The results show that the technique can effectively distinguish structural variations and outperform conventional methods.It can handle multisource engineering geological information and multiple discontinuity parameters.This technique can also minimize the interference of human factors and delineate homogeneous domains based on orientations or multi-parameter with arbitrary distributions to satisfy different engineering requirements.展开更多
With the increasing awareness of privacy protection and the improvement of relevant laws,federal learning has gradually become a new choice for cross-agency and cross-device machine learning.In order to solve the prob...With the increasing awareness of privacy protection and the improvement of relevant laws,federal learning has gradually become a new choice for cross-agency and cross-device machine learning.In order to solve the problems of privacy leakage,high computational overhead and high traffic in some federated learning schemes,this paper proposes amultiplicative double privacymask algorithm which is convenient for homomorphic addition aggregation.The combination of homomorphic encryption and secret sharing ensures that the server cannot compromise user privacy from the private gradient uploaded by the participants.At the same time,the proposed TQRR(Top-Q-Random-R)gradient selection algorithm is used to filter the gradient of encryption and upload efficiently,which reduces the computing overhead of 51.78%and the traffic of 64.87%on the premise of ensuring the accuracy of themodel,whichmakes the framework of privacy protection federated learning lighter to adapt to more miniaturized federated learning terminals.展开更多
This study aimed to address the challenge of accurately and reliably detecting tomatoes in dense planting environments,a critical prerequisite for the automation implementation of robotic harvesting.However,the heavy ...This study aimed to address the challenge of accurately and reliably detecting tomatoes in dense planting environments,a critical prerequisite for the automation implementation of robotic harvesting.However,the heavy reliance on extensive manually annotated datasets for training deep learning models still poses significant limitations to their application in real-world agricultural production environments.To overcome these limitations,we employed domain adaptive learning approach combined with the YOLOv5 model to develop a novel tomato detection model called as TDA-YOLO(tomato detection domain adaptation).We designated the normal illumination scenes in dense planting environments as the source domain and utilized various other illumination scenes as the target domain.To construct bridge mechanism between source and target domains,neural preset for color style transfer is introduced to generate a pseudo-dataset,which served to deal with domain discrepancy.Furthermore,this study combines the semi-supervised learning method to enable the model to extract domain-invariant features more fully,and uses knowledge distillation to improve the model's ability to adapt to the target domain.Additionally,for purpose of promoting inference speed and low computational demand,the lightweight FasterNet network was integrated into the YOLOv5's C3 module,creating a modified C3_Faster module.The experimental results demonstrated that the proposed TDA-YOLO model significantly outperformed original YOLOv5s model,achieving a mAP(mean average precision)of 96.80%for tomato detection across diverse scenarios in dense planting environments,increasing by 7.19 percentage points;Compared with the latest YOLOv8 and YOLOv9,it is also 2.17 and 1.19 percentage points higher,respectively.The model's average detection time per image was an impressive 15 milliseconds,with a FLOPs(floating point operations per second)count of 13.8 G.After acceleration processing,the detection accuracy of the TDA-YOLO model on the Jetson Xavier NX development board is 90.95%,the mAP value is 91.35%,and the detection time of each image is 21 ms,which can still meet the requirements of real-time detection of tomatoes in dense planting environment.The experimental results show that the proposed TDA-YOLO model can accurately and quickly detect tomatoes in dense planting environment,and at the same time avoid the use of a large number of annotated data,which provides technical support for the development of automatic harvesting systems for tomatoes and other fruits.展开更多
In this editorial,we comment on the article Adolescent suicide risk factors and the integration of social-emotional skills in school-based prevention programs by Liu et al.While the article focused on the issue of sui...In this editorial,we comment on the article Adolescent suicide risk factors and the integration of social-emotional skills in school-based prevention programs by Liu et al.While the article focused on the issue of suicide and social-emotional learning programs as a possible intervention,we here discuss evidence of other reported outcomes and if it could be an effective way to prevent substance abuse among adolescents.展开更多
Internet of Things(IoTs)provides better solutions in various fields,namely healthcare,smart transportation,home,etc.Recognizing Denial of Service(DoS)outbreaks in IoT platforms is significant in certifying the accessi...Internet of Things(IoTs)provides better solutions in various fields,namely healthcare,smart transportation,home,etc.Recognizing Denial of Service(DoS)outbreaks in IoT platforms is significant in certifying the accessibility and integrity of IoT systems.Deep learning(DL)models outperform in detecting complex,non-linear relationships,allowing them to effectually severe slight deviations fromnormal IoT activities that may designate a DoS outbreak.The uninterrupted observation and real-time detection actions of DL participate in accurate and rapid detection,permitting proactive reduction events to be executed,hence securing the IoT network’s safety and functionality.Subsequently,this study presents pigeon-inspired optimization with a DL-based attack detection and classification(PIODL-ADC)approach in an IoT environment.The PIODL-ADC approach implements a hyperparameter-tuned DL method for Distributed Denial-of-Service(DDoS)attack detection in an IoT platform.Initially,the PIODL-ADC model utilizes Z-score normalization to scale input data into a uniformformat.For handling the convolutional and adaptive behaviors of IoT,the PIODL-ADCmodel employs the pigeon-inspired optimization(PIO)method for feature selection to detect the related features,considerably enhancing the recognition’s accuracy.Also,the Elman Recurrent Neural Network(ERNN)model is utilized to recognize and classify DDoS attacks.Moreover,reptile search algorithm(RSA)based hyperparameter tuning is employed to improve the precision and robustness of the ERNN method.A series of investigational validations is made to ensure the accomplishment of the PIODL-ADC method.The experimental outcome exhibited that the PIODL-ADC method shows greater accomplishment when related to existing models,with a maximum accuracy of 99.81%.展开更多
With the high level of proliferation of connected mobile devices,the risk of intrusion becomes higher.Artificial Intelligence(AI)and Machine Learning(ML)algorithms started to feature in protection software and showed ...With the high level of proliferation of connected mobile devices,the risk of intrusion becomes higher.Artificial Intelligence(AI)and Machine Learning(ML)algorithms started to feature in protection software and showed effective results.These algorithms are nonetheless hindered by the lack of rich datasets and compounded by the appearance of new categories of malware such that the race between attackers’malware,especially with the assistance of Artificial Intelligence tools and protection solutions makes these systems and frameworks lose effectiveness quickly.In this article,we present a framework for mobile malware detection based on a new dataset containing new categories of mobile malware.We focus on categories of malware that were not tested before by Machine Learning algorithms proven effective in malware detection.We carefully select an optimal number of features,do necessary preprocessing,and then apply Machine Learning algorithms to discover malicious code effectively.From our experiments,we have found that the Random Forest algorithm is the best-performing algorithm with such mobile malware with detection rates of around 99%.We compared our results from this work and found that they are aligned well with our previous work.We also compared our work with State-of-the-Art works of others and found that the results are very close and competitive.展开更多
With the proportion of intelligent services in the industrial internet of things(IIoT)rising rapidly,its data dependency and decomposability increase the difficulty of scheduling computing resources.In this paper,we p...With the proportion of intelligent services in the industrial internet of things(IIoT)rising rapidly,its data dependency and decomposability increase the difficulty of scheduling computing resources.In this paper,we propose an intelligent service computing framework.In the framework,we take the long-term rewards of its important participants,edge service providers,as the optimization goal,which is related to service delay and computing cost.Considering the different update frequencies of data deployment and service offloading,double-timescale reinforcement learning is utilized in the framework.In the small-scale strategy,the frequent concurrency of services and the difference in service time lead to the fuzzy relationship between reward and action.To solve the fuzzy reward problem,a reward mapping-based reinforcement learning(RMRL)algorithm is proposed,which enables the agent to learn the relationship between reward and action more clearly.The large time scale strategy adopts the improved Monte Carlo tree search(MCTS)algorithm to improve the learning speed.The simulation results show that the strategy is superior to popular reinforcement learning algorithms such as double Q-learning(DDQN)and dueling Q-learning(dueling-DQN)in learning speed,and the reward is also increased by 14%.展开更多
基金financially supported by the National Natural Science Foundation of China,No.81303115,81774042 (both to XC)the Pearl River S&T Nova Program of Guangzhou,No.201806010025 (to XC)+3 种基金the Specialty Program of Guangdong Province Hospital of Chinese Medicine of China,No.YN2018ZD07 (to XC)the Natural Science Foundatior of Guangdong Province of China,No.2023A1515012174 (to JL)the Science and Technology Program of Guangzhou of China,No.20210201 0268 (to XC),20210201 0339 (to JS)Guangdong Provincial Key Laboratory of Research on Emergency in TCM,Nos.2018-75,2019-140 (to JS)
文摘Vascular etiology is the second most prevalent cause of cognitive impairment globally.Endothelin-1,which is produced and secreted by endothelial cells and astrocytes,is implicated in the pathogenesis of stroke.However,the way in which changes in astrocytic endothelin-1 lead to poststroke cognitive deficits following transient middle cerebral artery occlusion is not well understood.Here,using mice in which astrocytic endothelin-1 was overexpressed,we found that the selective overexpression of endothelin-1 by astrocytic cells led to ischemic stroke-related dementia(1 hour of ischemia;7 days,28 days,or 3 months of reperfusion).We also revealed that astrocytic endothelin-1 overexpression contributed to the role of neural stem cell proliferation but impaired neurogenesis in the dentate gyrus of the hippocampus after middle cerebral artery occlusion.Comprehensive proteome profiles and western blot analysis confirmed that levels of glial fibrillary acidic protein and peroxiredoxin 6,which were differentially expressed in the brain,were significantly increased in mice with astrocytic endothelin-1 overexpression in comparison with wild-type mice 28 days after ischemic stroke.Moreover,the levels of the enriched differentially expressed proteins were closely related to lipid metabolism,as indicated by Kyoto Encyclopedia of Genes and Genomes pathway analysis.Liquid chromatography-mass spectrometry nontargeted metabolite profiling of brain tissues showed that astrocytic endothelin-1 overexpression altered lipid metabolism products such as glycerol phosphatidylcholine,sphingomyelin,and phosphatidic acid.Overall,this study demonstrates that astrocytic endothelin-1 overexpression can impair hippocampal neurogenesis and that it is correlated with lipid metabolism in poststroke cognitive dysfunction.
文摘The increasing prevalence of Internet of Things(IoT)devices has introduced a new phase of connectivity in recent years and,concurrently,has opened the floodgates for growing cyber threats.Among the myriad of potential attacks,Denial of Service(DoS)attacks and Distributed Denial of Service(DDoS)attacks remain a dominant concern due to their capability to render services inoperable by overwhelming systems with an influx of traffic.As IoT devices often lack the inherent security measures found in more mature computing platforms,the need for robust DoS/DDoS detection systems tailored to IoT is paramount for the sustainable development of every domain that IoT serves.In this study,we investigate the effectiveness of three machine learning(ML)algorithms:extreme gradient boosting(XGB),multilayer perceptron(MLP)and random forest(RF),for the detection of IoTtargeted DoS/DDoS attacks and three feature engineering methods that have not been used in the existing stateof-the-art,and then employed the best performing algorithm to design a prototype of a novel real-time system towards detection of such DoS/DDoS attacks.The CICIoT2023 dataset was derived from the latest real-world IoT traffic,incorporates both benign and malicious network traffic patterns and after data preprocessing and feature engineering,the data was fed into our models for both training and validation,where findings suggest that while all threemodels exhibit commendable accuracy in detectingDoS/DDoS attacks,the use of particle swarmoptimization(PSO)for feature selection has made great improvements in the performance(accuracy,precsion recall and F1-score of 99.93%for XGB)of the ML models and their execution time(491.023 sceonds for XGB)compared to recursive feature elimination(RFE)and randomforest feature importance(RFI)methods.The proposed real-time system for DoS/DDoS attack detection entails the implementation of an platform capable of effectively processing and analyzing network traffic in real-time.This involvesemploying the best-performing ML algorithmfor detection and the integration of warning mechanisms.We believe this approach will significantly enhance the field of security research and continue to refine it based on future insights and developments.
基金supported in part by the National Natural Science Foundation of China(U2001213 and 61971191)in part by the Beijing Natural Science Foundation under Grant L182018 and L201011+2 种基金in part by National Key Research and Development Project(2020YFB1807204)in part by the Key project of Natural Science Foundation of Jiangxi Province(20202ACBL202006)in part by the Innovation Fund Designated for Graduate Students of Jiangxi Province(YC2020-S321)。
文摘Wi Fi and fingerprinting localization method have been a hot topic in indoor positioning because of their universality and location-related features.The basic assumption of fingerprinting localization is that the received signal strength indication(RSSI)distance is accord with the location distance.Therefore,how to efficiently match the current RSSI of the user with the RSSI in the fingerprint database is the key to achieve high-accuracy localization.In this paper,a particle swarm optimization-extreme learning machine(PSO-ELM)algorithm is proposed on the basis of the original fingerprinting localization.Firstly,we collect the RSSI of the experimental area to construct the fingerprint database,and the ELM algorithm is applied to the online stages to determine the corresponding relation between the location of the terminal and the RSSI it receives.Secondly,PSO algorithm is used to improve the bias and weight of ELM neural network,and the global optimal results are obtained.Finally,extensive simulation results are presented.It is shown that the proposed algorithm can effectively reduce mean error of localization and improve positioning accuracy when compared with K-Nearest Neighbor(KNN),Kmeans and Back-propagation(BP)algorithms.
基金the Fundamental Research Grant Scheme-FRGS/1/2021/ICT09/MMU/02/1,Ministry of Higher Education,Malaysia.
文摘This study comprehensively examines the current state of deep learning (DL) usage in indoor positioning.It emphasizes the significance and efficiency of convolutional neural networks (CNNs) and recurrent neuralnetworks (RNNs). Unlike prior studies focused on single sensor modalities like Wi-Fi or Bluetooth, this researchexplores the integration of multiple sensor modalities (e.g.,Wi-Fi, Bluetooth, Ultra-Wideband, ZigBee) to expandindoor localization methods, particularly in obstructed environments. It addresses the challenge of precise objectlocalization, introducing a novel hybrid DL approach using received signal information (RSI), Received SignalStrength (RSS), and Channel State Information (CSI) data to enhance accuracy and stability. Moreover, thestudy introduces a device-free indoor localization algorithm, offering a significant advancement with potentialobject or individual tracking applications. It recognizes the increasing importance of indoor positioning forlocation-based services. It anticipates future developments while acknowledging challenges such as multipathinterference, noise, data standardization, and scarcity of labeled data. This research contributes significantly toindoor localization technology, offering adaptability, device independence, and multifaceted DL-based solutionsfor real-world challenges and future advancements. Thus, the proposed work addresses challenges in objectlocalization precision and introduces a novel hybrid deep learning approach, contributing to advancing locationcentricservices.While deep learning-based indoor localization techniques have improved accuracy, challenges likedata noise, standardization, and availability of training data persist. However, ongoing developments are expectedto enhance indoor positioning systems to meet real-world demands.
基金in part by the National Social Science Foundation of China under Grant 20BTQ058in part by the Natural Science Foundation of Hunan Province under Grant 2023JJ50033。
文摘Federated learning is widely used to solve the problem of data decentralization and can provide privacy protectionfor data owners. However, since multiple participants are required in federated learning, this allows attackers tocompromise. Byzantine attacks pose great threats to federated learning. Byzantine attackers upload maliciouslycreated local models to the server to affect the prediction performance and training speed of the global model. Todefend against Byzantine attacks, we propose a Byzantine robust federated learning scheme based on backdoortriggers. In our scheme, backdoor triggers are embedded into benign data samples, and then malicious localmodels can be identified by the server according to its validation dataset. Furthermore, we calculate the adjustmentfactors of local models according to the parameters of their final layers, which are used to defend against datapoisoning-based Byzantine attacks. To further enhance the robustness of our scheme, each localmodel is weightedand aggregated according to the number of times it is identified as malicious. Relevant experimental data showthat our scheme is effective against Byzantine attacks in both independent identically distributed (IID) and nonindependentidentically distributed (non-IID) scenarios.
文摘Cyberspace is extremely dynamic,with new attacks arising daily.Protecting cybersecurity controls is vital for network security.Deep Learning(DL)models find widespread use across various fields,with cybersecurity being one of the most crucial due to their rapid cyberattack detection capabilities on networks and hosts.The capabilities of DL in feature learning and analyzing extensive data volumes lead to the recognition of network traffic patterns.This study presents novel lightweight DL models,known as Cybernet models,for the detection and recognition of various cyber Distributed Denial of Service(DDoS)attacks.These models were constructed to have a reasonable number of learnable parameters,i.e.,less than 225,000,hence the name“lightweight.”This not only helps reduce the number of computations required but also results in faster training and inference times.Additionally,these models were designed to extract features in parallel from 1D Convolutional Neural Networks(CNN)and Long Short-Term Memory(LSTM),which makes them unique compared to earlier existing architectures and results in better performance measures.To validate their robustness and effectiveness,they were tested on the CIC-DDoS2019 dataset,which is an imbalanced and large dataset that contains different types of DDoS attacks.Experimental results revealed that bothmodels yielded promising results,with 99.99% for the detectionmodel and 99.76% for the recognition model in terms of accuracy,precision,recall,and F1 score.Furthermore,they outperformed the existing state-of-the-art models proposed for the same task.Thus,the proposed models can be used in cyber security research domains to successfully identify different types of attacks with a high detection and recognition rate.
文摘In the era of the Internet of Things(IoT),the proliferation of connected devices has raised security concerns,increasing the risk of intrusions into diverse systems.Despite the convenience and efficiency offered by IoT technology,the growing number of IoT devices escalates the likelihood of attacks,emphasizing the need for robust security tools to automatically detect and explain threats.This paper introduces a deep learning methodology for detecting and classifying distributed denial of service(DDoS)attacks,addressing a significant security concern within IoT environments.An effective procedure of deep transfer learning is applied to utilize deep learning backbones,which is then evaluated on two benchmarking datasets of DDoS attacks in terms of accuracy and time complexity.By leveraging several deep architectures,the study conducts thorough binary and multiclass experiments,each varying in the complexity of classifying attack types and demonstrating real-world scenarios.Additionally,this study employs an explainable artificial intelligence(XAI)AI technique to elucidate the contribution of extracted features in the process of attack detection.The experimental results demonstrate the effectiveness of the proposed method,achieving a recall of 99.39%by the XAI bidirectional long short-term memory(XAI-BiLSTM)model.
基金employed by Petroleum Exploration and Production Research Institute of SINOPECfunded by the National Key R&D Program of China(2021YFC3000701).
文摘Random noise attenuation is significant in seismic data processing.Supervised deep learning-based denoising methods have been widely developed and applied in recent years.In practice,it is often time-consuming and laborious to obtain noise-free data for supervised learning.Therefore,we propose a novel deep learning framework to denoise prestack seismic data without clean labels,which trains a high-resolution residual neural network(SRResnet)with noisy data for input and the same valid data with different noise for output.Since valid signals in noisy sample pairs are spatially correlated and random noise is spatially independent and unpredictable,the model can learn the features of valid data while suppressing random noise.Noisy data targets are generated by a simple conventional method without fine-tuning parameters.The initial estimates allow signal or noise leakage as the network does not require clean labels.The Monte Carlo strategy is applied to select training patches for increasing valid patches and expanding training datasets.Transfer learning is used to improve the generalization of real data processing.The synthetic and real data tests perform better than the commonly used state-of-the-art denoising methods.
基金The research will be funded by the Multimedia University,Department of Information Technology,Persiaran Multimedia,63100,Cyberjaya,Selangor,Malaysia.
文摘In situations when the precise position of a machine is unknown,localization becomes crucial.This research focuses on improving the position prediction accuracy over long-range(LoRa)network using an optimized machine learning-based technique.In order to increase the prediction accuracy of the reference point position on the data collected using the fingerprinting method over LoRa technology,this study proposed an optimized machine learning(ML)based algorithm.Received signal strength indicator(RSSI)data from the sensors at different positions was first gathered via an experiment through the LoRa network in a multistory round layout building.The noise factor is also taken into account,and the signal-to-noise ratio(SNR)value is recorded for every RSSI measurement.This study concludes the examination of reference point accuracy with the modified KNN method(MKNN).MKNN was created to more precisely anticipate the position of the reference point.The findings showed that MKNN outperformed other algorithms in terms of accuracy and complexity.
基金The study was supported by Yuying Program Incubation Project of General Hospital of Center Theater(ZZYFH202104)Wuhan Young and Middle-Aged Medical Backbone Talent Project 2020(2020-55)Logistics Research Program Project 2019(CLB19J029).
文摘Objective:In the realm of Class I pathogens,Burkholderia pseudomallei(BP)stands out for its propensity to induce severe pathogenicity.Investigating the intricate interactions between BP and host cells is imperative for comprehending the dynamics of BP infection and discerning biomarkers indicative of the host cell response process.Methods:mRNA extraction from BP-infected mouse macrophages constituted the initial step of our study.Employing gene expression arrays,the extracted RNA underwent conversion into digital signals.The percentile shift method facilitated data processing,with the identification of genes manifesting significant differences accomplished through the application of the t-test.Subsequently,a comprehensive analysis involving Gene Ontology enrichment and Kyoto Encyclopedia of Genes and Genomes pathway was conducted on the differentially expressed genes(DEGs).Leveraging the ESTIMATE algorithm,gene signatures were utilized to compute risk scores for gene expression data.Support vector machine analysis and gene enrichment scores were instrumental in establishing correlations between biomarkers and macrophages,followed by an evaluation of the predictive power of the identified biomarkers.Results:The functional and pathway associations of the DEGs predominantly centered around G protein-coupled receptors.A noteworthy positive correlation emerged between the blue module,consisting of 416 genes,and the StromaScore.FZD4,identified through support vector machine analysis among intersecting genes,indicated a robust interaction with macrophages,suggesting its potential as a robust biomarker.FZD4 exhibited commendable predictive efficacy,with BP infection inducing its expression in both macrophages and mouse lung tissue.Western blotting in macrophages confirmed a significant upregulation of FZD4 expression from 0.5 to 24 h post-infection.In mouse lung tissue,FZD4 manifested higher expression in the cytoplasm of pulmonary epithelial cells in BP-infected lungs than in the control group.Conclusion:Thesefindings underscore the upregulation of FZD4 expression by BP in both macrophages and lung tissue,pointing to its prospective role as a biomarker in the pathogenesis of BP infection.
基金supported in part by the National Key Research and Development Program of China(No.2017YFA0402300)National Natural Science Foundation of China(Nos.U2241288 and 11974359)Major Science and Technology Project of Gansu Province(No.22ZD6FA021-5)。
文摘Recent work has validated a new method for estimating the grain size of microgranular materials in the range of tens to hundreds of micrometers using laser-induced breakdown spectroscopy(LIBS).In this situation,a piecewise univariate model must be constructed to estimate grain size due to the complex dependence of the plasma formation environment on grain size.In the present work,we tentatively construct a unified calibration model suitable for LIBS-based estimation of those grain sizes.Specifically,two unified multivariate calibration models are constructed based on back-propagation neural network(BPNN)algorithms using feature selection strategies with and without considering prior information.By detailed analysis of the performances of the two multivariate models,it was found that a unified calibration model can be successfully constructed based on BPNN algorithms for estimating the grain size in the range of tens to hundreds of micrometers.It was also found that the model constructed with a priorguided feature selection strategy had better prediction performance.This study has practical significance in developing the technology for material analysis using LIBS,especially when the LIBS signal exhibits a complex dependence on the material parameter to be estimated.
基金supported by the National Natural Science Foundation of China(Grant No.12072090)。
文摘This work proposes a recorded recurrent twin delayed deep deterministic(RRTD3)policy gradient algorithm to solve the challenge of constructing guidance laws for intercepting endoatmospheric maneuvering missiles with uncertainties and observation noise.The attack-defense engagement scenario is modeled as a partially observable Markov decision process(POMDP).Given the benefits of recurrent neural networks(RNNs)in processing sequence information,an RNN layer is incorporated into the agent’s policy network to alleviate the bottleneck of traditional deep reinforcement learning methods while dealing with POMDPs.The measurements from the interceptor’s seeker during each guidance cycle are combined into one sequence as the input to the policy network since the detection frequency of an interceptor is usually higher than its guidance frequency.During training,the hidden states of the RNN layer in the policy network are recorded to overcome the partially observable problem that this RNN layer causes inside the agent.The training curves show that the proposed RRTD3 successfully enhances data efficiency,training speed,and training stability.The test results confirm the advantages of the RRTD3-based guidance laws over some conventional guidance laws.
基金the National Natural Science Foundation of China(Grant Nos.41941017 and U1702241).
文摘Determining homogeneous domains statistically is helpful for engineering geological modeling and rock mass stability evaluation.In this text,a technique that can integrate lithology,geotechnical and structural information is proposed to delineate homogeneous domains.This technique is then applied to a high and steep slope along a road.First,geological and geotechnical domains were described based on lithology,faults,and shear zones.Next,topological manifolds were used to eliminate the incompatibility between orientations and other parameters(i.e.trace length and roughness)so that the data concerning various properties of each discontinuity can be matched and characterized in the same Euclidean space.Thus,the influence of implicit combined effect in between parameter sequences on the homogeneous domains could be considered.Deep learning technique was employed to quantify abstract features of the characterization images of discontinuity properties,and to assess the similarity of rock mass structures.The results show that the technique can effectively distinguish structural variations and outperform conventional methods.It can handle multisource engineering geological information and multiple discontinuity parameters.This technique can also minimize the interference of human factors and delineate homogeneous domains based on orientations or multi-parameter with arbitrary distributions to satisfy different engineering requirements.
基金supported by the National Natural Science Foundation of China(Grant Nos.62172436,62102452)the National Key Research and Development Program of China(2023YFB3106100,2021YFB3100100)the Natural Science Foundation of Shaanxi Province(2023-JC-YB-584).
文摘With the increasing awareness of privacy protection and the improvement of relevant laws,federal learning has gradually become a new choice for cross-agency and cross-device machine learning.In order to solve the problems of privacy leakage,high computational overhead and high traffic in some federated learning schemes,this paper proposes amultiplicative double privacymask algorithm which is convenient for homomorphic addition aggregation.The combination of homomorphic encryption and secret sharing ensures that the server cannot compromise user privacy from the private gradient uploaded by the participants.At the same time,the proposed TQRR(Top-Q-Random-R)gradient selection algorithm is used to filter the gradient of encryption and upload efficiently,which reduces the computing overhead of 51.78%and the traffic of 64.87%on the premise of ensuring the accuracy of themodel,whichmakes the framework of privacy protection federated learning lighter to adapt to more miniaturized federated learning terminals.
基金The National Natural Science Foundation of China (32371993)The Natural Science Research Key Project of Anhui Provincial University(2022AH040125&2023AH040135)The Key Research and Development Plan of Anhui Province (202204c06020022&2023n06020057)。
文摘This study aimed to address the challenge of accurately and reliably detecting tomatoes in dense planting environments,a critical prerequisite for the automation implementation of robotic harvesting.However,the heavy reliance on extensive manually annotated datasets for training deep learning models still poses significant limitations to their application in real-world agricultural production environments.To overcome these limitations,we employed domain adaptive learning approach combined with the YOLOv5 model to develop a novel tomato detection model called as TDA-YOLO(tomato detection domain adaptation).We designated the normal illumination scenes in dense planting environments as the source domain and utilized various other illumination scenes as the target domain.To construct bridge mechanism between source and target domains,neural preset for color style transfer is introduced to generate a pseudo-dataset,which served to deal with domain discrepancy.Furthermore,this study combines the semi-supervised learning method to enable the model to extract domain-invariant features more fully,and uses knowledge distillation to improve the model's ability to adapt to the target domain.Additionally,for purpose of promoting inference speed and low computational demand,the lightweight FasterNet network was integrated into the YOLOv5's C3 module,creating a modified C3_Faster module.The experimental results demonstrated that the proposed TDA-YOLO model significantly outperformed original YOLOv5s model,achieving a mAP(mean average precision)of 96.80%for tomato detection across diverse scenarios in dense planting environments,increasing by 7.19 percentage points;Compared with the latest YOLOv8 and YOLOv9,it is also 2.17 and 1.19 percentage points higher,respectively.The model's average detection time per image was an impressive 15 milliseconds,with a FLOPs(floating point operations per second)count of 13.8 G.After acceleration processing,the detection accuracy of the TDA-YOLO model on the Jetson Xavier NX development board is 90.95%,the mAP value is 91.35%,and the detection time of each image is 21 ms,which can still meet the requirements of real-time detection of tomatoes in dense planting environment.The experimental results show that the proposed TDA-YOLO model can accurately and quickly detect tomatoes in dense planting environment,and at the same time avoid the use of a large number of annotated data,which provides technical support for the development of automatic harvesting systems for tomatoes and other fruits.
文摘In this editorial,we comment on the article Adolescent suicide risk factors and the integration of social-emotional skills in school-based prevention programs by Liu et al.While the article focused on the issue of suicide and social-emotional learning programs as a possible intervention,we here discuss evidence of other reported outcomes and if it could be an effective way to prevent substance abuse among adolescents.
文摘Internet of Things(IoTs)provides better solutions in various fields,namely healthcare,smart transportation,home,etc.Recognizing Denial of Service(DoS)outbreaks in IoT platforms is significant in certifying the accessibility and integrity of IoT systems.Deep learning(DL)models outperform in detecting complex,non-linear relationships,allowing them to effectually severe slight deviations fromnormal IoT activities that may designate a DoS outbreak.The uninterrupted observation and real-time detection actions of DL participate in accurate and rapid detection,permitting proactive reduction events to be executed,hence securing the IoT network’s safety and functionality.Subsequently,this study presents pigeon-inspired optimization with a DL-based attack detection and classification(PIODL-ADC)approach in an IoT environment.The PIODL-ADC approach implements a hyperparameter-tuned DL method for Distributed Denial-of-Service(DDoS)attack detection in an IoT platform.Initially,the PIODL-ADC model utilizes Z-score normalization to scale input data into a uniformformat.For handling the convolutional and adaptive behaviors of IoT,the PIODL-ADCmodel employs the pigeon-inspired optimization(PIO)method for feature selection to detect the related features,considerably enhancing the recognition’s accuracy.Also,the Elman Recurrent Neural Network(ERNN)model is utilized to recognize and classify DDoS attacks.Moreover,reptile search algorithm(RSA)based hyperparameter tuning is employed to improve the precision and robustness of the ERNN method.A series of investigational validations is made to ensure the accomplishment of the PIODL-ADC method.The experimental outcome exhibited that the PIODL-ADC method shows greater accomplishment when related to existing models,with a maximum accuracy of 99.81%.
文摘With the high level of proliferation of connected mobile devices,the risk of intrusion becomes higher.Artificial Intelligence(AI)and Machine Learning(ML)algorithms started to feature in protection software and showed effective results.These algorithms are nonetheless hindered by the lack of rich datasets and compounded by the appearance of new categories of malware such that the race between attackers’malware,especially with the assistance of Artificial Intelligence tools and protection solutions makes these systems and frameworks lose effectiveness quickly.In this article,we present a framework for mobile malware detection based on a new dataset containing new categories of mobile malware.We focus on categories of malware that were not tested before by Machine Learning algorithms proven effective in malware detection.We carefully select an optimal number of features,do necessary preprocessing,and then apply Machine Learning algorithms to discover malicious code effectively.From our experiments,we have found that the Random Forest algorithm is the best-performing algorithm with such mobile malware with detection rates of around 99%.We compared our results from this work and found that they are aligned well with our previous work.We also compared our work with State-of-the-Art works of others and found that the results are very close and competitive.
基金supported by the National Natural Science Foundation of China(No.62171051)。
文摘With the proportion of intelligent services in the industrial internet of things(IIoT)rising rapidly,its data dependency and decomposability increase the difficulty of scheduling computing resources.In this paper,we propose an intelligent service computing framework.In the framework,we take the long-term rewards of its important participants,edge service providers,as the optimization goal,which is related to service delay and computing cost.Considering the different update frequencies of data deployment and service offloading,double-timescale reinforcement learning is utilized in the framework.In the small-scale strategy,the frequent concurrency of services and the difference in service time lead to the fuzzy relationship between reward and action.To solve the fuzzy reward problem,a reward mapping-based reinforcement learning(RMRL)algorithm is proposed,which enables the agent to learn the relationship between reward and action more clearly.The large time scale strategy adopts the improved Monte Carlo tree search(MCTS)algorithm to improve the learning speed.The simulation results show that the strategy is superior to popular reinforcement learning algorithms such as double Q-learning(DDQN)and dueling Q-learning(dueling-DQN)in learning speed,and the reward is also increased by 14%.