Despite the maturity of ensemble numerical weather prediction(NWP),the resulting forecasts are still,more often than not,under-dispersed.As such,forecast calibration tools have become popular.Among those tools,quantil...Despite the maturity of ensemble numerical weather prediction(NWP),the resulting forecasts are still,more often than not,under-dispersed.As such,forecast calibration tools have become popular.Among those tools,quantile regression(QR)is highly competitive in terms of both flexibility and predictive performance.Nevertheless,a long-standing problem of QR is quantile crossing,which greatly limits the interpretability of QR-calibrated forecasts.On this point,this study proposes a non-crossing quantile regression neural network(NCQRNN),for calibrating ensemble NWP forecasts into a set of reliable quantile forecasts without crossing.The overarching design principle of NCQRNN is to add on top of the conventional QRNN structure another hidden layer,which imposes a non-decreasing mapping between the combined output from nodes of the last hidden layer to the nodes of the output layer,through a triangular weight matrix with positive entries.The empirical part of the work considers a solar irradiance case study,in which four years of ensemble irradiance forecasts at seven locations,issued by the European Centre for Medium-Range Weather Forecasts,are calibrated via NCQRNN,as well as via an eclectic mix of benchmarking models,ranging from the naïve climatology to the state-of-the-art deep-learning and other non-crossing models.Formal and stringent forecast verification suggests that the forecasts post-processed via NCQRNN attain the maximum sharpness subject to calibration,amongst all competitors.Furthermore,the proposed conception to resolve quantile crossing is remarkably simple yet general,and thus has broad applicability as it can be integrated with many shallow-and deep-learning-based neural networks.展开更多
The distribution of the nuclear ground-state spin in a two-body random ensemble(TBRE)was studied using a general classification neural network(NN)model with two-body interaction matrix elements as input features and t...The distribution of the nuclear ground-state spin in a two-body random ensemble(TBRE)was studied using a general classification neural network(NN)model with two-body interaction matrix elements as input features and the corresponding ground-state spins as labels or output predictions.The quantum many-body system problem exceeds the capability of our optimized NNs in terms of accurately predicting the ground-state spin of each sample within the TBRE.However,our NN model effectively captured the statistical properties of the ground-state spin because it learned the empirical regularity of the ground-state spin distribution in TBRE,as discovered by physicists.展开更多
Smart Industrial environments use the Industrial Internet of Things(IIoT)for their routine operations and transform their industrial operations with intelligent and driven approaches.However,IIoT devices are vulnerabl...Smart Industrial environments use the Industrial Internet of Things(IIoT)for their routine operations and transform their industrial operations with intelligent and driven approaches.However,IIoT devices are vulnerable to cyber threats and exploits due to their connectivity with the internet.Traditional signature-based IDS are effective in detecting known attacks,but they are unable to detect unknown emerging attacks.Therefore,there is the need for an IDS which can learn from data and detect new threats.Ensemble Machine Learning(ML)and individual Deep Learning(DL)based IDS have been developed,and these individual models achieved low accuracy;however,their performance can be improved with the ensemble stacking technique.In this paper,we have proposed a Deep Stacked Neural Network(DSNN)based IDS,which consists of two stacked Convolutional Neural Network(CNN)models as base learners and Extreme Gradient Boosting(XGB)as the meta learner.The proposed DSNN model was trained and evaluated with the next-generation dataset,TON_IoT.Several pre-processing techniques were applied to prepare a dataset for the model,including ensemble feature selection and the SMOTE technique.Accuracy,precision,recall,F1-score,and false positive rates were used to evaluate the performance of the proposed ensemble model.Our experimental results showed that the accuracy for binary classification is 99.61%,which is better than in the baseline individual DL and ML models.In addition,the model proposed for IDS has been compared with similar models.The proposed DSNN achieved better performance metrics than the other models.The proposed DSNN model will be used to develop enhanced IDS for threat mitigation in smart industrial environments.展开更多
The back propagation(BP)neural network method is widely used in bathymetry based on multispectral satellite imagery.However,the classical BP neural network method faces a potential problem because it easily falls into...The back propagation(BP)neural network method is widely used in bathymetry based on multispectral satellite imagery.However,the classical BP neural network method faces a potential problem because it easily falls into a local minimum,leading to model training failure.This study confirmed that the local minimum problem of the BP neural network method exists in the bathymetry field and cannot be ignored.Furthermore,to solve the local minimum problem of the BP neural network method,a bathymetry method based on a BP neural network and ensemble learning(BPEL)is proposed.First,the remote sensing imagery and training sample were used as input datasets,and the BP method was used as the base learner to produce multiple water depth inversion results.Then,a new ensemble strategy,namely the minimum outlying degree method,was proposed and used to integrate the water depth inversion results.Finally,an ensemble bathymetric map was acquired.Anda Reef,northeastern Jiuzhang Atoll,and Pingtan coastal zone were selected as test cases to validate the proposed method.Compared with the BP neural network method,the root-mean-square error and the average relative error of the BPEL method can reduce by 0.65–2.84 m and 16%–46%in the three test cases at most.The results showed that the proposed BPEL method could solve the local minimum problem of the BP neural network method and obtain highly robust and accurate bathymetric maps.展开更多
The existing strategy for evaluating the damage condition of structures mostly focuses on feedback supplied by traditional visualmethods,which may result in an unreliable damage characterization due to inspector subje...The existing strategy for evaluating the damage condition of structures mostly focuses on feedback supplied by traditional visualmethods,which may result in an unreliable damage characterization due to inspector subjectivity or insufficient level of expertise.As a result,a robust,reliable,and repeatable method of damage identification is required.Ensemble learning algorithms for identifying structural damage are evaluated in this article,which use deep convolutional neural networks,including simple averaging,integrated stacking,separate stacking,and hybridweighted averaging ensemble and differential evolution(WAE-DE)ensemblemodels.Damage identification is carried out on three types of damage.The proposed algorithms are used to analyze the damage of 4585 structural images.The effectiveness of the ensemble learning techniques is evaluated using the confusion matrix.For the testing dataset,the confusion matrix achieved an accuracy of 94 percent and a minimum recall of 92 percent for the best model(WAE-DE)in distinguishing damage types as flexural,shear,combined,or undamaged.展开更多
Considering the defects of low accuracy and slow speed existing in traditional flood loss assessment, firstly, the technical route of flood loss assessment was presented based on the neural network ensemble. Secondly,...Considering the defects of low accuracy and slow speed existing in traditional flood loss assessment, firstly, the technical route of flood loss assessment was presented based on the neural network ensemble. Secondly, through the study of certain country of Poyang Lake district, the flood loss assessment indicators of the test area were analyzed and extracted by utilizing analytic hierarchy process (AHP), and the weights of the impact factors were assigned. Subsequently, the approaches to generate individuals and conclusions of neural network ensemble model were also investigated. In the platform of C# language and neural network library under AForge.NET open source, a flood loss assessment program which could rapidly build neural network ensemble models was developed. Finally, the proposed method was tested and verified. The comparison results between the assessment results of the proposed method and the actual statistical flood loss proved the feasibility of this method, thus a new approach for flood loss assessment was provided.展开更多
Existing web-based security applications have failed in many situations due to the great intelligence of attackers.Among web applications,Cross-Site Scripting(XSS)is one of the dangerous assaults experienced while mod...Existing web-based security applications have failed in many situations due to the great intelligence of attackers.Among web applications,Cross-Site Scripting(XSS)is one of the dangerous assaults experienced while modifying an organization's or user's information.To avoid these security challenges,this article proposes a novel,all-encompassing combination of machine learning(NB,SVM,k-NN)and deep learning(RNN,CNN,LSTM)frameworks for detecting and defending against XSS attacks with high accuracy and efficiency.Based on the representation,a novel idea for merging stacking ensemble with web applications,termed“hybrid stacking”,is proposed.In order to implement the aforementioned methods,four distinct datasets,each of which contains both safe and unsafe content,are considered.The hybrid detection method can adaptively identify the attacks from the URL,and the defense mechanism inherits the advantages of URL encoding with dictionary-based mapping to improve prediction accuracy,accelerate the training process,and effectively remove the unsafe JScript/JavaScript keywords from the URL.The simulation results show that the proposed hybrid model is more efficient than the existing detection methods.It produces more than 99.5%accurate XSS attack classification results(accuracy,precision,recall,f1_score,and Receiver Operating Characteristic(ROC))and is highly resistant to XSS attacks.In order to ensure the security of the server's information,the proposed hybrid approach is demonstrated in a real-time environment.展开更多
Potato cyst nematodes(PCNs)are a significant threat to potato production,having caused substantial damage in many countries.Predicting the future distribution of PCN species is crucial to implementing effective biosec...Potato cyst nematodes(PCNs)are a significant threat to potato production,having caused substantial damage in many countries.Predicting the future distribution of PCN species is crucial to implementing effective biosecurity strategies,especially given the impact of climate change on pest species invasion and distribution.Machine learning(ML),specifically ensemble models,has emerged as a powerful tool in predicting species distributions due to its ability to learn and make predictions based on complex data sets.Thus,this research utilised advanced machine learning techniques to predict the distribution of PCN species under climate change conditions,providing the initial element for invasion risk assessment.We first used Global Climate Models to generate homogeneous climate predictors to mitigate the variation among predictors.Then,five machine learning models were employed to build two groups of ensembles,single-algorithm ensembles(ESA)and multi-algorithm ensembles(EMA),and compared their performances.In this research,the EMA did not always perform better than the ESA,and the ESA of Artificial Neural Network gave the highest performance while being cost-effective.Prediction results indicated that the distribution range of PCNs would shift northward with a decrease in tropical zones and an increase in northern latitudes.However,the total area of suitable regions will not change significantly,occupying 16-20%of the total land surface(18%under current conditions).This research alerts policymakers and practitioners to the risk of PCNs’incursion into new regions.Additionally,this ML process offers the capability to track changes in the distribution of various species and provides scientifically grounded evidence for formulating long-term biosecurity plans for their control.展开更多
It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental informat...It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental information,as well as have high accuracy and quick inference speed.Network ensemble architecture is a good choice to improve network performance.However,it is unsuitable for real-time applications on the autonomous system.To tackle this problem,a new neural network ensemble named partial-shared ensemble network(PSENet)is presented.PSENet changes network ensemble architecture from parallel architecture to scatter architecture and merges multiple component networks together to accelerate the inference speed.To make component networks independent of each other,a training method is designed to train the network ensemble architecture.Experiments on Camvid and CIFAR-10 reveal that PSENet achieves quick inference speed while maintaining the ability of ensemble learning.In the real world,PSENet is deployed on the unmanned system and deals with vision tasks such as semantic segmentation and environmental prediction in different fields.展开更多
High quality, agricultural nutrient distribution maps are necessary for precision management, but depend on initial soil sample analyses and interpolation techniques. To examine the methodologies for and explore the c...High quality, agricultural nutrient distribution maps are necessary for precision management, but depend on initial soil sample analyses and interpolation techniques. To examine the methodologies for and explore the capability of interpolating soil properties based on neural network ensemble residual kriging, a silage field at Hayes, Northern Ireland, UK, was selected for this study with all samples being split into independent training and validation data sets. The training data set, comprised of five soil properties: soil pH, soil available P, soil available K, soil available Mg and soil available S,was modeled for spatial variability using 1) neural network ensemble residual kriging, 2) neural network ensemble and 3) kriging with their accuracies being estimated by means of the validation data sets. Ordinary kriging of the residuals provided accurate local estimates, while final estimates were produced as a sum of the artificial neural network (ANN)ensemble estimates and the ordinary kriging estimates of the residuals. Compared to kriging and neural network ensemble,the neural network ensemble residual kriging achieved better or similar accuracy for predicting and estimating contour maps. Thus, the results demonstrated that ANN ensemble residual kriging was an efficient alternative to the conventional geo-statistical models that were usually used for interpolation of a data set in the soil science area.展开更多
Land evaluation factors often contain continuous-, discrete- and nominal-valued attributes. In traditional land evaluation, these different attributes are usually graded into categorical indexes by land resource exper...Land evaluation factors often contain continuous-, discrete- and nominal-valued attributes. In traditional land evaluation, these different attributes are usually graded into categorical indexes by land resource experts, and the evaluation results rely heavily on experts' experiences. In order to overcome the shortcoming, we presented a fuzzy neural network ensemble method that did not require grading the evaluation factors into categorical indexes and could evaluate land resources by using the three kinds of attribute values directly. A fuzzy back propagation neural network (BPNN), a fuzzy radial basis function neural network (RBFNN), a fuzzy BPNN ensemble, and a fuzzy RBFNN ensemble were used to evaluate the land resources in Guangdong Province. The evaluation results by using the fuzzy BPNN ensemble and the fuzzy RBFNN ensemble were much better than those by using the single fuzzy BPNN and the single fuzzy RBFNN, and the error rate of the single fuzzy RBFNN or fuzzy RBFNN ensemble was lower than that of the single fuzzy BPNN or fuzzy BPNN ensemble, respectively. By using the fuzzy neural network ensembles, the validity of land resource evaluation was improved and reliance on land evaluators' experiences was considerably reduced.展开更多
An effective ensemble should consist of a set of networks that are both accurate and diverse. We propose a novel clustering-based selective algorithm for constructing neural network ensemble, where clustering technolo...An effective ensemble should consist of a set of networks that are both accurate and diverse. We propose a novel clustering-based selective algorithm for constructing neural network ensemble, where clustering technology is used to classify trained networks according to similarity and optimally select the most accurate individual network from each cluster to make up the ensemble. Empirical studies on regression of four typical datasets showed that this approach yields significantly smaller en- semble achieving better performance than other traditional ones such as Bagging and Boosting. The bias variance decomposition of the predictive error shows that the success of the proposed approach may lie in its properly tuning the bias/variance trade-off to reduce the prediction error (the sum of bias2 and variance).展开更多
A clustering algorithm based selective neural networks ensemble (CLUSEN) is proposed to predict the degree of malignancy in brain glioma. Since the degree prediction of malignancy is critical before brain surgery, m...A clustering algorithm based selective neural networks ensemble (CLUSEN) is proposed to predict the degree of malignancy in brain glioma. Since the degree prediction of malignancy is critical before brain surgery, many learning methods are used like rule induction algorithm, single neural networks, support vector machines, etc. Ensemble learning methods can improve the generalization of single learning machine, and are becoming popular in the machine learning and medical data processing communities. The procedure of CLUSEN can efficiently remove redundancy learning individuals and help improve the diversity of ensemble methods. CLUSEN is used to predict the degree of malignancy in brain glioma. Experimental results on a set of brain glioma data show that, compared to support vector machines, rule induction and single neural networks, the classification accuracy of CLUSEN is higher.展开更多
Residual useful life(RUL)prediction is a key issue for improving efficiency of aircraft engines and reducing their maintenance cost.Owing to various failure mechanism and operating environment,the application of class...Residual useful life(RUL)prediction is a key issue for improving efficiency of aircraft engines and reducing their maintenance cost.Owing to various failure mechanism and operating environment,the application of classical models in RUL prediction of aircraft engines is fairly difficult.In this study,a novel RUL prognostics method based on using ensemble recurrent neural network to process massive sensor data is proposed.First of all,sensor data obtained from the aircraft engines are preprocessed to eliminate singular values,reduce random fluctuation and preserve degradation trend of the raw sensor data.Secondly,three kinds of recurrent neural networks(RNN),including ordinary RNN,long shortterm memory(LSTM),and gated recurrent unit(GRU),are individually constructed.Thirdly,ensemble learning mechanism is designed to merge the above RNNs for producing a more accurate RUL prediction.The effectiveness of the proposed method is validated using two characteristically different turbofan engine datasets.Experimental results show a competitive performance of the proposed method in comparison with typical methods reported in literatures.展开更多
Irretrievable loss of vision is the predominant result of Glaucoma in the retina.Recently,multiple approaches have paid attention to the automatic detection of glaucoma on fundus images.Due to the interlace of blood v...Irretrievable loss of vision is the predominant result of Glaucoma in the retina.Recently,multiple approaches have paid attention to the automatic detection of glaucoma on fundus images.Due to the interlace of blood vessels and the herculean task involved in glaucoma detection,the exactly affected site of the optic disc of whether small or big size cup,is deemed challenging.Spatially Based Ellipse Fitting Curve Model(SBEFCM)classification is suggested based on the Ensemble for a reliable diagnosis of Glaucomain theOptic Cup(OC)and Optic Disc(OD)boundary correspondingly.This research deploys the Ensemble Convolutional Neural Network(CNN)classification for classifying Glaucoma or Diabetes Retinopathy(DR).The detection of the boundary between the OC and the OD is performed by the SBEFCM,which is the latest weighted ellipse fitting model.The SBEFCM that enhances and widens the multi-ellipse fitting technique is proposed here.There is a preprocessing of input fundus image besides segmentation of blood vessels to avoid interlacing surrounding tissues and blood vessels.The ascertaining of OCandODboundary,which characterizedmany output factors for glaucoma detection,has been developed by EnsembleCNNclassification,which includes detecting sensitivity,specificity,precision,andArea Under the receiver operating characteristic Curve(AUC)values accurately by an innovative SBEFCM.In terms of contrast,the proposed Ensemble CNNsignificantly outperformed the current methods.展开更多
There has been many methods in constructing neural network (NN) ensembles, where the method of simultaneous training has succeed in generalization performance and efficiency. But just like regular methods of constru...There has been many methods in constructing neural network (NN) ensembles, where the method of simultaneous training has succeed in generalization performance and efficiency. But just like regular methods of constructing NN ensembles, it follows the two steps, first training component networks, and then combining them. As the two steps being independent, an assumption is used to facilitate interactions among NNs during the training stage. This paper presents a compact ensemble method which integrates the two steps of ensemble construction into one step by attempting to train individual NNs in an ensemble and weigh the individual members adaptively according to their individual performance in the same learning process. This provides an opportunity for the individual NNs to interact with each other based on their real contributions to the ensemble. The classification performance of NN compact ensemble (NNCE) was validated through some benchmark problems in machine learning, including Australian credit card assessment, pima Indians diabetes, heart disease, breast cancer and glass. Compared with other ensembles, the classification error rate of NNCE can be decreased by 0.45% to 68%. In addition, the NNCE was applied to fault diagnosis for rolling element bearing. The 11 time-domain statistical features are extracted as the properties of data, and the NNCE is employed to classify the data. With the results of several experiments, the compact ensemble method is shown to give good generalization performance. The compact ensemble method can recognize the different fault types and various fault degrees of the same fault type.展开更多
This paper introduces supervised learning model, and surveys related research work. The paper is organised as follows. A supervised learning model is firstly described. The bias-variance trade-off is then discussed fo...This paper introduces supervised learning model, and surveys related research work. The paper is organised as follows. A supervised learning model is firstly described. The bias-variance trade-off is then discussed for the supervised learning model. Based on the bias-variance trade-off, both the single neural network approaches and the neural network ensemble approaches are overviewed, and problems with the existing approaches are indicated. Finally, the paper concludes with specifying potential future research directions.展开更多
Chemical processes are complex, for which traditional neural network models usually can not lead to satisfactory accuracy. Selective neural network ensemble is an effective way to enhance the generalization accuracy o...Chemical processes are complex, for which traditional neural network models usually can not lead to satisfactory accuracy. Selective neural network ensemble is an effective way to enhance the generalization accuracy of networks, but there are some problems, e.g., lacking of unified definition of diversity among component neural networks and difficult to improve the accuracy by selecting if the diversities of available networks are small. In this study, the output errors of networks are vectorized, the diversity of networks is defined based on the error vectors, and the size of ensemble is analyzed. Then an error vectorization based selective neural network ensemble (EVSNE) is proposed, in which the error vector of each network can offset that of the other networks by training the component networks orderly. Thus the component networks have large diversity. Experiments and comparisons over standard data sets and actual chemical process data set for production of high-density polyethylene demonstrate that EVSNE performs better in generalization ability.展开更多
Multitasking and multioutput neural networks models jointly learn related classification tasks from a shared structure.Hard parameters sharing is a multitasking approach that shares hidden layers between multiple task...Multitasking and multioutput neural networks models jointly learn related classification tasks from a shared structure.Hard parameters sharing is a multitasking approach that shares hidden layers between multiple taskspecific outputs.The output layers’weights are essential in transforming aggregated neurons outputs into tasks labels.This paper redirects the multioutput network research to prove that the ensemble of output layers prediction can improve network performance in classifying multi-label classification tasks.The network’s output layers initialized with different weights simulate multiple semi-independent classifiers that can make non-identical label sets predictions for the same instance.The ensemble of a multi-output neural network that learns to classify the same multi-label classification task per output layer can outperform an individual output layer neural network.We propose an ensemble strategy of output layers components in the multi-output neural network for multi-label classification(ENSOCOM).The baseline and proposed models are selected based on the size of the hidden layer and the number of output layers to evaluate the proposed method comprehensively.The ENSOCOM method improved the performance of the neural networks on five different multi-label datasets based on several evaluation metrics.The methods presented in this work can substitute the standard labels representation and predictions generation of any neural network.展开更多
基金supported by the National Natural Science Foundation of China (Project No.42375192)the China Meteorological Administration Climate Change Special Program (CMA-CCSP+1 种基金Project No.QBZ202315)support by the Vector Stiftung through the Young Investigator Group"Artificial Intelligence for Probabilistic Weather Forecasting."
文摘Despite the maturity of ensemble numerical weather prediction(NWP),the resulting forecasts are still,more often than not,under-dispersed.As such,forecast calibration tools have become popular.Among those tools,quantile regression(QR)is highly competitive in terms of both flexibility and predictive performance.Nevertheless,a long-standing problem of QR is quantile crossing,which greatly limits the interpretability of QR-calibrated forecasts.On this point,this study proposes a non-crossing quantile regression neural network(NCQRNN),for calibrating ensemble NWP forecasts into a set of reliable quantile forecasts without crossing.The overarching design principle of NCQRNN is to add on top of the conventional QRNN structure another hidden layer,which imposes a non-decreasing mapping between the combined output from nodes of the last hidden layer to the nodes of the output layer,through a triangular weight matrix with positive entries.The empirical part of the work considers a solar irradiance case study,in which four years of ensemble irradiance forecasts at seven locations,issued by the European Centre for Medium-Range Weather Forecasts,are calibrated via NCQRNN,as well as via an eclectic mix of benchmarking models,ranging from the naïve climatology to the state-of-the-art deep-learning and other non-crossing models.Formal and stringent forecast verification suggests that the forecasts post-processed via NCQRNN attain the maximum sharpness subject to calibration,amongst all competitors.Furthermore,the proposed conception to resolve quantile crossing is remarkably simple yet general,and thus has broad applicability as it can be integrated with many shallow-and deep-learning-based neural networks.
基金supported by the National Natural Science Foundation of China Youth Fund(12105234)。
文摘The distribution of the nuclear ground-state spin in a two-body random ensemble(TBRE)was studied using a general classification neural network(NN)model with two-body interaction matrix elements as input features and the corresponding ground-state spins as labels or output predictions.The quantum many-body system problem exceeds the capability of our optimized NNs in terms of accurately predicting the ground-state spin of each sample within the TBRE.However,our NN model effectively captured the statistical properties of the ground-state spin because it learned the empirical regularity of the ground-state spin distribution in TBRE,as discovered by physicists.
文摘Smart Industrial environments use the Industrial Internet of Things(IIoT)for their routine operations and transform their industrial operations with intelligent and driven approaches.However,IIoT devices are vulnerable to cyber threats and exploits due to their connectivity with the internet.Traditional signature-based IDS are effective in detecting known attacks,but they are unable to detect unknown emerging attacks.Therefore,there is the need for an IDS which can learn from data and detect new threats.Ensemble Machine Learning(ML)and individual Deep Learning(DL)based IDS have been developed,and these individual models achieved low accuracy;however,their performance can be improved with the ensemble stacking technique.In this paper,we have proposed a Deep Stacked Neural Network(DSNN)based IDS,which consists of two stacked Convolutional Neural Network(CNN)models as base learners and Extreme Gradient Boosting(XGB)as the meta learner.The proposed DSNN model was trained and evaluated with the next-generation dataset,TON_IoT.Several pre-processing techniques were applied to prepare a dataset for the model,including ensemble feature selection and the SMOTE technique.Accuracy,precision,recall,F1-score,and false positive rates were used to evaluate the performance of the proposed ensemble model.Our experimental results showed that the accuracy for binary classification is 99.61%,which is better than in the baseline individual DL and ML models.In addition,the model proposed for IDS has been compared with similar models.The proposed DSNN achieved better performance metrics than the other models.The proposed DSNN model will be used to develop enhanced IDS for threat mitigation in smart industrial environments.
基金The National Natural Science Foundation of China under contract No.42001401the China Postdoctoral Science Foundation under contract No.2020M671431+1 种基金the Fundamental Research Funds for the Central Universities under contract No.0209-14380096the Guangxi Innovative Development Grand Grant under contract No.2018AA13005.
文摘The back propagation(BP)neural network method is widely used in bathymetry based on multispectral satellite imagery.However,the classical BP neural network method faces a potential problem because it easily falls into a local minimum,leading to model training failure.This study confirmed that the local minimum problem of the BP neural network method exists in the bathymetry field and cannot be ignored.Furthermore,to solve the local minimum problem of the BP neural network method,a bathymetry method based on a BP neural network and ensemble learning(BPEL)is proposed.First,the remote sensing imagery and training sample were used as input datasets,and the BP method was used as the base learner to produce multiple water depth inversion results.Then,a new ensemble strategy,namely the minimum outlying degree method,was proposed and used to integrate the water depth inversion results.Finally,an ensemble bathymetric map was acquired.Anda Reef,northeastern Jiuzhang Atoll,and Pingtan coastal zone were selected as test cases to validate the proposed method.Compared with the BP neural network method,the root-mean-square error and the average relative error of the BPEL method can reduce by 0.65–2.84 m and 16%–46%in the three test cases at most.The results showed that the proposed BPEL method could solve the local minimum problem of the BP neural network method and obtain highly robust and accurate bathymetric maps.
文摘The existing strategy for evaluating the damage condition of structures mostly focuses on feedback supplied by traditional visualmethods,which may result in an unreliable damage characterization due to inspector subjectivity or insufficient level of expertise.As a result,a robust,reliable,and repeatable method of damage identification is required.Ensemble learning algorithms for identifying structural damage are evaluated in this article,which use deep convolutional neural networks,including simple averaging,integrated stacking,separate stacking,and hybridweighted averaging ensemble and differential evolution(WAE-DE)ensemblemodels.Damage identification is carried out on three types of damage.The proposed algorithms are used to analyze the damage of 4585 structural images.The effectiveness of the ensemble learning techniques is evaluated using the confusion matrix.For the testing dataset,the confusion matrix achieved an accuracy of 94 percent and a minimum recall of 92 percent for the best model(WAE-DE)in distinguishing damage types as flexural,shear,combined,or undamaged.
基金Project(41061041)supported by the National Natural Science Foundation of ChinaProject(2010gzs0084)supported by the Natural Science Foundation of Jiangxi Province,China
文摘Considering the defects of low accuracy and slow speed existing in traditional flood loss assessment, firstly, the technical route of flood loss assessment was presented based on the neural network ensemble. Secondly, through the study of certain country of Poyang Lake district, the flood loss assessment indicators of the test area were analyzed and extracted by utilizing analytic hierarchy process (AHP), and the weights of the impact factors were assigned. Subsequently, the approaches to generate individuals and conclusions of neural network ensemble model were also investigated. In the platform of C# language and neural network library under AForge.NET open source, a flood loss assessment program which could rapidly build neural network ensemble models was developed. Finally, the proposed method was tested and verified. The comparison results between the assessment results of the proposed method and the actual statistical flood loss proved the feasibility of this method, thus a new approach for flood loss assessment was provided.
基金supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MEST)No.2015R1A3A2031159,2016R1A5A1008055.
文摘Existing web-based security applications have failed in many situations due to the great intelligence of attackers.Among web applications,Cross-Site Scripting(XSS)is one of the dangerous assaults experienced while modifying an organization's or user's information.To avoid these security challenges,this article proposes a novel,all-encompassing combination of machine learning(NB,SVM,k-NN)and deep learning(RNN,CNN,LSTM)frameworks for detecting and defending against XSS attacks with high accuracy and efficiency.Based on the representation,a novel idea for merging stacking ensemble with web applications,termed“hybrid stacking”,is proposed.In order to implement the aforementioned methods,four distinct datasets,each of which contains both safe and unsafe content,are considered.The hybrid detection method can adaptively identify the attacks from the URL,and the defense mechanism inherits the advantages of URL encoding with dictionary-based mapping to improve prediction accuracy,accelerate the training process,and effectively remove the unsafe JScript/JavaScript keywords from the URL.The simulation results show that the proposed hybrid model is more efficient than the existing detection methods.It produces more than 99.5%accurate XSS attack classification results(accuracy,precision,recall,f1_score,and Receiver Operating Characteristic(ROC))and is highly resistant to XSS attacks.In order to ensure the security of the server's information,the proposed hybrid approach is demonstrated in a real-time environment.
基金funded by the National Key R&D Program of China(2021YFD1400200)the Taishan Scholar Constructive Engineering Foundation of Shandong,China(tstp20221135)。
文摘Potato cyst nematodes(PCNs)are a significant threat to potato production,having caused substantial damage in many countries.Predicting the future distribution of PCN species is crucial to implementing effective biosecurity strategies,especially given the impact of climate change on pest species invasion and distribution.Machine learning(ML),specifically ensemble models,has emerged as a powerful tool in predicting species distributions due to its ability to learn and make predictions based on complex data sets.Thus,this research utilised advanced machine learning techniques to predict the distribution of PCN species under climate change conditions,providing the initial element for invasion risk assessment.We first used Global Climate Models to generate homogeneous climate predictors to mitigate the variation among predictors.Then,five machine learning models were employed to build two groups of ensembles,single-algorithm ensembles(ESA)and multi-algorithm ensembles(EMA),and compared their performances.In this research,the EMA did not always perform better than the ESA,and the ESA of Artificial Neural Network gave the highest performance while being cost-effective.Prediction results indicated that the distribution range of PCNs would shift northward with a decrease in tropical zones and an increase in northern latitudes.However,the total area of suitable regions will not change significantly,occupying 16-20%of the total land surface(18%under current conditions).This research alerts policymakers and practitioners to the risk of PCNs’incursion into new regions.Additionally,this ML process offers the capability to track changes in the distribution of various species and provides scientifically grounded evidence for formulating long-term biosecurity plans for their control.
基金supported by the National Key Research and Development Program of China under Grant 2019YFC1511401the National Natural Science Foundation of China under Grant 62173038 and 61103157+1 种基金Science Foundation for Young Scholars of Tobacco Research Institute of Chinese Academy of Agricultural Sciences under Grant 2021B05Key Scientific and Tech-nological Research and Development Project of China National Tobacco Corporation under Grant 110202102007.
文摘It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental information,as well as have high accuracy and quick inference speed.Network ensemble architecture is a good choice to improve network performance.However,it is unsuitable for real-time applications on the autonomous system.To tackle this problem,a new neural network ensemble named partial-shared ensemble network(PSENet)is presented.PSENet changes network ensemble architecture from parallel architecture to scatter architecture and merges multiple component networks together to accelerate the inference speed.To make component networks independent of each other,a training method is designed to train the network ensemble architecture.Experiments on Camvid and CIFAR-10 reveal that PSENet achieves quick inference speed while maintaining the ability of ensemble learning.In the real world,PSENet is deployed on the unmanned system and deals with vision tasks such as semantic segmentation and environmental prediction in different fields.
基金Project supported in part by the National Natural Science Foundation of China (No. 40201021) Zhejiang Provincial Natural Science Foundation of China (No. 402016).
文摘High quality, agricultural nutrient distribution maps are necessary for precision management, but depend on initial soil sample analyses and interpolation techniques. To examine the methodologies for and explore the capability of interpolating soil properties based on neural network ensemble residual kriging, a silage field at Hayes, Northern Ireland, UK, was selected for this study with all samples being split into independent training and validation data sets. The training data set, comprised of five soil properties: soil pH, soil available P, soil available K, soil available Mg and soil available S,was modeled for spatial variability using 1) neural network ensemble residual kriging, 2) neural network ensemble and 3) kriging with their accuracies being estimated by means of the validation data sets. Ordinary kriging of the residuals provided accurate local estimates, while final estimates were produced as a sum of the artificial neural network (ANN)ensemble estimates and the ordinary kriging estimates of the residuals. Compared to kriging and neural network ensemble,the neural network ensemble residual kriging achieved better or similar accuracy for predicting and estimating contour maps. Thus, the results demonstrated that ANN ensemble residual kriging was an efficient alternative to the conventional geo-statistical models that were usually used for interpolation of a data set in the soil science area.
基金the National Natural Science Foundation of China (No.40671145)the Natural Science Foundation of Guangdong Province (Nos.04300504 and 05006623)and the Science and Technology Plan Foundation of Guangdong Province (Nos.2005B20701008,2005B10101028,and 2004B20701006).
文摘Land evaluation factors often contain continuous-, discrete- and nominal-valued attributes. In traditional land evaluation, these different attributes are usually graded into categorical indexes by land resource experts, and the evaluation results rely heavily on experts' experiences. In order to overcome the shortcoming, we presented a fuzzy neural network ensemble method that did not require grading the evaluation factors into categorical indexes and could evaluate land resources by using the three kinds of attribute values directly. A fuzzy back propagation neural network (BPNN), a fuzzy radial basis function neural network (RBFNN), a fuzzy BPNN ensemble, and a fuzzy RBFNN ensemble were used to evaluate the land resources in Guangdong Province. The evaluation results by using the fuzzy BPNN ensemble and the fuzzy RBFNN ensemble were much better than those by using the single fuzzy BPNN and the single fuzzy RBFNN, and the error rate of the single fuzzy RBFNN or fuzzy RBFNN ensemble was lower than that of the single fuzzy BPNN or fuzzy BPNN ensemble, respectively. By using the fuzzy neural network ensembles, the validity of land resource evaluation was improved and reliance on land evaluators' experiences was considerably reduced.
文摘An effective ensemble should consist of a set of networks that are both accurate and diverse. We propose a novel clustering-based selective algorithm for constructing neural network ensemble, where clustering technology is used to classify trained networks according to similarity and optimally select the most accurate individual network from each cluster to make up the ensemble. Empirical studies on regression of four typical datasets showed that this approach yields significantly smaller en- semble achieving better performance than other traditional ones such as Bagging and Boosting. The bias variance decomposition of the predictive error shows that the success of the proposed approach may lie in its properly tuning the bias/variance trade-off to reduce the prediction error (the sum of bias2 and variance).
基金Project supported by National Natural Science Foundation of China (Grant No. 20503015)
文摘A clustering algorithm based selective neural networks ensemble (CLUSEN) is proposed to predict the degree of malignancy in brain glioma. Since the degree prediction of malignancy is critical before brain surgery, many learning methods are used like rule induction algorithm, single neural networks, support vector machines, etc. Ensemble learning methods can improve the generalization of single learning machine, and are becoming popular in the machine learning and medical data processing communities. The procedure of CLUSEN can efficiently remove redundancy learning individuals and help improve the diversity of ensemble methods. CLUSEN is used to predict the degree of malignancy in brain glioma. Experimental results on a set of brain glioma data show that, compared to support vector machines, rule induction and single neural networks, the classification accuracy of CLUSEN is higher.
基金the National Natural Science Foundationof China(Nos.11672098,11502063)the Natural Science Foundation of Anhui Province(No.1608085QA07).
文摘Residual useful life(RUL)prediction is a key issue for improving efficiency of aircraft engines and reducing their maintenance cost.Owing to various failure mechanism and operating environment,the application of classical models in RUL prediction of aircraft engines is fairly difficult.In this study,a novel RUL prognostics method based on using ensemble recurrent neural network to process massive sensor data is proposed.First of all,sensor data obtained from the aircraft engines are preprocessed to eliminate singular values,reduce random fluctuation and preserve degradation trend of the raw sensor data.Secondly,three kinds of recurrent neural networks(RNN),including ordinary RNN,long shortterm memory(LSTM),and gated recurrent unit(GRU),are individually constructed.Thirdly,ensemble learning mechanism is designed to merge the above RNNs for producing a more accurate RUL prediction.The effectiveness of the proposed method is validated using two characteristically different turbofan engine datasets.Experimental results show a competitive performance of the proposed method in comparison with typical methods reported in literatures.
文摘Irretrievable loss of vision is the predominant result of Glaucoma in the retina.Recently,multiple approaches have paid attention to the automatic detection of glaucoma on fundus images.Due to the interlace of blood vessels and the herculean task involved in glaucoma detection,the exactly affected site of the optic disc of whether small or big size cup,is deemed challenging.Spatially Based Ellipse Fitting Curve Model(SBEFCM)classification is suggested based on the Ensemble for a reliable diagnosis of Glaucomain theOptic Cup(OC)and Optic Disc(OD)boundary correspondingly.This research deploys the Ensemble Convolutional Neural Network(CNN)classification for classifying Glaucoma or Diabetes Retinopathy(DR).The detection of the boundary between the OC and the OD is performed by the SBEFCM,which is the latest weighted ellipse fitting model.The SBEFCM that enhances and widens the multi-ellipse fitting technique is proposed here.There is a preprocessing of input fundus image besides segmentation of blood vessels to avoid interlacing surrounding tissues and blood vessels.The ascertaining of OCandODboundary,which characterizedmany output factors for glaucoma detection,has been developed by EnsembleCNNclassification,which includes detecting sensitivity,specificity,precision,andArea Under the receiver operating characteristic Curve(AUC)values accurately by an innovative SBEFCM.In terms of contrast,the proposed Ensemble CNNsignificantly outperformed the current methods.
基金supported by National Natural Science Foundation of China(Grant No.50575179)National Hi-tech Research and Development Program of China(863 Program,Grant No.2006AA04Z420)
文摘There has been many methods in constructing neural network (NN) ensembles, where the method of simultaneous training has succeed in generalization performance and efficiency. But just like regular methods of constructing NN ensembles, it follows the two steps, first training component networks, and then combining them. As the two steps being independent, an assumption is used to facilitate interactions among NNs during the training stage. This paper presents a compact ensemble method which integrates the two steps of ensemble construction into one step by attempting to train individual NNs in an ensemble and weigh the individual members adaptively according to their individual performance in the same learning process. This provides an opportunity for the individual NNs to interact with each other based on their real contributions to the ensemble. The classification performance of NN compact ensemble (NNCE) was validated through some benchmark problems in machine learning, including Australian credit card assessment, pima Indians diabetes, heart disease, breast cancer and glass. Compared with other ensembles, the classification error rate of NNCE can be decreased by 0.45% to 68%. In addition, the NNCE was applied to fault diagnosis for rolling element bearing. The 11 time-domain statistical features are extracted as the properties of data, and the NNCE is employed to classify the data. With the results of several experiments, the compact ensemble method is shown to give good generalization performance. The compact ensemble method can recognize the different fault types and various fault degrees of the same fault type.
基金Supported by the National Natural Science Foundation of China(60133010)
文摘This paper introduces supervised learning model, and surveys related research work. The paper is organised as follows. A supervised learning model is firstly described. The bias-variance trade-off is then discussed for the supervised learning model. Based on the bias-variance trade-off, both the single neural network approaches and the neural network ensemble approaches are overviewed, and problems with the existing approaches are indicated. Finally, the paper concludes with specifying potential future research directions.
基金Supported by the National Natural Science Foundation of China (61074153, 61104131)the Fundamental Research Fundsfor Central Universities of China (ZY1111, JD1104)
文摘Chemical processes are complex, for which traditional neural network models usually can not lead to satisfactory accuracy. Selective neural network ensemble is an effective way to enhance the generalization accuracy of networks, but there are some problems, e.g., lacking of unified definition of diversity among component neural networks and difficult to improve the accuracy by selecting if the diversities of available networks are small. In this study, the output errors of networks are vectorized, the diversity of networks is defined based on the error vectors, and the size of ensemble is analyzed. Then an error vectorization based selective neural network ensemble (EVSNE) is proposed, in which the error vector of each network can offset that of the other networks by training the component networks orderly. Thus the component networks have large diversity. Experiments and comparisons over standard data sets and actual chemical process data set for production of high-density polyethylene demonstrate that EVSNE performs better in generalization ability.
基金The authors would like to thank the Deanship of Scientific Research at UmmAl-Qura University for supporting this work by Grant Code: (22UQU4340018DSR02).
文摘Multitasking and multioutput neural networks models jointly learn related classification tasks from a shared structure.Hard parameters sharing is a multitasking approach that shares hidden layers between multiple taskspecific outputs.The output layers’weights are essential in transforming aggregated neurons outputs into tasks labels.This paper redirects the multioutput network research to prove that the ensemble of output layers prediction can improve network performance in classifying multi-label classification tasks.The network’s output layers initialized with different weights simulate multiple semi-independent classifiers that can make non-identical label sets predictions for the same instance.The ensemble of a multi-output neural network that learns to classify the same multi-label classification task per output layer can outperform an individual output layer neural network.We propose an ensemble strategy of output layers components in the multi-output neural network for multi-label classification(ENSOCOM).The baseline and proposed models are selected based on the size of the hidden layer and the number of output layers to evaluate the proposed method comprehensively.The ENSOCOM method improved the performance of the neural networks on five different multi-label datasets based on several evaluation metrics.The methods presented in this work can substitute the standard labels representation and predictions generation of any neural network.