Non-crossing Quantile Regression Neural Network as a Calibration Tool for Ensemble Weather Forecasts

Despite the maturity of ensemble numerical weather prediction(NWP),the resulting forecasts are still,more often than not,under-dispersed.As such,forecast calibration tools have become popular.Among those tools,quantile regression(QR)is highly competitive in terms of both flexibility and predictive performance.Nevertheless,a long-standing problem of QR is quantile crossing,which greatly limits the interpretability of QR-calibrated forecasts.On this point,this study proposes a non-crossing quantile regression neural network(NCQRNN),for calibrating ensemble NWP forecasts into a set of reliable quantile forecasts without crossing.The overarching design principle of NCQRNN is to add on top of the conventional QRNN structure another hidden layer,which imposes a non-decreasing mapping between the combined output from nodes of the last hidden layer to the nodes of the output layer,through a triangular weight matrix with positive entries.The empirical part of the work considers a solar irradiance case study,in which four years of ensemble irradiance forecasts at seven locations,issued by the European Centre for Medium-Range Weather Forecasts,are calibrated via NCQRNN,as well as via an eclectic mix of benchmarking models,ranging from the naïve climatology to the state-of-the-art deep-learning and other non-crossing models.Formal and stringent forecast verification suggests that the forecasts post-processed via NCQRNN attain the maximum sharpness subject to calibration,amongst all competitors.Furthermore,the proposed conception to resolve quantile crossing is remarkably simple yet general,and thus has broad applicability as it can be integrated with many shallow-and deep-learning-based neural networks.

Neural network study of the nuclear ground-state spin distribution within a random interaction ensemble

The distribution of the nuclear ground-state spin in a two-body random ensemble(TBRE)was studied using a general classification neural network(NN)model with two-body interaction matrix elements as input features and the corresponding ground-state spins as labels or output predictions.The quantum many-body system problem exceeds the capability of our optimized NNs in terms of accurately predicting the ground-state spin of each sample within the TBRE.However,our NN model effectively captured the statistical properties of the ground-state spin because it learned the empirical regularity of the ground-state spin distribution in TBRE,as discovered by physicists.

Shallow water bathymetry based on a back propagation neural network and ensemble learning using multispectral satellite imagery

The back propagation(BP)neural network method is widely used in bathymetry based on multispectral satellite imagery.However,the classical BP neural network method faces a potential problem because it easily falls into a local minimum,leading to model training failure.This study confirmed that the local minimum problem of the BP neural network method exists in the bathymetry field and cannot be ignored.Furthermore,to solve the local minimum problem of the BP neural network method,a bathymetry method based on a BP neural network and ensemble learning(BPEL)is proposed.First,the remote sensing imagery and training sample were used as input datasets,and the BP method was used as the base learner to produce multiple water depth inversion results.Then,a new ensemble strategy,namely the minimum outlying degree method,was proposed and used to integrate the water depth inversion results.Finally,an ensemble bathymetric map was acquired.Anda Reef,northeastern Jiuzhang Atoll,and Pingtan coastal zone were selected as test cases to validate the proposed method.Compared with the BP neural network method,the root-mean-square error and the average relative error of the BPEL method can reduce by 0.65–2.84 m and 16%–46%in the three test cases at most.The results showed that the proposed BPEL method could solve the local minimum problem of the BP neural network method and obtain highly robust and accurate bathymetric maps.

Structural Damage Identification Using Ensemble Deep Convolutional Neural Network Models

The existing strategy for evaluating the damage condition of structures mostly focuses on feedback supplied by traditional visualmethods,which may result in an unreliable damage characterization due to inspector subjectivity or insufficient level of expertise.As a result,a robust,reliable,and repeatable method of damage identification is required.Ensemble learning algorithms for identifying structural damage are evaluated in this article,which use deep convolutional neural networks,including simple averaging,integrated stacking,separate stacking,and hybridweighted averaging ensemble and differential evolution(WAE-DE)ensemblemodels.Damage identification is carried out on three types of damage.The proposed algorithms are used to analyze the damage of 4585 structural images.The effectiveness of the ensemble learning techniques is evaluated using the confusion matrix.For the testing dataset,the confusion matrix achieved an accuracy of 94 percent and a minimum recall of 92 percent for the best model(WAE-DE)in distinguishing damage types as flexural,shear,combined,or undamaged.

基于改进SMOTE算法和Ensemble模型的学习结果预测方法

为解决不同领域的数据分类和预测任务中单个机器学习算法适用性较差的问题,以及缓解数据集严重不平衡对预测性能的影响,提出了基于合成少数类过采样(SMOTE)和Ensemble集成模型的数据分类方法。传统SMOTE算法通过对少数类样本进行插值来生成新的合成样本,合成样本中存在噪声和样本间相似性较高的问题。为此,提出了改进的SMOTE算法,通过距离计算移除噪声样本和易混淆样本,得到高区分度的纯净合成样本。然后,利用Ensemble方法调整样本和分类器权重,并组成分类效果更好的强分类器。在公开在线学习数据集Kalboard360上的实验结果表明,使用极限随机树(ERT)分类器时,结合改进SMOTE和Ensemble模型后实现了97.9%的预测准确度,比单个ERT分类器提升了5.5%,证明所提改进SMOTE算法能够生成高质量的均衡化数据,且集成学习模型的性能显著优于单个机器学习算法。

Lightweight Network Ensemble Architecture for Environmental Perception on the Autonomous System

It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental information,as well as have high accuracy and quick inference speed.Network ensemble architecture is a good choice to improve network performance.However,it is unsuitable for real-time applications on the autonomous system.To tackle this problem,a new neural network ensemble named partial-shared ensemble network(PSENet)is presented.PSENet changes network ensemble architecture from parallel architecture to scatter architecture and merges multiple component networks together to accelerate the inference speed.To make component networks independent of each other,a training method is designed to train the network ensemble architecture.Experiments on Camvid and CIFAR-10 reveal that PSENet achieves quick inference speed while maintaining the ability of ensemble learning.In the real world,PSENet is deployed on the unmanned system and deals with vision tasks such as semantic segmentation and environmental prediction in different fields.

Neural Network Ensemble Residual Kriging Application for Spatial Variability of Soil Properties

High quality, agricultural nutrient distribution maps are necessary for precision management, but depend on initial soil sample analyses and interpolation techniques. To examine the methodologies for and explore the capability of interpolating soil properties based on neural network ensemble residual kriging, a silage field at Hayes, Northern Ireland, UK, was selected for this study with all samples being split into independent training and validation data sets. The training data set, comprised of five soil properties: soil pH, soil available P, soil available K, soil available Mg and soil available S,was modeled for spatial variability using 1) neural network ensemble residual kriging, 2) neural network ensemble and 3) kriging with their accuracies being estimated by means of the validation data sets. Ordinary kriging of the residuals provided accurate local estimates, while final estimates were produced as a sum of the artificial neural network (ANN)ensemble estimates and the ordinary kriging estimates of the residuals. Compared to kriging and neural network ensemble,the neural network ensemble residual kriging achieved better or similar accuracy for predicting and estimating contour maps. Thus, the results demonstrated that ANN ensemble residual kriging was an efficient alternative to the conventional geo-statistical models that were usually used for interpolation of a data set in the soil science area.

Improving Land Resource Evaluation Using Fuzzy Neural Network Ensembles

陆地评估因素经常包含连续 -- ，分离值、名字值的属性。在传统的陆地评估，这些不同属性通常由陆路被分级进范畴的索引资源专家，和评估结果重重地依靠专家的经验。以便克服缺点，我们介绍了没要求 grading 的一个模糊神经网络整体方法评估因素进范畴的索引并且能由直接使用三种属性价值评估土地资源。模糊的背繁殖神经网络(BPNN ) ，一个模糊光线的基础函数神经网络(RBFNN ) ，一个模糊 BPNN 整体，和一个模糊 RBFNN 整体被用来在广东评估土地资源省。由使用模糊 BPNN 整体和模糊 RBFNN 整体的评估结果是比由使用单个模糊 BPNN 和单个模糊 RBFNN 的那些好一些的，并且单个模糊 RBFNN 或模糊 RBFNN 整体的错误率分别地比单个模糊 BPNN 或模糊 BPNN 整体的低。由使用模糊神经网络整体，土地资源评估的有效性被改进，陆地计算程序的经验上的信赖更加被减少。

Prediction of malignancy selective neural networks degree in brain glioma using ensemble

A clustering algorithm based selective neural networks ensemble （CLUSEN） is proposed to predict the degree of malignancy in brain glioma. Since the degree prediction of malignancy is critical before brain surgery, many learning methods are used like rule induction algorithm, single neural networks, support vector machines, etc. Ensemble learning methods can improve the generalization of single learning machine, and are becoming popular in the machine learning and medical data processing communities. The procedure of CLUSEN can efficiently remove redundancy learning individuals and help improve the diversity of ensemble methods. CLUSEN is used to predict the degree of malignancy in brain glioma. Experimental results on a set of brain glioma data show that, compared to support vector machines, rule induction and single neural networks, the classification accuracy of CLUSEN is higher.

Clustering-based selective neural network ensemble

An effective ensemble should consist of a set of networks that are both accurate and diverse. We propose a novel clustering-based selective algorithm for constructing neural network ensemble, where clustering technology is used to classify trained networks according to similarity and optimally select the most accurate individual network from each cluster to make up the ensemble. Empirical studies on regression of four typical datasets showed that this approach yields significantly smaller en- semble achieving better performance than other traditional ones such as Bagging and Boosting. The bias variance decomposition of the predictive error shows that the success of the proposed approach may lie in its properly tuning the bias/variance trade-off to reduce the prediction error (the sum of bias2 and variance).

Ensemble Recurrent Neural Network-Based Residual Useful Life Prognostics of Aircraft Engines

Residual useful life(RUL)prediction is a key issue for improving efficiency of aircraft engines and reducing their maintenance cost.Owing to various failure mechanism and operating environment,the application of classical models in RUL prediction of aircraft engines is fairly difficult.In this study,a novel RUL prognostics method based on using ensemble recurrent neural network to process massive sensor data is proposed.First of all,sensor data obtained from the aircraft engines are preprocessed to eliminate singular values,reduce random fluctuation and preserve degradation trend of the raw sensor data.Secondly,three kinds of recurrent neural networks(RNN),including ordinary RNN,long shortterm memory(LSTM),and gated recurrent unit(GRU),are individually constructed.Thirdly,ensemble learning mechanism is designed to merge the above RNNs for producing a more accurate RUL prediction.The effectiveness of the proposed method is validated using two characteristically different turbofan engine datasets.Experimental results show a competitive performance of the proposed method in comparison with typical methods reported in literatures.

Enhanced Detection of Glaucoma on Ensemble Convolutional Neural Network for Clinical Informatics

Irretrievable loss of vision is the predominant result of Glaucoma in the retina.Recently,multiple approaches have paid attention to the automatic detection of glaucoma on fundus images.Due to the interlace of blood vessels and the herculean task involved in glaucoma detection,the exactly affected site of the optic disc of whether small or big size cup,is deemed challenging.Spatially Based Ellipse Fitting Curve Model(SBEFCM)classification is suggested based on the Ensemble for a reliable diagnosis of Glaucomain theOptic Cup(OC)and Optic Disc(OD)boundary correspondingly.This research deploys the Ensemble Convolutional Neural Network(CNN)classification for classifying Glaucoma or Diabetes Retinopathy(DR).The detection of the boundary between the OC and the OD is performed by the SBEFCM,which is the latest weighted ellipse fitting model.The SBEFCM that enhances and widens the multi-ellipse fitting technique is proposed here.There is a preprocessing of input fundus image besides segmentation of blood vessels to avoid interlacing surrounding tissues and blood vessels.The ascertaining of OCandODboundary,which characterizedmany output factors for glaucoma detection,has been developed by EnsembleCNNclassification,which includes detecting sensitivity,specificity,precision,andArea Under the receiver operating characteristic Curve(AUC)values accurately by an innovative SBEFCM.In terms of contrast,the proposed Ensemble CNNsignificantly outperformed the current methods.

Neural Network Compact Ensemble and Its Applications

There has been many methods in constructing neural network （NN） ensembles, where the method of simultaneous training has succeed in generalization performance and efficiency. But just like regular methods of constructing NN ensembles, it follows the two steps, first training component networks, and then combining them. As the two steps being independent, an assumption is used to facilitate interactions among NNs during the training stage. This paper presents a compact ensemble method which integrates the two steps of ensemble construction into one step by attempting to train individual NNs in an ensemble and weigh the individual members adaptively according to their individual performance in the same learning process. This provides an opportunity for the individual NNs to interact with each other based on their real contributions to the ensemble. The classification performance of NN compact ensemble （NNCE） was validated through some benchmark problems in machine learning, including Australian credit card assessment, pima Indians diabetes, heart disease, breast cancer and glass. Compared with other ensembles, the classification error rate of NNCE can be decreased by 0.45% to 68%. In addition, the NNCE was applied to fault diagnosis for rolling element bearing. The 11 time-domain statistical features are extracted as the properties of data, and the NNCE is employed to classify the data. With the results of several experiments, the compact ensemble method is shown to give good generalization performance. The compact ensemble method can recognize the different fault types and various fault degrees of the same fault type.

From Designing A Single Neural Network to Designing Neural Network Ensembles

This paper introduces supervised learning model, and surveys related research work. The paper is organised as follows. A supervised learning model is firstly described. The bias-variance trade-off is then discussed for the supervised learning model. Based on the bias-variance trade-off, both the single neural network approaches and the neural network ensemble approaches are overviewed, and problems with the existing approaches are indicated. Finally, the paper concludes with specifying potential future research directions.

ENSOCOM: Ensemble of Multi-Output Neural Network’s Components for Multi-Label Classification

Multitasking and multioutput neural networks models jointly learn related classification tasks from a shared structure.Hard parameters sharing is a multitasking approach that shares hidden layers between multiple taskspecific outputs.The output layers’weights are essential in transforming aggregated neurons outputs into tasks labels.This paper redirects the multioutput network research to prove that the ensemble of output layers prediction can improve network performance in classifying multi-label classification tasks.The network’s output layers initialized with different weights simulate multiple semi-independent classifiers that can make non-identical label sets predictions for the same instance.The ensemble of a multi-output neural network that learns to classify the same multi-label classification task per output layer can outperform an individual output layer neural network.We propose an ensemble strategy of output layers components in the multi-output neural network for multi-label classification(ENSOCOM).The baseline and proposed models are selected based on the size of the hidden layer and the number of output layers to evaluate the proposed method comprehensively.The ENSOCOM method improved the performance of the neural networks on five different multi-label datasets based on several evaluation metrics.The methods presented in this work can substitute the standard labels representation and predictions generation of any neural network.

Coronavirus Detection Using Two Step-AS Clustering and Ensemble Neural Network Model

This study presents a model of computer-aided intelligence capable of automatically detecting positive COVID-19 instances for use in regular medical applications.The proposed model is based on an Ensemble boosting Neural Network architecture and can automatically detect discriminatory features on chestX-ray images through Two Step-As clustering algorithm with rich filter families,abstraction and weight-sharing properties.In contrast to the generally used transformational learning approach,the proposed model was trained before and after clustering.The compilation procedure divides the datasets samples and categories into numerous sub-samples and subcategories and then assigns new group labels to each new group,with each subject group displayed as a distinct category.The retrieved characteristics discriminant cases were used to feed the Multiple Neural Network method,which was then utilised to classify the instances.The Two Step-AS clustering method has been modified by pre-aggregating the dataset before applying Multiple Neural Network algorithm to detect COVID-19 cases from chest X-ray findings.Models forMultiple Neural Network and Two Step-As clustering algorithms were optimised by utilising Ensemble Bootstrap Aggregating algorithm to reduce the number of hyper parameters they include.The testswere carried out using theCOVID-19 public radiology database,and a cross-validationmethod ensured accuracy.The proposed classifier with an accuracy of 98.02%percent was found to provide the most efficient outcomes possible.The result is a lowcost,quick and reliable intelligence tool for detecting COVID-19 infection.

An Optimized Convolutional Neural Network Architecture Based on Evolutionary Ensemble Learning

Convolutional Neural Networks(CNNs)models succeed in vast domains.CNNs are available in a variety of topologies and sizes.The challenge in this area is to develop the optimal CNN architecture for a particular issue in order to achieve high results by using minimal computational resources to train the architecture.Our proposed framework to automated design is aimed at resolving this problem.The proposed framework is focused on a genetic algorithm that develops a population of CNN models in order to find the architecture that is the best fit.In comparison to the co-authored work,our proposed framework is concerned with creating lightweight architectures with a limited number of parameters while retaining a high degree of validity accuracy utilizing an ensemble learning technique.This architecture is intended to operate on low-resource machines,rendering it ideal for implementation in a number of environments.Four common benchmark image datasets are used to test the proposed framework,and it is compared to peer competitors’work utilizing a range of parameters,including accuracy,the number of model parameters used,the number of GPUs used,and the number of GPU days needed to complete the method.Our experimental findings demonstrated a significant advantage in terms of GPU days,accuracy,and the number of parameters in the discovered model.

Tissue specific prediction of N^(6)-methyladenine sites based on an ensemble of multi-input hybrid neural network

N^(6)-Methyladenine is a dynamic and reversible post translational modification,which plays an essential role in various biological processes.Because of the current inability to identify m6A-containing mRNAs,computational approaches have been developed to identify m6A sites in DNA sequences.Aiming to improve prediction performance,we introduced a novel ensemble computational approach based on three hybrid deep neural networks,including a convolutional neural network,a capsule network,and a bidirectional gated recurrent unit(BiGRU)with the self-attention mechanism,to identify m6A sites in four tissues of three species.Across a total of 11 datasets,we selected different feature subsets,after optimized from 4933 dimensional features,as input for the deep hybrid neural networks.In addition,to solve the deviation caused by the relatively small number of experimentally verified samples,we constructed an ensemble model through integrating five sub-classifiers based on different training datasets.When compared through 5-fold cross-validation and independent tests,our model showed its superiority to previous methods,im6A-TS-CNN and iRNA-m6A.

Ensemble Neural Network in Classifying Handwritten Arabic Numerals

A method has been proposed to classify handwritten Arabic numerals in its compressed form using partitioning approach, Leader algorithm and Neural network. Handwritten numerals are represented in a matrix form. Compressing the matrix representation by merging adjacent pair of rows using logical OR operation reduces its size in half. Considering each row as a partitioned portion, clusters are formed for same partition of same digit separately. Leaders of clusters of partitions are used to recognize the patterns by Divide and Conquer approach using proposed ensemble neural network. Experimental results show that the proposed method recognize the patterns accurately.

Bootstrapped Multi-Model Neural-Network Super-Ensembles for Wind Speed and Power Forecasting

The bootstrap resampling method is applied to an ensemble artificial neural network (ANN) approach (which combines machine learning with physical data obtained from a numerical weather prediction model) to provide a multi-ANN model super-ensemble for application to multi-step-ahead forecasting of wind speed and of the associated power generated from a wind turbine. A statistical combination of the individual forecasts from the various ANNs of the super-ensemble is used to construct the best deterministic forecast, as well as the prediction uncertainty interval associated with this forecast. The bootstrapped neural-network methodology is validated using measured wind speed and power data acquired from a wind turbine in an operational wind farm located in northern China.