Lightweight Network Ensemble Architecture for Environmental Perception on the Autonomous System

It is important for the autonomous system to understand environmental information.For the autonomous system,it is desirable to have a strong generalization ability to deal with different complex environmental information,as well as have high accuracy and quick inference speed.Network ensemble architecture is a good choice to improve network performance.However,it is unsuitable for real-time applications on the autonomous system.To tackle this problem,a new neural network ensemble named partial-shared ensemble network(PSENet)is presented.PSENet changes network ensemble architecture from parallel architecture to scatter architecture and merges multiple component networks together to accelerate the inference speed.To make component networks independent of each other,a training method is designed to train the network ensemble architecture.Experiments on Camvid and CIFAR-10 reveal that PSENet achieves quick inference speed while maintaining the ability of ensemble learning.In the real world,PSENet is deployed on the unmanned system and deals with vision tasks such as semantic segmentation and environmental prediction in different fields.

Rapid assessment of flood loss based on neural network ensemble

Considering the defects of low accuracy and slow speed existing in traditional flood loss assessment, firstly, the technical route of flood loss assessment was presented based on the neural network ensemble. Secondly, through the study of certain country of Poyang Lake district, the flood loss assessment indicators of the test area were analyzed and extracted by utilizing analytic hierarchy process (AHP), and the weights of the impact factors were assigned. Subsequently, the approaches to generate individuals and conclusions of neural network ensemble model were also investigated. In the platform of C# language and neural network library under AForge.NET open source, a flood loss assessment program which could rapidly build neural network ensemble models was developed. Finally, the proposed method was tested and verified. The comparison results between the assessment results of the proposed method and the actual statistical flood loss proved the feasibility of this method, thus a new approach for flood loss assessment was provided.

Improving Land Resource Evaluation Using Fuzzy Neural Network Ensembles

Land evaluation factors often contain continuous-, discrete- and nominal-valued attributes. In traditional land evaluation, these different attributes are usually graded into categorical indexes by land resource experts, and the evaluation results rely heavily on experts＇ experiences. In order to overcome the shortcoming, we presented a fuzzy neural network ensemble method that did not require grading the evaluation factors into categorical indexes and could evaluate land resources by using the three kinds of attribute values directly. A fuzzy back propagation neural network （BPNN）, a fuzzy radial basis function neural network （RBFNN）, a fuzzy BPNN ensemble, and a fuzzy RBFNN ensemble were used to evaluate the land resources in Guangdong Province. The evaluation results by using the fuzzy BPNN ensemble and the fuzzy RBFNN ensemble were much better than those by using the single fuzzy BPNN and the single fuzzy RBFNN, and the error rate of the single fuzzy RBFNN or fuzzy RBFNN ensemble was lower than that of the single fuzzy BPNN or fuzzy BPNN ensemble, respectively. By using the fuzzy neural network ensembles, the validity of land resource evaluation was improved and reliance on land evaluators＇ experiences was considerably reduced.

A New Selective Neural Network Ensemble Method Based on Error Vectorization and Its Application in High-density Polyethylene (HDPE) Cascade Reaction Process

Chemical processes are complex, for which traditional neural network models usually can not lead to satisfactory accuracy. Selective neural network ensemble is an effective way to enhance the generalization accuracy of networks, but there are some problems, e.g., lacking of unified definition of diversity among component neural networks and difficult to improve the accuracy by selecting if the diversities of available networks are small. In this study, the output errors of networks are vectorized, the diversity of networks is defined based on the error vectors, and the size of ensemble is analyzed. Then an error vectorization based selective neural network ensemble (EVSNE) is proposed, in which the error vector of each network can offset that of the other networks by training the component networks orderly. Thus the component networks have large diversity. Experiments and comparisons over standard data sets and actual chemical process data set for production of high-density polyethylene demonstrate that EVSNE performs better in generalization ability.

From Designing A Single Neural Network to Designing Neural Network Ensembles

This paper introduces supervised learning model, and surveys related research work. The paper is organised as follows. A supervised learning model is firstly described. The bias-variance trade-off is then discussed for the supervised learning model. Based on the bias-variance trade-off, both the single neural network approaches and the neural network ensemble approaches are overviewed, and problems with the existing approaches are indicated. Finally, the paper concludes with specifying potential future research directions.

Neural Network Ensemble Residual Kriging Application for Spatial Variability of Soil Properties

High quality, agricultural nutrient distribution maps are necessary for precision management, but depend on initial soil sample analyses and interpolation techniques. To examine the methodologies for and explore the capability of interpolating soil properties based on neural network ensemble residual kriging, a silage field at Hayes, Northern Ireland, UK, was selected for this study with all samples being split into independent training and validation data sets. The training data set, comprised of five soil properties: soil pH, soil available P, soil available K, soil available Mg and soil available S,was modeled for spatial variability using 1) neural network ensemble residual kriging, 2) neural network ensemble and 3) kriging with their accuracies being estimated by means of the validation data sets. Ordinary kriging of the residuals provided accurate local estimates, while final estimates were produced as a sum of the artificial neural network (ANN)ensemble estimates and the ordinary kriging estimates of the residuals. Compared to kriging and neural network ensemble,the neural network ensemble residual kriging achieved better or similar accuracy for predicting and estimating contour maps. Thus, the results demonstrated that ANN ensemble residual kriging was an efficient alternative to the conventional geo-statistical models that were usually used for interpolation of a data set in the soil science area.

Deep neural network ensembles for remote sensing land cover and land use classification

With the advancement of satellite technology,a considerable amount of very high-resolution imagery has become available to be used for the Land Cover and Land Use(LCLU)classification task aiming to categorize remotely sensed images based on their semantic content.Recently,Deep Neural Networks(DNNs)have been widely used for different applications in the field of remote sensing and they have profound impacts;however,improvement of the generalizability and robustness of the DNNs needs to be progressed further to achieve higher accuracy for a variety of sensing geometries and categories.We address this problem by deploying three different Deep Neural Network Ensemble(DNNE)methods and creating a comparative analysis for the LCLU classification task.DNNE enables improvement of the performance of DNNs by ensuring the diversity of the models that are combined.Thus,enhances the generalizability of the models and produces more robust and generalizable outcomes for LCLU classification tasks.The experimental results on NWPU-RESISC45 and AID datasets demonstrate that utilizing the aggregated information from multiple DNNs leads to an increase in classification performance,achieves state-of-the-art,and promotes researchers to make use of DNNE.

Anomaly Detection of Complex Networks Based on Intuitionistic Fuzzy Set Ensemble

Ensemble learning for anomaly detection of data structured into a complex network has been barely studied due to the inconsistent performance of complex network characteristics and the lack of inherent objective function. We propose the intuitionistic fuzzy set（IFS）-based anomaly detection, a new two-phase ensemble method for anomaly detection based on IFS, and apply it to the abnormal behavior detection problem in temporal complex networks.Firstly, it constructs the IFS of a single network characteristic, which quantifies the degree of membership,non-membership and hesitation of each network characteristic to the defined linguistic variables so that makes the unuseful or noise characteristics become part of the detection. To build an objective intuitionistic fuzzy relationship, we propose a Gaussian distribution-based membership function which gives a variable hesitation degree. Then, for the fuzzification of multiple network characteristics, the intuitionistic fuzzy weighted geometric operator is adopted to fuse multiple IFSs and to avoid the inconsistence of multiple characteristics. Finally, the score function and precision function are used to sort the fused IFS. Finally, we carry out extensive experiments on several complex network datasets for anomaly detection, and the results demonstrate the superiority of our method to state-of-the-art approaches, validating the effectiveness of our method.

Enhanced Detection of Glaucoma on Ensemble Convolutional Neural Network for Clinical Informatics

Irretrievable loss of vision is the predominant result of Glaucoma in the retina.Recently,multiple approaches have paid attention to the automatic detection of glaucoma on fundus images.Due to the interlace of blood vessels and the herculean task involved in glaucoma detection,the exactly affected site of the optic disc of whether small or big size cup,is deemed challenging.Spatially Based Ellipse Fitting Curve Model(SBEFCM)classification is suggested based on the Ensemble for a reliable diagnosis of Glaucomain theOptic Cup(OC)and Optic Disc(OD)boundary correspondingly.This research deploys the Ensemble Convolutional Neural Network(CNN)classification for classifying Glaucoma or Diabetes Retinopathy(DR).The detection of the boundary between the OC and the OD is performed by the SBEFCM,which is the latest weighted ellipse fitting model.The SBEFCM that enhances and widens the multi-ellipse fitting technique is proposed here.There is a preprocessing of input fundus image besides segmentation of blood vessels to avoid interlacing surrounding tissues and blood vessels.The ascertaining of OCandODboundary,which characterizedmany output factors for glaucoma detection,has been developed by EnsembleCNNclassification,which includes detecting sensitivity,specificity,precision,andArea Under the receiver operating characteristic Curve(AUC)values accurately by an innovative SBEFCM.In terms of contrast,the proposed Ensemble CNNsignificantly outperformed the current methods.

Neural Network Compact Ensemble and Its Applications

There has been many methods in constructing neural network （NN） ensembles, where the method of simultaneous training has succeed in generalization performance and efficiency. But just like regular methods of constructing NN ensembles, it follows the two steps, first training component networks, and then combining them. As the two steps being independent, an assumption is used to facilitate interactions among NNs during the training stage. This paper presents a compact ensemble method which integrates the two steps of ensemble construction into one step by attempting to train individual NNs in an ensemble and weigh the individual members adaptively according to their individual performance in the same learning process. This provides an opportunity for the individual NNs to interact with each other based on their real contributions to the ensemble. The classification performance of NN compact ensemble （NNCE） was validated through some benchmark problems in machine learning, including Australian credit card assessment, pima Indians diabetes, heart disease, breast cancer and glass. Compared with other ensembles, the classification error rate of NNCE can be decreased by 0.45% to 68%. In addition, the NNCE was applied to fault diagnosis for rolling element bearing. The 11 time-domain statistical features are extracted as the properties of data, and the NNCE is employed to classify the data. With the results of several experiments, the compact ensemble method is shown to give good generalization performance. The compact ensemble method can recognize the different fault types and various fault degrees of the same fault type.

BIDIRECTIONAL ASSOCIATIVE MEMORY ENSEMBLE

The multiple classifier system （MCS）, composed of multiple diverse classifiers or feed-forward neural networks, can significantly improve the classification or generalization ability of a single classifier. Enlightened by the fundamental idea of MCS, the ensemble is introduced into the quick learning for bidirectional associative memory （QLBAM） to construct a BAM ensemble, for improving the storage capacity and the error-correction capability without destroying the simple structure of the component BAM. Simulations show that, with an appropriate ＂overproduce and choose＂ strategy or ＂thinning＂ algorithm, the proposed BAM ensemble significantly outperforms the single QLBAM in both storage capacity and noise-tolerance capability.

STUDY ON THE METEOROLOGICAL PREDICTION MODEL USING THE LEARNING ALGORITHM OF NEURAL ENSEMBLE BASED ON PSO ALGORITHMS

Because of the difficulty in deciding on the structure of BP neural network in operational meteorological application and the tendency for the network to transform to an issue of local solution, a hybrid Particle Swarm Optimization Algorithm based on Artificial Neural Network (PSO-BP) model is proposed for monthly mean rainfall of the whole area of Guangxi. It combines Particle Swarm Optimization (PSO) with BP, that is, the number of hidden nodes and connection weights are optimized by the implementation of PSO operation. The method produces a better network architecture and initial connection weights, trains the traditional backward propagation again by training samples. The ensemble strategy is carried out for the linear programming to calculate the best weights based on the "east sum of the error absolute value" as the optimal rule. The weighted coefficient of each ensemble individual is obtained. The results show that the method can effectively improve learning and generalization ability of the neural network.

A new ensemble feature selection and its application to pattern classification

Neural network ensemble based on rough sets reduct is proposed to decrease the computational complexity of conventional ensemble feature selection algorithm. First, a dynamic reduction technology combining genetic algorithm with resampling method is adopted to obtain reducts with good generalization ability. Second, Multiple BP neural networks based on different reducts are built as base classifiers. According to the idea of selective ensemble, the neural network ensemble with best generalization ability can be found by search strategies. Finally, classification based on neural network ensemble is implemented by combining the predictions of component networks with voting. The method has been verified in the experiment of remote sensing image and five UCI datasets classification. Compared with conventional ensemble feature selection algorithms, it costs less time and lower computing complexity, and the classification accuracy is satisfactory.

Adaptively Enhancing Facial Expression Crucial Regions via a Local Non-local Joint Network

Facial expression recognition(FER)is still challenging due to the small interclass discrepancy in facial expression data.In view of the significance of facial crucial regions for FER,many existing studies utilize the prior information from some annotated crucial points to improve the performance of FER.However,it is complicated and time-consuming to manually annotate facial crucial points,especially for vast wild expression images.Based on this,a local non-local joint network is proposed to adaptively enhance the facial crucial regions in feature learning of FER in this paper.In the proposed method,two parts are constructed based on facial local and non-local information,where an ensemble of multiple local networks is proposed to extract local features corresponding to multiple facial local regions and a non-local attention network is addressed to explore the significance of each local region.In particular,the attention weights obtained by the non-local network are fed into the local part to achieve interactive feedback between the facial global and local information.Interestingly,the non-local weights corresponding to local regions are gradually updated and higher weights are given to more crucial regions.Moreover,U-Net is employed to extract the integrated features of deep semantic information and low hierarchical detail information of expression images.Finally,experimental results illustrate that the proposed method achieves more competitive performance than several state-of-the-art methods on five benchmark datasets.

Deep Facial Emotion Recognition Using Local Features Based on Facial Landmarks for Security System

Emotion recognition based on facial expressions is one of the most critical elements of human-machine interfaces.Most conventional methods for emotion recognition using facial expressions use the entire facial image to extract features and then recognize specific emotions through a pre-trained model.In contrast,this paper proposes a novel feature vector extraction method using the Euclidean distance between the landmarks changing their positions according to facial expressions,especially around the eyes,eyebrows,nose,andmouth.Then,we apply a newclassifier using an ensemble network to increase emotion recognition accuracy.The emotion recognition performance was compared with the conventional algorithms using public databases.The results indicated that the proposed method achieved higher accuracy than the traditional based on facial expressions for emotion recognition.In particular,our experiments with the FER2013 database show that our proposed method is robust to lighting conditions and backgrounds,with an average of 25% higher performance than previous studies.Consequently,the proposed method is expected to recognize facial expressions,especially fear and anger,to help prevent severe accidents by detecting security-related or dangerous actions in advance.

A Cause-Selecting Control Chart Method for Monitoring and Diagnosing Dependent Manufacturing Process Stages

Many industrial products are normally processed through multiple manufacturing process stages before it becomes a final product.Statistical process control techniques often utilize standard Shewhart control charts to monitor these process stages.If the process stages are independent,this is a meaningful procedure.However,they are not independent in many manufacturing scenarios.The standard Shewhart control charts can not provide the information to determine which process stage or group of process stages has caused the problems(i.e.,standard Shewhart control charts could not diagnose dependent manufacturing process stages).This study proposes a selective neural network ensemble-based cause-selecting system of control charts to monitor these process stages and distinguish incoming quality problems and problems in the current stage of a manufacturing process.Numerical results show that the proposed method is an improvement over the use of separate Shewhart control chart for each of dependent process stages,and even ordinary quality practitioners who lack of expertise in theoretical analysis can implement regression estimation and neural computing readily.

Analysis of Negative Correlation Learning

This paper describes negative correlation learning for designing neural network ensembles. Negative correlation learning has been firstly analysed in terms of minimising mutual information on a regression task. By minimising the mutual information between variables extracted by two neural networks, they are forced to convey different information a-bout some features of their input. Based on the decision boundaries and correct response sets, negative correlation learning has been further studied on two pattern classification problems. The purpose of examining the decision boundaries and the correct response sets is not only to illustrate the learning behavior of negative correlation learning, but also to cast light on how to design more effective neural network ensembles. The experimental results showed the decision boundary of the trained neural network ensemble by negative correlation learning is almost as good as the optimum decision boundary.

Ensemble Prediction of Monsoon Index with a Genetic Neural Network Model

After the consideration of the nonlinear nature changes of monsoon index,and the subjective determination of network structure in traditional artificial neural network prediction modeling,monthly and seasonal monsoon intensity index prediction is studied in this paper by using nonlinear genetic neural network ensemble prediction（GNNEP）modeling.It differs from traditional prediction modeling in the following aspects： （1）Input factors of the GNNEP model of monsoon index were selected from a large quantity of preceding period high correlation factors,such as monthly sea temperature fields,monthly 500-hPa air temperature fields,monthly 200-hPa geopotential height fields,etc.,and they were also highly information-condensed and system dimensionality-reduced by using the empirical orthogonal function（EOF）method,which effectively condensed the useful information of predictors and therefore controlled the size of network structure of the GNNEP model.（2）In the input design of the GNNEP model,a mean generating function（MGF）series of predictand（monsoon index）was added as an input factor;the contrast analysis of results of predic- tion experiments by a physical variable predictor-predictand MGF GNNEP model and a physical variable predictor GNNEP model shows that the incorporation of the periodical variation of predictand（monsoon index）is very effective in improving the prediction of monsoon index.（3）Different from the traditional neural network modeling,the GNNEP modeling is able to objectively determine the network structure of the GNNNEP model,and the model constructed has a better generalization capability.In the case of identical predictors,prediction modeling samples,and independent prediction samples,the prediction accuracy of our GNNEP model combined with the system dimensionality reduction technique of predictors is clearly higher than that of the traditional stepwise regression model using the traditional treatment technique of predictors,suggesting that the GNNEP model opens up a vast range of possibilities for operational weather prediction.

Recent Advances in Evolutionary Computation

Evolutionary computation has experienced a tremendous growth in the last decade in both theoretical analyses and industrial applications. Its scope has evolved beyond its original meaning of ＂biological evolution＂ toward a wide variety of nature inspired computational algorithms and techniques, including evolutionary, neural, ecological, social and economical computation, etc, in a unified framework. Many research topics in evolutionary computation nowadays are not necessarily ＂evolutionary＂. This paper provides an overview of some recent advances in evolutionary computation that have been made in CERCIA at the University of Birmingham, UK. It covers a wide range of topics in optimization, learning and design using evolutionary approaches and techniques, and theoretical results in the computational time complexity of evolutionary algorithms. Some issues related to future development of evolutionary computation are also discussed.