Prediction of users online purchase behavior based on selective ensemble learning

A probabilistic multi-dimensional selective ensemble learning method and its application in the prediction of users' online purchase behavior are studied in this work.Firstly, the classifier is integrated based on the dimension of predicted probability, and the pruning algorithm based on greedy forward search is obtained by combining the two indicators of accuracy and complementarity.Then the pruning algorithm is integrated into the Stacking ensemble method to establish a user online shopping behavior prediction model based on the probabilistic multi-dimensional selective ensemble method.Finally, the research method is compared with the prediction results of individual learners in ensemble learning and the Stacking ensemble method without pruning.The experimental results show that the proposed method can reduce the scale of integration, improve the prediction accuracy of the model, and predict the user's online purchase behavior.

DDoS Attack Detection in Cloud Computing Based on Ensemble Feature Selection and Deep Learning

Intrusion Detection System(IDS)in the cloud Computing(CC)environment has received paramount interest over the last few years.Among the latest approaches,Deep Learning(DL)-based IDS methods allow the discovery of attacks with the highest performance.In the CC environment,Distributed Denial of Service(DDoS)attacks are widespread.The cloud services will be rendered unavailable to legitimate end-users as a consequence of the overwhelming network traffic,resulting in financial losses.Although various researchers have proposed many detection techniques,there are possible obstacles in terms of detection performance due to the use of insignificant traffic features.Therefore,in this paper,a hybrid deep learning mode based on hybridizing Convolutional Neural Network(CNN)with Long-Short-Term Memory(LSTM)is used due to its robustness and efficiency in detecting normal and attack traffic.Besides,the ensemble feature selection,mutualization aggregation between Particle Swarm Optimizer(PSO),Grey Wolf Optimizer(PSO),Krill Hird(KH),andWhale Optimization Algorithm(WOA),is used to select the most important features that would influence the detection performance in detecting DDoS attack in CC.A benchmark dataset proposed by the Canadian Institute of Cybersecurity(CIC),called CICIDS 2017 is used to evaluate the proposed IDS.The results revealed that the proposed IDS outperforms the state-of-the-art IDSs,as it achieved 97.9%,98.3%,97.9%,98.1%,respectively.As a result,the proposed IDS achieves the requirements of getting high security,automatic,efficient,and self-decision detection of DDoS attacks.

An Intelligent Hybrid Ensemble Gene Selection Model for Autism Using DNN

Autism Spectrum Disorder(ASD)is a complicated neurodevelopmen-tal disorder that is often identified in toddlers.The microarray data is used as a diagnostic tool to identify the genetics of the disorder.However,microarray data is large and has a high volume.Consequently,it suffers from the problem of dimensionality.In microarray data,the sample size and variance of the gene expression will lead to overfitting and misclassification.Identifying the autism gene(feature)subset from microarray data is an important and challenging research area.It has to be efficiently addressed to improve gene feature selection and classification.To overcome the challenges,a novel Intelligent Hybrid Ensem-ble Gene Selection(IHEGS)model is proposed in this paper.The proposed model integrates the intelligence of different feature selection techniques over the data partitions.In this model,the initial gene selection is carried out by data perturba-tion,and thefinal autism gene subset is obtained by functional perturbation,which reduces the problem of dimensionality in microarray data.The functional perturbation module employs three meta-heuristic swarm intelligence-based tech-niques for gene selection.The obtained gene subset is validated by the Deep Neural Network(DNN)model.The proposed model is implemented using python with six National Center for Biotechnology Information(NCBI)gene expression datasets.From the comparative study with other existing state-of-the-art systems,the proposed model provides stable results in terms of feature selection and clas-sification accuracy.

A new ensemble feature selection and its application to pattern classification

Neural network ensemble based on rough sets reduct is proposed to decrease the computational complexity of conventional ensemble feature selection algorithm. First, a dynamic reduction technology combining genetic algorithm with resampling method is adopted to obtain reducts with good generalization ability. Second, Multiple BP neural networks based on different reducts are built as base classifiers. According to the idea of selective ensemble, the neural network ensemble with best generalization ability can be found by search strategies. Finally, classification based on neural network ensemble is implemented by combining the predictions of component networks with voting. The method has been verified in the experiment of remote sensing image and five UCI datasets classification. Compared with conventional ensemble feature selection algorithms, it costs less time and lower computing complexity, and the classification accuracy is satisfactory.

A Cause-Selecting Control Chart Method for Monitoring and Diagnosing Dependent Manufacturing Process Stages

Many industrial products are normally processed through multiple manufacturing process stages before it becomes a final product.Statistical process control techniques often utilize standard Shewhart control charts to monitor these process stages.If the process stages are independent,this is a meaningful procedure.However,they are not independent in many manufacturing scenarios.The standard Shewhart control charts can not provide the information to determine which process stage or group of process stages has caused the problems(i.e.,standard Shewhart control charts could not diagnose dependent manufacturing process stages).This study proposes a selective neural network ensemble-based cause-selecting system of control charts to monitor these process stages and distinguish incoming quality problems and problems in the current stage of a manufacturing process.Numerical results show that the proposed method is an improvement over the use of separate Shewhart control chart for each of dependent process stages,and even ordinary quality practitioners who lack of expertise in theoretical analysis can implement regression estimation and neural computing readily.

Selective Ensemble Extreme Learning Machine Modeling of Effluent Quality in Wastewater Treatment Plants

Real-time and reliable measurements of the effluent quality are essential to improve operating efficiency and reduce energy consumption for the wastewater treatment process.Due to the low accuracy and unstable performance of the traditional effluent quality measurements,we propose a selective ensemble extreme learning machine modeling method to enhance the effluent quality predictions.Extreme learning machine algorithm is inserted into a selective ensemble frame as the component model since it runs much faster and provides better generalization performance than other popular learning algorithms.Ensemble extreme learning machine models overcome variations in different trials of simulations for single model.Selective ensemble based on genetic algorithm is used to further exclude some bad components from all the available ensembles in order to reduce the computation complexity and improve the generalization performance.The proposed method is verified with the data from an industrial wastewater treatment plant,located in Shenyang,China.Experimental results show that the proposed method has relatively stronger generalization and higher accuracy than partial least square,neural network partial least square,single extreme learning machine and ensemble extreme learning machine model.

Image Region Selection and Ensemble for Face Recognition

In this paper, a novel framework for face recognition, namely Selective Ensemble of Image Regions （SEIR）, is proposed. In this framework, all possible regions in the face image are regarded as a certain kind of features. There are two main steps in SEIR： the first step is to automatically select several regions from all possible candidates; the second step is to construct classifier ensemble from the selected regions. An implementation of SEIR based on multiple eigenspaces, namely SEME, is also proposed in this paper. SEME is analyzed and compared with eigenface, PCA ＋ LDA, eigenfeature, and eigenface ＋ eigenfeature through experiments. The experimental results show that SEME achieves the best performance.

Adaptive Spectral Clustering Ensemble Selection via Resampling and Population-Based Incremental Learning Algorithm

In this paper, we explore a novel ensemble method for spectral clustering. In contrast to the traditional clustering ensemble methods that combine all the obtained clustering results, we propose the adaptive spectral clustering ensemble method to achieve a better clustering solution. This method can adaptively assess the number of the component members, which is not owned by many other algorithms. The component clusterings of the ensemble system are generated by spectral clustering （SC） which bears some good characteristics to engender the diverse committees. The selection process works by evaluating the generated component spectral clustering through resampling technique and population-based incremental learning algorithm （PBIL）. Experimental results on UCI datasets demonstrate that the proposed algorithm can achieve better results compared with traditional clustering ensemble methods, especially when the number of component clusterings is large.

An air combat maneuver pattern extraction based on time series segmentation and clustering analysis

Target maneuver recognition is a prerequisite for air combat situation awareness,trajectory prediction,threat assessment and maneuver decision.To get rid of the dependence of the current target maneuver recognition method on empirical criteria and sample data,and automatically and adaptively complete the task of extracting the target maneuver pattern,in this paper,an air combat maneuver pattern extraction based on time series segmentation and clustering analysis is proposed by combining autoencoder,G-G clustering algorithm and the selective ensemble clustering analysis algorithm.Firstly,the autoencoder is used to extract key features of maneuvering trajectory to remove the impacts of redundant variables and reduce the data dimension;Then,taking the time information into account,the segmentation of Maneuver characteristic time series is realized with the improved FSTS-AEGG algorithm,and a large number of maneuver primitives are extracted;Finally,the maneuver primitives are grouped into some categories by using the selective ensemble multiple time series clustering algorithm,which can prove that each class represents a maneuver action.The maneuver pattern extraction method is applied to small scale air combat trajectory and can recognize and correctly partition at least 71.3%of maneuver actions,indicating that the method is effective and satisfies the requirements for engineering accuracy.In addition,this method can provide data support for various target maneuvering recognition methods proposed in the literature,greatly reduce the workload and improve the recognition accuracy.

Dynamic ensemble selection of convolutional neural networks and its application in flower classification

In recent years,convolutional neural networks(CNNs)have achieved great success in image classification.However,CNN models usually have complex network structures that tend to cause some related problems,such as redundancy of network parameters,low training efficiency,overfitting,and weak generalization ability.To solve these problems and improve the accuracy of flower classification,the advantages of CNNs were combined with those of ensemble learning and a method was developed for the dynamic ensemble selection of CNNs.First,MobileNet models pre-trained on a public dataset were transferred to flower datasets to train thirteen different MobileNet classifiers,and a resampling strategy was used to enhance the diversity of individual models.Second,the thirteen classifiers were sorted by a classifier sorting algorithm,before ensemble selection,to avoid an exhaustive search.Finally,with the credibility of recognition results,a classifier subset was dynamically selected and integrated to identify the flower species from their images.To verify the effectiveness,the proposed method was used to classify the images of five flower species.The accuracy of the proposed method was 95.50%,an improvement of 1.62%,3.94%,22.04%,13.77%,and 0.44%,over those of MobileNet,Inception-v1,ResNet-50,Inception-ResNet-v2,and the linear ensemble method,respectively.In addition,the performance of the proposed method was compared with five other methods for flower classification.The experimental results demonstrated the accuracy and robustness of the proposed method.

Measuring air traffic complexity based on small samples

Air traffic complexity is an objective metric for evaluating the operational condition of the airspace. It has several applications, such as airspace design and traffic flow management.Therefore, identifying a reliable method to accurately measure traffic complexity is important. Considering that many factors correlate with traffic complexity in complicated nonlinear ways,researchers have proposed several complexity evaluation methods based on machine learning models which were trained with large samples. However, the high cost of sample collection usually results in limited training set. In this paper, an ensemble learning model is proposed for measuring air traffic complexity within a sector based on small samples. To exploit the classification information within each factor, multiple diverse factor subsets（FSSs） are generated under guidance from factor noise and independence analysis. Then, a base complexity evaluator is built corresponding to each FSS. The final complexity evaluation result is obtained by integrating all results from the base evaluators. Experimental studies using real-world air traffic operation data demonstrate the advantages of our model for small-sample-based traffic complexity evaluation over other stateof-the-art methods.

A New Hybrid Machine Learning Model for Short-Term Climate Prediction by Performing Classification Prediction and Regression Prediction Simultaneously

Machine learning methods are effective tools for improving short-term climate prediction.However,commonly used methods often carry out classification and regression prediction modeling separately and independently.Such a single modeling approach may obtain inconsistent prediction results in classification and regression and thus may not meet the needs of practical applications well.To address this issue,this study proposes a selective Naive Bayes ensemble model(SENB-EM)by introducing causal effect and voting strategy on Naive Bayes.The new model can not only screen effective predictors but also perform classification and regression prediction simultaneously.After being applied to the area prediction of summer western North Pacific subtropical high(WNPSH)from 2008 to 2021,it is found that the accuracy classification score(a metric to assess the overall classification prediction accuracy)and the time correlation coefficient(TCC)of SENB-EM can reach 1.0 and 0.81,respectively.After integrating the results of different models[including multiple linear regression ensemble model(MLR-EM),SENB-EM,and Chinese Multimodel Ensemble Prediction System(CMME)used by National Climate Center(NCC)]for 2017-2021,the TCC of the ensemble results of SENB-EM and CMME can reach 0.92(the highest result among them).This indicates that the prediction results of the summer WNPSH area provided by SENB-EM have a high reference value for the real-time prediction.It is worth noting that,except for the numerical prediction results,the SENB-EM model can also give the range of numerical prediction intervals and predictions for anomalous degrees of the WNPSH area,thus providing more reference information for meteorological forecasters.Overall,as a new hybrid machine learning model,the SENB-EM has a good prediction ability;the approach of performing classification prediction and regression prediction simultaneously through integration is informative to short-term climate prediction.