Selective ensemble modeling based on nonlinear frequency spectral feature extraction for predicting load parameter in ball mills

Strong mechanical vibration and acoustical signals of grinding process contain useful information related to load parameters in ball mills. It is a challenge to extract latent features and construct soft sensor model with high dimensional frequency spectra of these signals. This paper aims to develop a selective ensemble modeling approach based on nonlinear latent frequency spectral feature extraction for accurate measurement of material to ball volume ratio. Latent features are first extracted from different vibrations and acoustic spectral segments by kernel partial least squares. Algorithms of bootstrap and least squares support vector machines are employed to produce candidate sub-models using these latent features as inputs. Ensemble sub-models are selected based on genetic algorithm optimization toolbox. Partial least squares regression is used to combine these sub-models to eliminate collinearity among their prediction outputs. Results indicate that the proposed modeling approach has better prediction performance than previous ones.

Prediction of users online purchase behavior based on selective ensemble learning

A probabilistic multi-dimensional selective ensemble learning method and its application in the prediction of users' online purchase behavior are studied in this work.Firstly, the classifier is integrated based on the dimension of predicted probability, and the pruning algorithm based on greedy forward search is obtained by combining the two indicators of accuracy and complementarity.Then the pruning algorithm is integrated into the Stacking ensemble method to establish a user online shopping behavior prediction model based on the probabilistic multi-dimensional selective ensemble method.Finally, the research method is compared with the prediction results of individual learners in ensemble learning and the Stacking ensemble method without pruning.The experimental results show that the proposed method can reduce the scale of integration, improve the prediction accuracy of the model, and predict the user's online purchase behavior.

A New Selective Neural Network Ensemble Method Based on Error Vectorization and Its Application in High-density Polyethylene (HDPE) Cascade Reaction Process

Chemical processes are complex, for which traditional neural network models usually can not lead to satisfactory accuracy. Selective neural network ensemble is an effective way to enhance the generalization accuracy of networks, but there are some problems, e.g., lacking of unified definition of diversity among component neural networks and difficult to improve the accuracy by selecting if the diversities of available networks are small. In this study, the output errors of networks are vectorized, the diversity of networks is defined based on the error vectors, and the size of ensemble is analyzed. Then an error vectorization based selective neural network ensemble (EVSNE) is proposed, in which the error vector of each network can offset that of the other networks by training the component networks orderly. Thus the component networks have large diversity. Experiments and comparisons over standard data sets and actual chemical process data set for production of high-density polyethylene demonstrate that EVSNE performs better in generalization ability.

Augmenting Internet of Medical Things Security:Deep Ensemble Integration and Methodological Fusion

When it comes to smart healthcare business systems,network-based intrusion detection systems are crucial for protecting the system and its networks from malicious network assaults.To protect IoMT devices and networks in healthcare and medical settings,our proposed model serves as a powerful tool for monitoring IoMT networks.This study presents a robust methodology for intrusion detection in Internet of Medical Things(IoMT)environments,integrating data augmentation,feature selection,and ensemble learning to effectively handle IoMT data complexity.Following rigorous preprocessing,including feature extraction,correlation removal,and Recursive Feature Elimi-nation(RFE),selected features are standardized and reshaped for deep learning models.Augmentation using the BAT algorithm enhances dataset variability.Three deep learning models,Transformer-based neural networks,self-attention Deep Convolutional Neural Networks(DCNNs),and Long Short-Term Memory(LSTM)networks,are trained to capture diverse data aspects.Their predictions form a meta-feature set for a subsequent meta-learner,which combines model strengths.Conventional classifiers validate meta-learner features for broad algorithm suitability.This comprehensive method demonstrates high accuracy and robustness in IoMT intrusion detection.Evaluations were conducted using two datasets:the publicly available WUSTL-EHMS-2020 dataset,which contains two distinct categories,and the CICIoMT2024 dataset,encompassing sixteen categories.Experimental results showcase the method’s exceptional performance,achieving optimal scores of 100%on the WUSTL-EHMS-2020 dataset and 99%on the CICIoMT2024.

Ensemble Filter-Wrapper Text Feature Selection Methods for Text Classification

Feature selection is a crucial technique in text classification for improving the efficiency and effectiveness of classifiers or machine learning techniques by reducing the dataset’s dimensionality.This involves eliminating irrelevant,redundant,and noisy features to streamline the classification process.Various methods,from single feature selection techniques to ensemble filter-wrapper methods,have been used in the literature.Metaheuristic algorithms have become popular due to their ability to handle optimization complexity and the continuous influx of text documents.Feature selection is inherently multi-objective,balancing the enhancement of feature relevance,accuracy,and the reduction of redundant features.This research presents a two-fold objective for feature selection.The first objective is to identify the top-ranked features using an ensemble of three multi-univariate filter methods:Information Gain(Infogain),Chi-Square(Chi^(2)),and Analysis of Variance(ANOVA).This aims to maximize feature relevance while minimizing redundancy.The second objective involves reducing the number of selected features and increasing accuracy through a hybrid approach combining Artificial Bee Colony(ABC)and Genetic Algorithms(GA).This hybrid method operates in a wrapper framework to identify the most informative subset of text features.Support Vector Machine(SVM)was employed as the performance evaluator for the proposed model,tested on two high-dimensional multiclass datasets.The experimental results demonstrated that the ensemble filter combined with the ABC+GA hybrid approach is a promising solution for text feature selection,offering superior performance compared to other existing feature selection algorithms.

Metaheuristic-Driven Two-Stage Ensemble Deep Learning for Lung/Colon Cancer Classification

This study investigates the application of deep learning,ensemble learning,metaheuristic optimization,and image processing techniques for detecting lung and colon cancers,aiming to enhance treatment efficacy and improve survival rates.We introduce a metaheuristic-driven two-stage ensemble deep learning model for efficient lung/colon cancer classification.The diagnosis of lung and colon cancers is attempted using several unique indicators by different versions of deep Convolutional Neural Networks(CNNs)in feature extraction and model constructions,and utilizing the power of various Machine Learning(ML)algorithms for final classification.Specifically,we consider different scenarios consisting of two-class colon cancer,three-class lung cancer,and fiveclass combined lung/colon cancer to conduct feature extraction using four CNNs.These extracted features are then integrated to create a comprehensive feature set.In the next step,the optimization of the feature selection is conducted using a metaheuristic algorithm based on the Electric Eel Foraging Optimization(EEFO).This optimized feature subset is subsequently employed in various ML algorithms to determine the most effective ones through a rigorous evaluation process.The top-performing algorithms are refined using the High-Performance Filter(HPF)and integrated into an ensemble learning framework employing weighted averaging.Our findings indicate that the proposed ensemble learning model significantly surpasses existing methods in classification accuracy across all datasets,achieving accuracies of 99.85%for the two-class,98.70%for the three-class,and 98.96%for the five-class datasets.

Ensemble-SISPLS近红外光谱变量选择方法

近红外光谱具有高维小样本的特点,变量选择是提高定量分析模型稳健性和可解释性的一种有效方法。确定独立筛选(SIS)是一种基于边际相关性的超高维数据变量选择方法,广泛用于基因微阵列数据的变量选择。SIS具有将数据维度降低至样本大小规模的能力,其降维能力与LASSO相当,在相当宽泛的近似条件下,由于具有安全筛选性质,所有重要变量被保留的概率趋于1。基于确定独立筛选偏最小二乘(SIS-SPLS)的变量选择是一种迭代式的SIS变量选择方法,首先利用SIS方法完成光谱重要变量的初选;然后根据重要变量的边际相关性大小进行逐步前向选择:建立偏最小二乘回归模型,依据贝叶斯信息准则(BIC)确定最终的变量选择结果。SIS-SPLS以逐步前向选择的方式实现对重要变量的增量式筛选,随着潜变量个数的增加及因变量残差的逐步减小, SIS-SPLS方法选择的变量个数将趋于稳定。然而仅以边际相关性对变量重要性进行评价,当光谱变量个数远大于样本数时,该方法也存在选择的变量过多、变量选择结果不够稳健等问题。为进一步提高小样本情况下变量选择的稳健性,将集成学习引入SIS-SPLS方法之中,提出了一种集成SIS-SPLS变量选择方法(Ensemble-SISPLS)。该方法首先对校正集样本进行自助重采样,对采样得到的每一个校正子集分别使用SIS-SPLS方法进行变量筛选,通过投票机制并设置频次阈值对所有校正子集的变量选择结果进行集成,选择出现频次大于给定阈值的变量并建立偏最小二乘回归模型,计算5折交叉验证均方根误差。对频次阈值和潜变量个数两个关键参数使用网格搜索法进行优选,根据子模型的交叉验证均方根误差和变量个数对子模型性能进行综合评价,以最优子模型包含的变量作为最终的变量选择结果。分别在Corn数据集和当归数据集上进行变量选择实验,比较Ensemble-SISPLS, SIS-SPLS和UVE-PLS三种变量选择方法的性能。其中当归数据集共77个样本,样本采自甘肃岷县和渭源县,使用Nicolet-6700型近红外光谱仪扫描得到所有样本的近红外光谱并对当归中的阿魏酸含量进行预测。Ensemble-SISPLS方法在Corn数据集上选择的变量个数、 RMSEP和决定系数分别为22, 0.000 8和0.999 8; SIS-SPLS方法在Corn数据集上选择的变量个数、 RMSEP和决定系数分别为97, 0.007 3和0.998 8。Ensemble-SISPLS方法在当归数据集上选择的变量个数、 RMSEP和决定系数分别为24, 0.018 1和0.996 3; SIS-SPLS方法在当归数据集上选择的变量个数、 RMSEP和决定系数分别为38, 0.022 6和0.994 3。结果表明,该方法进一步提高了变量选择结果的稳健性和预测能力。Ensemble-SISPLS变量选择方法有效结合了SIS-SPLS较强的变量选择能力和集成学习良好的泛化能力,提高了变量选择的稳健性。此外,由于在子模型的预测能力和变量个数之间进行了折中,一定程度上减少了选择变量的个数,提高了模型的可解释性。

Characterization of three-dimensional channel reservoirs using ensemble Kalman filter assisted by principal component analysis

Ensemble-based analyses are useful to compare equiprobable scenarios of the reservoir models.However,they require a large suite of reservoir models to cover high uncertainty in heterogeneous and complex reservoir models.For stable convergence in ensemble Kalman filter(EnKF),increasing ensemble size can be one of the solutions,but it causes high computational cost in large-scale reservoir systems.In this paper,we propose a preprocessing of good initial model selection to reduce the ensemble size,and then,EnKF is utilized to predict production performances stochastically.In the model selection scheme,representative models are chosen by using principal component analysis(PCA)and clustering analysis.The dimension of initial models is reduced using PCA,and the reduced models are grouped by clustering.Then,we choose and simulate representative models from the cluster groups to compare errors of production predictions with historical observation data.One representative model with the minimum error is considered as the best model,and we use the ensemble members near the best model in the cluster plane for applying EnKF.We demonstrate the proposed scheme for two 3D models that EnKF provides reliable assimilation results with much reduced computation time.

Optimized Ensemble Algorithm for Predicting Metamaterial Antenna Parameters

Metamaterial Antenna is a subclass of antennas that makes use of metamaterial to improve performance.Metamaterial antennas can overcome the bandwidth constraint associated with tiny antennas.Machine learning is receiving a lot of interest in optimizing solutions in a variety of areas.Machine learning methods are already a significant component of ongoing research and are anticipated to play a critical role in today’s technology.The accuracy of the forecast is mostly determined by the model used.The purpose of this article is to provide an optimal ensemble model for predicting the bandwidth and gain of the Metamaterial Antenna.Support Vector Machines(SVM),Random Forest,K-Neighbors Regressor,and Decision Tree Regressor were utilized as the basic models.The Adaptive Dynamic Polar Rose Guided Whale Optimization method,named AD-PRS-Guided WOA,was used to pick the optimal features from the datasets.The suggested model is compared to models based on five variables and to the average ensemble model.The findings indicate that the presented model using Random Forest results in a Root Mean Squared Error(RMSE)of(0.0102)for bandwidth and RMSE of(0.0891)for gain.This is superior to other models and can accurately predict antenna bandwidth and gain.

A new ensemble feature selection and its application to pattern classification

Neural network ensemble based on rough sets reduct is proposed to decrease the computational complexity of conventional ensemble feature selection algorithm. First, a dynamic reduction technology combining genetic algorithm with resampling method is adopted to obtain reducts with good generalization ability. Second, Multiple BP neural networks based on different reducts are built as base classifiers. According to the idea of selective ensemble, the neural network ensemble with best generalization ability can be found by search strategies. Finally, classification based on neural network ensemble is implemented by combining the predictions of component networks with voting. The method has been verified in the experiment of remote sensing image and five UCI datasets classification. Compared with conventional ensemble feature selection algorithms, it costs less time and lower computing complexity, and the classification accuracy is satisfactory.

Ensemble feature selection integrating elitist roles and quantum game model

To accelerate the selection process of feature subsets in the rough set theory （RST）, an ensemble elitist roles based quantum game （EERQG） algorithm is proposed for feature selec- tion. Firstly, the multilevel elitist roles based dynamics equilibrium strategy is established, and both immigration and emigration of elitists are able to be self-adaptive to balance between exploration and exploitation for feature selection. Secondly, the utility matrix of trust margins is introduced to the model of multilevel elitist roles to enhance various elitist roles＇ performance of searching the optimal feature subsets, and the win-win utility solutions for feature selec- tion can be attained. Meanwhile, a novel ensemble quantum game strategy is designed as an intriguing exhibiting structure to perfect the dynamics equilibrium of multilevel elitist roles. Finally, the en- semble manner of multilevel elitist roles is employed to achieve the global minimal feature subset, which will greatly improve the fea- sibility and effectiveness. Experiment results show the proposed EERQG algorithm has superiority compared to the existing feature selection algorithms.

Ensemble Learning Based Collaborative Filtering with Instance Selectionand Enhanced Clustering

Recommender system is a tool to suggest items to the users from the extensive history of the user’s feedback.Though,it is an emerging research area concerning academics and industries,where it suffers from sparsity,scalability,and cold start problems.This paper addresses sparsity,and scalability problems of model-based collaborative recommender system based on ensemble learning approach and enhanced clustering algorithm for movie recommendations.In this paper,an effective movie recommendation system is proposed by Classification and Regression Tree(CART)algorithm,enhanced Balanced Iterative Reducing and Clustering using Hierarchies(BIRCH)algorithm and truncation method.In this research paper,a new hyper parameters tuning is added in BIRCH algorithm to enhance the cluster formation process,where the proposed algorithm is named as enhanced BIRCH.The proposed model yields quality movie recommendation to the new user using Gradient boost classification with broad coverage.In this paper,the proposed model is tested on Movielens dataset,and the performance is evaluated by means of Mean Absolute Error(MAE),precision,recall and f-measure.The experimental results showed the superiority of proposed model in movie recommendation compared to the existing models.The proposed model obtained 0.52 and 0.57 MAE value on Movielens 100k and 1M datasets.Further,the proposed model obtained 0.83 of precision,0.86 of recall and 0.86 of f-measure on Movielens 100k dataset,which are effective compared to the existing models in movie recommendation.

Construction of Parsimonious Event Risk Scores by an Ensemble Method. An Illustration for Short-Term Predictions in Chronic Heart Failure Patients from the GISSI-HF Trial

Selecting which explanatory variables to include in a given score is a common difficulty, as a balance must be found between statistical fit and practical application. This article presents a methodology for constructing parsimonious event risk scores combining a stepwise selection of variables with ensemble scores obtained by aggregation of several scores, using several classifiers, bootstrap samples and various modalities of random selection of variables. Selection methods based on a probabilistic model can be used to achieve a stepwise selection for a given classifier such as logistic regression, but not directly for an ensemble classifier constructed by aggregation of several classifiers. Three selection methods are proposed in this framework, two involving a backward selection of the variables based on their coefficients in an ensemble score and the third involving a forward selection of the variables maximizing the AUC. The stepwise selection allows constructing a succession of scores, with the practitioner able to choose which score best fits his needs. These three methods are compared in an application to construct parsimonious short-term event risk scores in chronic HF patients, using as event the composite endpoint of death or hospitalization for worsening HF within 180 days of a visit. Focusing on the fastest method, four scores are constructed, yielding out-of-bag AUCs ranging from 0.81 (26 variables) to 0.76 (2 variables).

Common Spatial Pattern Ensemble Classifier and Its Application in Brain-Computer Interface

Abstract-Common spatial pattern （CSP） algorithm is a successful tool in feature estimate of brain-computer interface （BCI）. However, CSP is sensitive to outlier and may result in poor outcomes since it is based on pooling the covariance matrices of trials. In this paper, we propose a simple yet effective approach, named common spatial pattern ensemble （CSPE） classifier, to improve CSP performance. Through division of recording channels, multiple CSP filters are constructed. By projection, log-operation, and subtraction on the original signal, an ensemble classifier, majority voting, is achieved and outlier contaminations are alleviated. Experiment results demonstrate that the proposed CSPE classifier is robust to various artifacts and can achieve an average accuracy of 83.02%.

Enhanced Classification Accuracy for Cardiotocogram Data with Ensemble Feature Selection and Classifier Ensemble

In this paper ensemble learning based feature selection and classifier ensemble model is proposed to improve classification accuracy. The hypothesis is that good feature sets contain features that are highly correlated with the class from ensemble feature selection to SVM ensembles which can be achieved on the performance of classification accuracy. The proposed approach consists of two phases: (i) to select feature sets that are likely to be the support vectors by applying ensemble based feature selection methods;and (ii) to construct an SVM ensemble using the selected features. The proposed approach was evaluated by experiments on Cardiotocography dataset. Four feature selection techniques were used: (i) Correlation-based, (ii) Consistency-based, (iii) ReliefF and (iv) Information Gain. Experimental results showed that using the ensemble of Information Gain feature selection and Correlation-based feature selection with SVM ensembles achieved higher classification accuracy than both single SVM classifier and ensemble feature selection with SVM classifier.

Ensemble habitat suitability modeling of stomatopods with Oratosquilla oratoria as an example

Stomatopods are better known as mantis shrimp with considerable ecological importance in wide coastal waters globally. Some stomatopod species are exploited commercially, including Oratosquilla oratoria in the Northwest Pacific. Yet, few studies have published to promote accurate habitat identification of stomatopods, obstructing scientific management and conservation of these valuable organisms. This study provides an ensemble modeling framework for habitat suitability modeling of stomatopods, utilizing the O. oratoria stock in the Bohai Sea as an example. Two modeling techniques(i.e., generalized additive model(GAM) and geographical weighted regression(GWR)) were applied to select environmental predictors(especially the selection between two types of sediment metrics) that better characterize O. oratoria distribution and build separate habitat suitability models(HSM). The performance of the individual HSMs were compared on interpolation accuracy and transferability.Then, they were integrated to check whether the ensemble model outperforms either individual model, according to fishers’ knowledge and scientific survey data. As a result, grain-size metrics of sediment outperformed sediment content metrics in modeling O. oratoria habitat, possibly because grain-size metrics not only reflect the effect of substrates on burrow development, but also link to sediment heat capacity which influences individual thermoregulation. Moreover, the GWR-based HSM outperformed the GAM-based HSM in interpolation accuracy,while the latter one displayed better transferability. On balance, the ensemble HSM appeared to improve the predictive performance overall, as it could avoid dependence on a single model type and successfully identified fisher-recognized and survey-indicated suitable habitats in either sparsely sampled or well investigated areas.

A Study of Ensemble Feature Selection and Adversarial Training for Malicious User Detection

The continuously booming of information technology has shed light on developing a variety of communication networks,multimedia,social networks and Internet of Things applications.However,users inevitably suffer from the intrusion of malicious users.Some studies focus on static characteristics of malicious users,which is easy to be bypassed by camouflaged malicious users.In this paper,we present a malicious user detection method based on ensemble feature selection and adversarial training.Firstly,the feature selection alleviates the dimension disaster problem and achieves more accurate classification performance.Secondly,we embed features into the multidimensional space and aggregate it into a feature map to encode the explicit content preference and implicit interaction preference.Thirdly,we use an effective ensemble learning which could avoid over-fitting and has good noise resistance.Finally,we propose a datadriven neural network detection model with the regularization technique adversarial training to deeply analyze the characteristics.It simplifies the parameters,obtaining more robust interaction features and pattern features.We demonstrate the effectiveness of our approach with numerical simulation results for malicious user detection,where the robustness issues are notable concerns.

Ensemble Variable Selection for Naive Bayes to Improve Customer Behaviour Analysis

Executing customer analysis in a systemic way is one of the possible solutions for each enterprise to understand the behavior of consumer patterns in an efficient and in-depth manner.Further investigation of customer patterns helps thefirm to develop efficient decisions and in turn,helps to optimize the enter-prise’s business and maximizes consumer satisfaction correspondingly.To con-duct an effective assessment about the customers,Naive Bayes(also called Simple Bayes),a machine learning model is utilized.However,the efficacious of the simple Bayes model is utterly relying on the consumer data used,and the existence of uncertain and redundant attributes in the consumer data enables the simple Bayes model to attain the worst prediction in consumer data because of its presumption regarding the attributes applied.However,in practice,the NB pre-mise is not true in consumer data,and the analysis of these redundant attributes enables simple Bayes model to get poor prediction results.In this work,an ensem-ble attribute selection methodology is performed to overcome the problem with consumer data and to pick a steady uncorrelated attribute set to model with the NB classifier.In ensemble variable selection,two different strategies are applied:one is based upon data perturbation(or homogeneous ensemble,same feature selector is applied to a different subsamples derived from the same learning set)and the other one is based upon function perturbation(or heterogeneous ensemble different feature selector is utilized to the same learning set).Further-more,the feature set captured from both ensemble strategies is applied to NB indi-vidually and the outcome obtained is computed.Finally,the experimental outcomes show that the proposed ensemble strategies perform efficiently in choosing a steady attribute set and increasing NB classification performance efficiently.

An Intelligent Hybrid Ensemble Gene Selection Model for Autism Using DNN

Autism Spectrum Disorder(ASD)is a complicated neurodevelopmen-tal disorder that is often identified in toddlers.The microarray data is used as a diagnostic tool to identify the genetics of the disorder.However,microarray data is large and has a high volume.Consequently,it suffers from the problem of dimensionality.In microarray data,the sample size and variance of the gene expression will lead to overfitting and misclassification.Identifying the autism gene(feature)subset from microarray data is an important and challenging research area.It has to be efficiently addressed to improve gene feature selection and classification.To overcome the challenges,a novel Intelligent Hybrid Ensem-ble Gene Selection(IHEGS)model is proposed in this paper.The proposed model integrates the intelligence of different feature selection techniques over the data partitions.In this model,the initial gene selection is carried out by data perturba-tion,and thefinal autism gene subset is obtained by functional perturbation,which reduces the problem of dimensionality in microarray data.The functional perturbation module employs three meta-heuristic swarm intelligence-based tech-niques for gene selection.The obtained gene subset is validated by the Deep Neural Network(DNN)model.The proposed model is implemented using python with six National Center for Biotechnology Information(NCBI)gene expression datasets.From the comparative study with other existing state-of-the-art systems,the proposed model provides stable results in terms of feature selection and clas-sification accuracy.

Reactions’Descriptors Selection and Yield Estimation Using Metaheuristic Algorithms and Voting Ensemble

Bioactive compounds in plants,which can be synthesized using N-arylationmethods such as the Buchwald-Hartwig reaction,are essential in drug discovery for their pharmacological effects.Important descriptors are necessary for the estimation of yields in these reactions.This study explores ten metaheuristic algorithms for descriptor selection and model a voting ensemble for evaluation.The algorithms were evaluated based on computational time and the number of selected descriptors.Analyses show that robust performance is obtained with more descriptors,compared to cases where fewer descriptors are selected.The essential descriptor was deduced based on the frequency of occurrence within the 50 extracted data subsets,and better performance was achieved with the voting ensemble than other algorithms with RMSE of 6.4270 and R^(2) of 0.9423.The results and deductions from this study can be readily applied in the decision-making process of chemical synthesis by saving the computational cost associated with initial descriptor selection for yield estimation.The ensemble model has also shown robust performance in its yield estimation ability and efficiency.