A feature selection method combined with ridge regression and recursive feature elimination in quantitative analysis of laser induced breakdown spectroscopy

In the spectral analysis of laser-induced breakdown spectroscopy,abundant characteristic spectral lines and severe interference information exist simultaneously in the original spectral data.Here,a feature selection method called recursive feature elimination based on ridge regression(Ridge-RFE)for the original spectral data is recommended to make full use of the valid information of spectra.In the Ridge-RFE method,the absolute value of the ridge regression coefficient was used as a criterion to screen spectral characteristic,the feature with the absolute value of minimum weight in the input subset features was removed by recursive feature elimination(RFE),and the selected features were used as inputs of the partial least squares regression(PLS)model.The Ridge-RFE method based PLS model was used to measure the Fe,Si,Mg,Cu,Zn and Mn for 51 aluminum alloy samples,and the results showed that the root mean square error of prediction decreased greatly compared to the PLS model with full spectrum as input.The overall results demonstrate that the Ridge-RFE method is more efficient to extract the redundant features,make PLS model for better quantitative analysis results and improve model generalization ability.

An efficient stock market prediction model using hybrid feature reduction method based on variational autoencoders and recursive feature elimination

In this study,the hourly directions of eight banking stocks in Borsa Istanbul were predicted using linear-based,deep-learning(LSTM)and ensemble learning(Light-GBM)models.These models were trained with four different feature sets and their performances were evaluated in terms of accuracy and F-measure metrics.While the first experiments directly used the own stock features as the model inputs,the second experiments utilized reduced stock features through Variational AutoEncoders(VAE).In the last experiments,in order to grasp the effects of the other banking stocks on individual stock performance,the features belonging to other stocks were also given as inputs to our models.While combining other stock features was done for both own(named as allstock_own)and VAE-reduced(named as allstock_VAE)stock features,the expanded dimensions of the feature sets were reduced by Recursive Feature Elimination.As the highest success rate increased up to 0.685 with allstock_own and LSTM with attention model,the combination of allstock_VAE and LSTM with the attention model obtained an accuracy rate of 0.675.Although the classification results achieved with both feature types was close,allstock_VAE achieved these results using nearly 16.67%less features compared to allstock_own.When all experimental results were examined,it was found out that the models trained with allstock_own and allstock_VAE achieved higher accuracy rates than those using individual stock features.It was also concluded that the results obtained with the VAE-reduced stock features were similar to those obtained by own stock features.

DR-XGBoost: An XGBoost model for field-road segmentation based on dual feature extraction and recursive feature elimination

Field-road segmentation is one of the key tasks in the processing of the trajectory of agricultural machinery.To improve the accuracy of the field-road segmentation,this study proposed an XGBoost model based on dual feature extraction and recursive feature elimination called DR-XGBoost.DR-XGBoost takes only a small amount of agricultural machine trajectory features as input.Firstly,the model adopted the dual feature extraction method we designed to rapidly expand the number of features and then adequately extract local trajectory features by the time window and feature extraction operator.Secondly,the model applies the recursive feature elimination algorithm to eliminate redundant features from the perspective of the model segmentation effect and thus reduce the computational consumption of model training.Thirdly,it trains XGBoost to complete the trajectory segmentation.To evaluate the effectiveness of DR-XGBoost,we conducted a series of experiments on a real trajectory dataset of agricultural machines.The model achieves a 98.2%Macro-F1 score on the dataset,which is 10.9%higher than the previous state-of-art.The proposal of DR-XGBoost fills the knowledge gap of trajectory feature extraction for agricultural machinery and provides a reasonable and effective feature selection scheme for the field-road segmentation problem.

Analysis of Feature Importance and Interpretation for Malware Classification

This study was conducted to enable prompt classification of malware,which was becoming increasingly sophisticated.To do this,we analyzed the important features of malware and the relative importance of selected features according to a learning model to assess how those important features were identified.Initially,the analysis features were extracted using Cuckoo Sandbox,an open-source malware analysis tool,then the features were divided into five categories using the extracted information.The 804 extracted features were reduced by 70%after selecting only the most suitable ones for malware classification using a learning model-based feature selection method called the recursive feature elimination.Next,these important features were analyzed.The level of contribution from each one was assessed by the Random Forest classifier method.The results showed that System call features were mostly allocated.At the end,it was possible to accurately identify the malware type using only 36 to 76 features for each of the four types of malware with the most analysis samples available.These were the Trojan,Adware,Downloader,and Backdoor malware.

Diabetes Prediction Algorithm Using Recursive Ridge Regression L2

At present,the prevalence of diabetes is increasing because the human body cannot metabolize the glucose level.Accurate prediction of diabetes patients is an important research area.Many researchers have proposed techniques to predict this disease through data mining and machine learning methods.In prediction,feature selection is a key concept in preprocessing.Thus,the features that are relevant to the disease are used for prediction.This condition improves the prediction accuracy.Selecting the right features in the whole feature set is a complicated process,and many researchers are concentrating on it to produce a predictive model with high accuracy.In this work,a wrapper-based feature selection method called recursive feature elimination is combined with ridge regression(L2)to form a hybrid L2 regulated feature selection algorithm for overcoming the overfitting problem of data set.Overfitting is a major problem in feature selection,where the new data are unfit to the model because the training data are small.Ridge regression is mainly used to overcome the overfitting problem.The features are selected by using the proposed feature selection method,and random forest classifier is used to classify the data on the basis of the selected features.This work uses the Pima Indians Diabetes data set,and the evaluated results are compared with the existing algorithms to prove the accuracy of the proposed algorithm.The accuracy of the proposed algorithm in predicting diabetes is 100%,and its area under the curve is 97%.The proposed algorithm outperforms existing algorithms.

An Intrusion Detection System for SDN Using Machine Learning

Software Defined Networking(SDN)has emerged as a promising and exciting option for the future growth of the internet.SDN has increased the flexibility and transparency of the managed,centralized,and controlled network.On the other hand,these advantages create a more vulnerable environment with substantial risks,culminating in network difficulties,system paralysis,online banking frauds,and robberies.These issues have a significant detrimental impact on organizations,enterprises,and even economies.Accuracy,high performance,and real-time systems are necessary to achieve this goal.Using a SDN to extend intelligent machine learning methodologies in an Intrusion Detection System(IDS)has stimulated the interest of numerous research investigators over the last decade.In this paper,a novel HFS-LGBM IDS is proposed for SDN.First,the Hybrid Feature Selection algorithm consisting of two phases is applied to reduce the data dimension and to obtain an optimal feature subset.In thefirst phase,the Correlation based Feature Selection(CFS)algorithm is used to obtain the feature subset.The optimal feature set is obtained by applying the Random Forest Recursive Feature Elimination(RF-RFE)in the second phase.A LightGBM algorithm is then used to detect and classify different types of attacks.The experimental results based on NSL-KDD dataset show that the proposed system produces outstanding results compared to the existing methods in terms of accuracy,precision,recall and f-measure.

Landslide susceptibility mapping using hybrid random forest with GeoDetector and RFE for factor optimization

The present study aims to develop two hybrid models to optimize the factors and enhance the predictive ability of the landslide susceptibility models.For this,a landslide inventory map was created with 406 historical landslides and 2030 non-landslide points,which was randomly divided into two datasets for model training(70%)and model testing(30%).22 factors were initially selected to establish a landslide factor database.We applied the GeoDetector and recursive feature elimination method(RFE)to address factor optimization to reduce information redundancy and collinearity in the data.Thereafter,the frequency ratio method,multicollinearity test,and interactive detector were used to analyze and evaluate the optimized factors.Subsequently,the random forest(RF)model was used to create a landslide susceptibility map with original and optimized factors.The resultant hybrid models GeoDetector-RF and RFE-RF were evaluated and compared by the area under the receiver operating characteristic curve(AUC)and accuracy.The accuracy of the two hybrid models(0.868 for GeoDetector-RF and 0.869 for RFE-RF)were higher than that of the RF model(0.860),indicating that the hybrid models with factor optimization have high reliability and predictability.Both RFE-RF GeoDetector-RF had higher AUC values,respectively 0.863 and 0.860,than RF(0.853).These results confirm the ability of factor optimization methods to improve the performance of landslide susceptibility models.

Risk prediction platform for pancreatic fistula after pancreatoduodenectomy using artificial intelligence

BACKGROUND Despite advancements in operative technique and improvements in postoperative managements,postoperative pancreatic fistula(POPF)is a life-threatening complication following pancreatoduodenectomy(PD).There are some reports to predict POPF preoperatively or intraoperatively,but the accuracy of those is questionable.Artificial intelligence(AI)technology is being actively used in the medical field,but few studies have reported applying it to outcomes after PD.AIM To develop a risk prediction platform for POPF using an AI model.METHODS Medical records were reviewed from 1769 patients at Samsung Medical Center who underwent PD from 2007 to 2016.A total of 38 variables were inserted into AI-driven algorithms.The algorithms tested to make the risk prediction platform were random forest(RF)and a neural network(NN)with or without recursive feature elimination(RFE).The median imputation method was used for missing values.The area under the curve(AUC)was calculated to examine the discriminative power of algorithm for POPF prediction.RESULTS The number of POPFs was 221(12.5%)according to the International Study Group of Pancreatic Fistula definition 2016.After median imputation,AUCs using 38 variables were 0.68±0.02 with RF and 0.71±0.02 with NN.The maximal AUC using NN with RFE was 0.74.Sixteen risk factors for POPF were identified by AI algorithm:Pancreatic duct diameter,body mass index,preoperative serum albumin,lipase level,amount of intraoperative fluid infusion,age,platelet count,extrapancreatic location of tumor,combined venous resection,co-existing pancreatitis,neoadjuvant radiotherapy,American Society of Anesthesiologists’score,sex,soft texture of the pancreas,underlying heart disease,and preoperative endoscopic biliary decompression.We developed a web-based POPF prediction platform,and this application is freely available at http://popfrisk.smchbp.org.CONCLUSION This study is the first to predict POPF with multiple risk factors using AI.This platform is reliable(AUC 0.74),so it could be used to select patients who need especially intense therapy and to preoperatively establish an effective treatment strategy.