Machine Learning and Synthetic Minority Oversampling Techniques for Imbalanced Data: Improving Machine Failure Prediction

Prediction of machine failure is challenging as the dataset is often imbalanced with a low failure rate.The common approach to han-dle classification involving imbalanced data is to balance the data using a sampling approach such as random undersampling,random oversampling,or Synthetic Minority Oversampling Technique(SMOTE)algorithms.This paper compared the classification performance of three popular classifiers(Logistic Regression,Gaussian Naïve Bayes,and Support Vector Machine)in predicting machine failure in the Oil and Gas industry.The original machine failure dataset consists of 20,473 hourly data and is imbalanced with 19945(97%)‘non-failure’and 528(3%)‘failure data’.The three independent variables to predict machine failure were pressure indicator,flow indicator,and level indicator.The accuracy of the classifiers is very high and close to 100%,but the sensitivity of all classifiers using the original dataset was close to zero.The performance of the three classifiers was then evaluated for data with different imbalance rates(10%to 50%)generated from the original data using SMOTE,SMOTE-Support Vector Machine(SMOTE-SVM)and SMOTE-Edited Nearest Neighbour(SMOTE-ENN).The classifiers were evaluated based on improvement in sensitivity and F-measure.Results showed that the sensitivity of all classifiers increases as the imbalance rate increases.SVM with radial basis function(RBF)kernel has the highest sensitivity when data is balanced(50:50)using SMOTE(Sensitivitytest=0.5686,Ftest=0.6927)compared to Naïve Bayes(Sensitivitytest=0.4033,Ftest=0.6218)and Logistic Regression(Sensitivitytest=0.4194,Ftest=0.621).Overall,the Gaussian Naïve Bayes model consistently improves sensitivity and F-measure as the imbalance ratio increases,but the sensitivity is below 50%.The classifiers performed better when data was balanced using SMOTE-SVM compared to SMOTE and SMOTE-ENN.

Imbalanced Data Classification Using SVM Based on Improved Simulated Annealing Featuring Synthetic Data Generation and Reduction

Imbalanced data classification is one of the major problems in machine learning.This imbalanced dataset typically has significant differences in the number of data samples between its classes.In most cases,the performance of the machine learning algorithm such as Support Vector Machine(SVM)is affected when dealing with an imbalanced dataset.The classification accuracy is mostly skewed toward the majority class and poor results are exhibited in the prediction of minority-class samples.In this paper,a hybrid approach combining data pre-processing technique andSVMalgorithm based on improved Simulated Annealing(SA)was proposed.Firstly,the data preprocessing technique which primarily aims at solving the resampling strategy of handling imbalanced datasets was proposed.In this technique,the data were first synthetically generated to equalize the number of samples between classes and followed by a reduction step to remove redundancy and duplicated data.Next is the training of a balanced dataset using SVM.Since this algorithm requires an iterative process to search for the best penalty parameter during training,an improved SA algorithm was proposed for this task.In this proposed improvement,a new acceptance criterion for the solution to be accepted in the SA algorithm was introduced to enhance the accuracy of the optimization process.Experimental works based on ten publicly available imbalanced datasets have demonstrated higher accuracy in the classification tasks using the proposed approach in comparison with the conventional implementation of SVM.Registering at an average of 89.65%of accuracy for the binary class classification has demonstrated the good performance of the proposed works.

Fault Diagnosis of Power Transformer Based on Improved ACGAN Under Imbalanced Data

The imbalance of dissolved gas analysis(DGA)data will lead to over-fitting,weak generalization and poor recognition performance for fault diagnosis models based on deep learning.To handle this problem,a novel transformer fault diagnosis method based on improved auxiliary classifier generative adversarial network(ACGAN)under imbalanced data is proposed in this paper,which meets both the requirements of balancing DGA data and supplying accurate diagnosis results.The generator combines one-dimensional convolutional neural networks(1D-CNN)and long short-term memories(LSTM),which can deeply extract the features from DGA samples and be greatly beneficial to ACGAN’s data balancing and fault diagnosis.The discriminator adopts multilayer perceptron networks(MLP),which prevents the discriminator from losing important features of DGA data when the network is too complex and the number of layers is too large.The experimental results suggest that the presented approach can effectively improve the adverse effects of DGA data imbalance on the deep learning models,enhance fault diagnosis performance and supply desirable diagnosis accuracy up to 99.46%.Furthermore,the comparison results indicate the fault diagnosis performance of the proposed approach is superior to that of other conventional methods.Therefore,the method presented in this study has excellent and reliable fault diagnosis performance for various unbalanced datasets.In addition,the proposed approach can also solve the problems of insufficient and imbalanced fault data in other practical application fields.

Over-sampling algorithm for imbalanced data classification

For imbalanced datasets, the focus of classification is to identify samples of the minority class. The performance of current data mining algorithms is not good enough for processing imbalanced datasets. The synthetic minority over-sampling technique(SMOTE) is specifically designed for learning from imbalanced datasets, generating synthetic minority class examples by interpolating between minority class examples nearby. However, the SMOTE encounters the overgeneralization problem. The densitybased spatial clustering of applications with noise(DBSCAN) is not rigorous when dealing with the samples near the borderline.We optimize the DBSCAN algorithm for this problem to make clustering more reasonable. This paper integrates the optimized DBSCAN and SMOTE, and proposes a density-based synthetic minority over-sampling technique(DSMOTE). First, the optimized DBSCAN is used to divide the samples of the minority class into three groups, including core samples, borderline samples and noise samples, and then the noise samples of minority class is removed to synthesize more effective samples. In order to make full use of the information of core samples and borderline samples,different strategies are used to over-sample core samples and borderline samples. Experiments show that DSMOTE can achieve better results compared with SMOTE and Borderline-SMOTE in terms of precision, recall and F-value.

Fault diagnosis of HVAC system with imbalanced data using multi-scale convolution composite neural network

Accurate fault diagnosis of heating,ventilation,and air conditioning(HVAC)systems is of significant importance for maintaining normal operation,reducing energy consumption,and minimizing maintenance costs.However,in practical applications,it is challenging to obtain sufficient fault data for HVAC systems,leading to imbalanced data,where the number of fault samples is much smaller than that of normal samples.Moreover,most existing HVAC system fault diagnosis methods heavily rely on balanced training sets to achieve high fault diagnosis accuracy.Therefore,to address this issue,a composite neural network fault diagnosis model is proposed,which combines SMOTETomek,multi-scale one-dimensional convolutional neural networks(M1DCNN),and support vector machine(SVM).This method first utilizes SMOTETomek to augment the minority class samples in the imbalanced dataset,achieving a balanced number of faulty and normal data.Then,it employs the M1DCNN model to extract feature information from the augmented dataset.Finally,it replaces the original Softmax classifier with an SVM classifier for classification,thus enhancing the fault diagnosis accuracy.Using the SMOTETomek-M1DCNN-SVM method,we conducted fault diagnosis validation on both the ASHRAE RP-1043 dataset and experimental dataset with an imbalance ratio of 1:10.The results demonstrate the superiority of this approach,providing a novel and promising solution for intelligent building management,with accuracy and F1 scores of 98.45%and 100%for the RP-1043 dataset and experimental dataset,respectively.

Constraint Learning-based Optimal Power Dispatch for Active Distribution Networks with Extremely Imbalanced Data

Transition towards carbon-neutral power systems has necessitated optimization of power dispatch in active distribution networks(ADNs)to facilitate integration of distributed renewable generation.Due to unavailability of network topology and line impedance in many distribution networks,physical model-based methods may not be applicable to their operations.To tackle this challenge,some studies have proposed constraint learning,which replicates physical models by training a neural network to evaluate feasibility of a decision(i.e.,whether a decision satisfies all critical constraints or not).To ensure accuracy of this trained neural network,training set should contain sufficient feasible and infeasible samples.However,since ADNs are mostly operated in a normal status,only very few historical samples are infeasible.Thus,the historical dataset is highly imbalanced,which poses a significant obstacle to neural network training.To address this issue,we propose an enhanced constraint learning method.First,it leverages constraint learning to train a neural network as surrogate of ADN's model.Then,it introduces Synthetic Minority Oversampling Technique to generate infeasible samples to mitigate imbalance of historical dataset.By incorporating historical and synthetic samples into the training set,we can significantly improve accuracy of neural network.Furthermore,we establish a trust region to constrain and thereafter enhance reliability of the solution.Simulations confirm the benefits of the proposed method in achieving desirable optimality and feasibility while maintaining low computational complexity.

Conditional self-attention generative adversarial network with differential evolution algorithm for imbalanced data classification

Imbalanced data classification is an important research topic in real-world applications,like fault diagnosis in an aircraft manufacturing system.The over-sampling method is often used to solve this problem.It generates samples according to the distance between minority data.However,the traditional over-sampling method may change the original data distribution,which is harmful to the classification performance.In this paper,we propose a new method called Conditional SelfAttention Generative Adversarial Network with Differential Evolution(CSAGAN-DE)for imbalanced data classification.The new method aims at improving the classification performance of minority data by enhancing the quality of the generation of minority data.In CSAGAN-DE,the minority data are fed into the self-attention generative adversarial network to approximate the data distribution and create new data for the minority class.Then,the differential evolution algorithm is employed to automatically determine the number of generated minority data for achieving a satisfactory classification performance.Several experiments are conducted to evaluate the performance of the new CSAGAN-DE method.The results show that the new method can efficiently improve the classification performance compared with other related methods.

GraphCWGAN-GP:A Novel Data Augmenting Approach for Imbalanced Encrypted Traffic Classification

Encrypted traffic classification has become a hot issue in network security research.The class imbalance problem of traffic samples often causes the deterioration of Machine Learning based classifier performance.Although the Generative Adversarial Network(GAN)method can generate new samples by learning the feature distribution of the original samples,it is confronted with the problems of unstable training andmode collapse.To this end,a novel data augmenting approach called Graph CWGAN-GP is proposed in this paper.The traffic data is first converted into grayscale images as the input for the proposed model.Then,the minority class data is augmented with our proposed model,which is built by introducing conditional constraints and a new distance metric in typical GAN.Finally,the classical deep learning model is adopted as a classifier to classify datasets augmented by the Condition GAN(CGAN),Wasserstein GAN-Gradient Penalty(WGAN-GP)and Graph CWGAN-GP,respectively.Compared with the state-of-the-art GAN methods,the Graph CWGAN-GP cannot only control the modes of the data to be generated,but also overcome the problem of unstable training and generate more realistic and diverse samples.The experimental results show that the classification precision,recall and F1-Score of theminority class in the balanced dataset augmented in this paper have improved by more than 2.37%,3.39% and 4.57%,respectively.

An Effective Classifier Model for Imbalanced Network Attack Data

Recently,machine learning algorithms have been used in the detection and classification of network attacks.The performance of the algorithms has been evaluated by using benchmark network intrusion datasets such as DARPA98,KDD’99,NSL-KDD,UNSW-NB15,and Caida DDoS.However,these datasets have two major challenges:imbalanced data and highdimensional data.Obtaining high accuracy for all attack types in the dataset allows for high accuracy in imbalanced datasets.On the other hand,having a large number of features increases the runtime load on the algorithms.A novel model is proposed in this paper to overcome these two concerns.The number of features in the model,which has been tested at CICIDS2017,is initially optimized by using genetic algorithms.This optimum feature set has been used to classify network attacks with six well-known classifiers according to high f1-score and g-mean value in minimumtime.Afterwards,amulti-layer perceptron based ensemble learning approach has been applied to improve the models’overall performance.The experimental results showthat the suggested model is acceptable for feature selection as well as classifying network attacks in an imbalanced dataset,with a high f1-score(0.91)and g-mean(0.99)value.Furthermore,it has outperformed base classifier models and voting procedures.

A Rebalancing Framework for Classification of Imbalanced Medical Appointment No-show Data

Purpose: This paper aims to improve the classification performance when the data is imbalanced by applying different sampling techniques available in Machine Learning.Design/methodology/approach: The medical appointment no-show dataset is imbalanced, and when classification algorithms are applied directly to the dataset, it is biased towards the majority class, ignoring the minority class. To avoid this issue, multiple sampling techniques such as Random Over Sampling(ROS), Random Under Sampling(RUS), Synthetic Minority Oversampling TEchnique(SMOTE), ADAptive SYNthetic Sampling(ADASYN), Edited Nearest Neighbor(ENN), and Condensed Nearest Neighbor(CNN) are applied in order to make the dataset balanced. The performance is assessed by the Decision Tree classifier with the listed sampling techniques and the best performance is identified.Findings: This study focuses on the comparison of the performance metrics of various sampling methods widely used. It is revealed that, compared to other techniques, the Recall is high when ENN is applied CNN and ADASYN have performed equally well on the Imbalanced data.Research limitations: The testing was carried out with limited dataset and needs to be tested with a larger dataset.Practical implications: This framework will be useful whenever the data is imbalanced in real world scenarios, which ultimately improves the performance.Originality/value: This paper uses the rebalancing framework on medical appointment no-show dataset to predict the no-shows and removes the bias towards minority class.

A Modified Generative Adversarial Network for Fault Diagnosis in High-Speed Train Components with Imbalanced and Heterogeneous Monitoring Data

Data-driven methods are widely considered for fault diagnosis in complex systems.However,in practice,the between-class imbalance due to limited faulty samples may deteriorate their classification performance.To address this issue,synthetic minority methods for enhancing data have been proved to be effective in many applications.Generative adversarial networks(GANs),capable of automatic features extraction,can also be adopted for augmenting the faulty samples.However,the monitoring data of a complex system may include not only continuous signals but also discrete/categorical signals.Since the current GAN methods still have some challenges in handling such heterogeneous monitoring data,a Mixed Dual Discriminator GAN(noted as M-D2GAN)is proposed in this work.In order to render the expanded fault samples more aligned with the real situation and improve the accuracy and robustness of the fault diagnosis model,different types of variables are generated in different ways,including floating-point,integer,categorical,and hierarchical.For effectively considering the class imbalance problem,proper modifications are made to the GAN model,where a normal class discriminator is added.A practical case study concerning the braking system of a high-speed train is carried out to verify the effectiveness of the proposed framework.Compared to the classic GAN,the proposed framework achieves better results with respect to F-measure and G-mean metrics.

A Stacked Ensemble Deep Learning Approach for Imbalanced Multi-Class Water Quality Index Prediction

A common difficulty in building prediction models with real-world environmental datasets is the skewed distribution of classes.There are significantly more samples for day-to-day classes,while rare events such as polluted classes are uncommon.Consequently,the limited availability of minority outcomes lowers the classifier’s overall reliability.This study assesses the capability of machine learning(ML)algorithms in tackling imbalanced water quality data based on the metrics of precision,recall,and F1 score.It intends to balance the misled accuracy towards the majority of data.Hence,10 ML algorithms of its performance are compared.The classifiers included are AdaBoost,SupportVector Machine,Linear Discriminant Analysis,k-Nearest Neighbors,Naive Bayes,Decision Trees,Random Forest,Extra Trees,Bagging,and the Multilayer Perceptron.This study also uses the Easy Ensemble Classifier,Balanced Bagging,andRUSBoost algorithm to evaluatemulti-class imbalanced learning methods.The comparison results revealed that a highaccuracy machine learning model is not always good in recall and sensitivity.This paper’s stacked ensemble deep learning(SE-DL)generalization model effectively classifies the water quality index(WQI)based on 23 input variables.The proposed algorithm achieved a remarkable average of 95.69%,94.96%,92.92%,and 93.88%for accuracy,precision,recall,and F1 score,respectively.In addition,the proposed model is compared against two state-of-the-art classifiers,the XGBoost(eXtreme Gradient Boosting)and Light Gradient Boosting Machine,where performance metrics of balanced accuracy and g-mean are included.The experimental setup concluded XGBoost with a higher balanced accuracy and G-mean.However,the SE-DL model has a better and more balanced performance in the F1 score.The SE-DL model aligns with the goal of this study to ensure the balance between accuracy and completeness for each water quality class.The proposed algorithm is also capable of higher efficiency at a lower computational time against using the standard SyntheticMinority Oversampling Technique(SMOTE)approach to imbalanced datasets.

A Machine Learning-Based Web Application for Heart Disease Prediction

This work leveraged predictive modeling techniques in machine learning (ML) to predict heart disease using a dataset sourced from the Center for Disease Control and Prevention in the US. The dataset was preprocessed and used to train five machine learning models: random forest, support vector machine, logistic regression, extreme gradient boosting and light gradient boosting. The goal was to use the best performing model to develop a web application capable of reliably predicting heart disease based on user-provided data. The extreme gradient boosting classifier provided the most reliable results with precision, recall and F1-score of 97%, 72%, and 83% respectively for Class 0 (no heart disease) and 21% (precision), 81% (recall) and 34% (F1-score) for Class 1 (heart disease). The model was further deployed as a web application.

Imbalanced Classification in Diabetics Using Ensembled Machine Learning

Diabetics is one of the world’s most common diseases which are caused by continued high levels of blood sugar.The risk of diabetics can be lowered if the diabetic is found at the early stage.In recent days,several machine learning models were developed to predict the diabetic presence at an early stage.In this paper,we propose an embedded-based machine learning model that combines the split-vote method and instance duplication to leverage an imbalanced dataset called PIMA Indian to increase the prediction of diabetics.The proposed method uses both the concept of over-sampling and under-sampling along with model weighting to increase the performance of classification.Different measures such as Accuracy,Precision,Recall,and F1-Score are used to evaluate the model.The results we obtained using K-Nearest Neighbor(kNN),Naïve Bayes(NB),Support Vector Machines(SVM),Random Forest(RF),Logistic Regression(LR),and Decision Trees(DT)were 89.32%,91.44%,95.78%,89.3%,81.76%,and 80.38%respectively.The SVM model is more efficient than other models which are 21.38%more than exiting machine learning-based works.

A Rasterized Lightning Disaster Risk Method for Imbalanced Sets Using Neural

Over the past 10 years,lightning disaster has caused a large number of casualties and considerable economic loss worldwide.Lightning poses a huge threat to various industries.In an attempt to reduce the risk of lightning-caused disaster,many scholars have carried out in-depth research on lightning.However,these studies focus primarily on the lightning itself and other meteorological elements are ignored.In addition,the methods for assessing the risk of lightning disaster fail to give detailed attention to regional features(lightning disaster risk).This paper proposes a grid-based risk assessment method based on data from multiple sources.First,this paper considers the impact of lightning,the population density,the economy,and geographical environment data on the occurrence of lightning disasters;Second,this paper solves the problem of imbalanced lightning disaster data in geographic grid samples based on the K-means clustering algorithm;Third,the method calculates the feature of lightning disaster in each small field with the help of neural network structure,and the calculation results are then visually reflected in a zoning map by the Jenks natural breaks algorithm.The experimental results show that our method can solve the problem of imbalanced lightning disaster data,and offer 81%accuracy in lightning disaster risk assessment.

Fault Diagnosis Method Based on Xgboost and LR Fusion Model under Data Imbalance

Diagnosis methods based on machine learning and deep learning are widely used in the field of motor fault diagnosis.However,due to the fact that the data imbalance caused by the high cost of obtaining fault data will lead to insufficient generalization performance of the diagnosis method.In response to this problem,a motor fault monitoring system is proposed,which includes a fault diagnosis method(Xgb_LR)based on the optimized gradient boosting decision tree(Xgboost)and logistic regression(LR)fusion model and a data augmentation method named data simulation neighborhood interpolation(DSNI).The Xgb_LR method combines the advantages of the two models and has positive adaptability to imbalanced data.Simultaneously,the DSNI method can be used as an auxiliary method of the diagnosis method to reduce the impact of data imbalance by expanding the original data(signal).Simulation experiments verify the effectiveness of the proposed methods.

LexDeep:Hybrid Lexicon and Deep Learning Sentiment Analysis Using Twitter for Unemployment-Related Discussions During COVID-19

The COVID-19 pandemic has spread globally,resulting in financialinstability in many countries and reductions in the per capita grossdomestic product.Sentiment analysis is a cost-effective method for acquiringsentiments based on household income loss,as expressed on social media.However,limited research has been conducted in this domain using theLexDeep approach.This study aimed to explore social trend analytics usingLexDeep,which is a hybrid sentiment analysis technique,on Twitter to capturethe risk of household income loss during the COVID-19 pandemic.First,tweet data were collected using Twint with relevant keywords before(9 March2019 to 17 March 2020)and during(18 March 2020 to 21 August 2021)thepandemic.Subsequently,the tweets were annotated using VADER(lexiconbased)and fed into deep learning classifiers,and experiments were conductedusing several embeddings,namely simple embedding,Global Vectors,andWord2Vec,to classify the sentiments expressed in the tweets.The performanceof each LexDeep model was evaluated and compared with that of a supportvector machine(SVM).Finally,the unemployment rates before and duringCOVID-19 were analysed to gain insights into the differences in unemploymentpercentages through social media input and analysis.The resultsdemonstrated that all LexDeep models with simple embedding outperformedthe SVM.This confirmed the superiority of the proposed LexDeep modelover a classical machine learning classifier in performing sentiment analysistasks for domain-specific sentiments.In terms of the risk of income loss,the unemployment issue is highly politicised on both the regional and globalscales;thus,if a country cannot combat this issue,the global economy will alsobe affected.Future research should develop a utility maximisation algorithmfor household welfare evaluation,given the percentage risk of income lossowing to COVID-19.

Supervised Learning Algorithm on Unstructured Documents for the Classification of Job Offers: Case of Cameroun

Nowadays, in data science, supervised learning algorithms are frequently used to perform text classification. However, African textual data, in general, have been studied very little using these methods. This article notes the particularity of the data and measures the level of precision of predictions of naive Bayes algorithms, decision tree, and SVM (Support Vector Machine) on a corpus of computer jobs taken on the internet. This is due to the data imbalance problem in machine learning. However, this problem essentially focuses on the distribution of the number of documents in each class or subclass. Here, we delve deeper into the problem to the word count distribution in a set of documents. The results are compared with those obtained on a set of French IT offers. It appears that the precision of the classification varies between 88% and 90% for French offers against 67%, at most, for Cameroonian offers. The contribution of this study is twofold. Indeed, it clearly shows that, in a similar job category, job offers on the internet in Cameroon are more unstructured compared to those available in France, for example. Moreover, it makes it possible to emit a strong hypothesis according to which sets of texts having a symmetrical distribution of the number of words obtain better results with supervised learning algorithms.

Joint Sample Position Based Noise Filtering and Mean Shift Clustering for Imbalanced Classification Learning

The problem of imbalanced data classification learning has received much attention.Conventional classification algorithms are susceptible to data skew to favor majority samples and ignore minority samples.Majority weighted minority oversampling technique(MWMOTE)is an effective approach to solve this problem,however,it may suffer from the shortcomings of inadequate noise filtering and synthesizing the same samples as the original minority data.To this end,we propose an improved MWMOTE method named joint sample position based noise filtering and mean shift clustering(SPMSC)to solve these problems.Firstly,in order to effectively eliminate the effect of noisy samples,SPMsC uses a new noise filtering mechanism to determine whether a minority sample is noisy or not based on its position and distribution relative to the majority sample.Note that MWMOTE may generate duplicate samples,we then employ the mean shift algorithm to cluster minority samples to reduce synthetic replicate samples.Finally,data cleaning is performed on the processed data to further eliminate class overlap.Experiments on extensive benchmark datasets demonstrate the effectiveness of SPMsC compared with other sampling methods.

Credit Card Fraud Detection Using Weighted Support Vector Machine

Credit card fraudulent data is highly imbalanced, and it has presented an overwhelmingly large portion of nonfraudulent transactions and a small portion of fraudulent transactions. The measures used to judge the veracity of the detection algorithms become critical to the deployment of a model that accurately scores fraudulent transactions taking into account case imbalance, and the cost of identifying a case as genuine when, in fact, the case is a fraudulent transaction. In this paper, a new criterion to judge classification algorithms, which considers the cost of misclassification, is proposed, and several undersampling techniques are compared by this new criterion. At the same time, a weighted support vector machine (SVM) algorithm considering the financial cost of misclassification is introduced, proving to be more practical for credit card fraud detection than traditional methodologies. This weighted SVM uses transaction balances as weights for fraudulent transactions, and a uniformed weight for nonfraudulent transactions. The results show this strategy greatly improve performance of credit card fraud detection.