A Hierarchical Clustering and Fixed-Layer Local Learning Based Support Vector Machine Algorithm for Large Scale Classification Problems

It is a challenging topic to develop an efficient algorithm for large scale classification problems in many applications of machine learning. In this paper, a hierarchical clustering and fixed-layer local learning (HCFLL) based support vector machine(SVM) algorithm is proposed to deal with this problem. Firstly, HCFLL hierarchically clusters a given dataset into a modified clustering feature tree based on the ideas of unsupervised clustering and supervised clustering. Then it locally trains SVM on each labeled subtree at a fixed-layer of the tree. The experimental results show that compared with the existing popular algorithms such as core vector machine and decision-tree support vector machine, HCFLL can significantly improve the training and testing speeds with comparable testing accuracy.

Turbopump Condition Monitoring Using Incremental Clustering and One-class Support Vector Machine

Turbopump condition monitoring is a significant approach to ensure the safety of liquid rocket engine （LRE）.Because of lack of fault samples,a monitoring system cannot be trained on all possible condition patterns.Thus it is important to differentiate abnormal or unknown patterns from normal pattern with novelty detection methods.One-class support vector machine （OCSVM） that has been commonly used for novelty detection cannot deal well with large scale samples.In order to model the normal pattern of the turbopump with OCSVM and so as to monitor the condition of the turbopump,a monitoring method that integrates OCSVM with incremental clustering is presented.In this method,the incremental clustering is used for sample reduction by extracting representative vectors from a large training set.The representative vectors are supposed to distribute uniformly in the object region and fulfill the region.And training OCSVM on these representative vectors yields a novelty detector.By applying this method to the analysis of the turbopump＇s historical test data,it shows that the incremental clustering algorithm can extract 91 representative points from more than 36 000 training vectors,and the OCSVM detector trained on these 91 representative points can recognize spikes in vibration signals caused by different abnormal events such as vane shedding,rub-impact and sensor faults.This monitoring method does not need fault samples during training as classical recognition methods.The method resolves the learning problem of large samples and is an alternative method for condition monitoring of the LRE turbopump.

A Kernel Clustering Algorithm for Fast Training of Support Vector Machines

A new algorithm named kernel bisecting k-means and sample removal(KBK-SR) is proposed as sampling preprocessing for support vector machine(SVM) training to improve the efficiency.The proposed algorithm tends to quickly produce balanced clusters of similar sizes in the kernel feature space,which makes it efficient and effective for reducing training samples.Theoretical analysis and experimental results on three UCI real data benchmarks both show that,with very short sampling time,the proposed algorithm dramatically accelerates SVM sampling and training while maintaining high test accuracy.

Fault Diagnosis Model Based on Fuzzy Support Vector Machine Combined with Weighted Fuzzy Clustering

A fault diagnosis model is proposed based on fuzzy support vector machine (FSVM) combined with fuzzy clustering (FC).Considering the relationship between the sample point and non-self class,FC algorithm is applied to generate fuzzy memberships.In the algorithm,sample weights based on a distribution density function of data point and genetic algorithm (GA) are introduced to enhance the performance of FC.Then a multi-class FSVM with radial basis function kernel is established according to directed acyclic graph algorithm,the penalty factor and kernel parameter of which are optimized by GA.Finally,the model is executed for multi-class fault diagnosis of rolling element bearings.The results show that the presented model achieves high performances both in identifying fault types and fault degrees.The performance comparisons of the presented model with SVM and distance-based FSVM for noisy case demonstrate the capacity of dealing with noise and generalization.

Performance Analysis of Support Vector Machine (SVM) on Challenging Datasets for Forest Fire Detection

This article delves into the analysis of performance and utilization of Support Vector Machines (SVMs) for the critical task of forest fire detection using image datasets. With the increasing threat of forest fires to ecosystems and human settlements, the need for rapid and accurate detection systems is of utmost importance. SVMs, renowned for their strong classification capabilities, exhibit proficiency in recognizing patterns associated with fire within images. By training on labeled data, SVMs acquire the ability to identify distinctive attributes associated with fire, such as flames, smoke, or alterations in the visual characteristics of the forest area. The document thoroughly examines the use of SVMs, covering crucial elements like data preprocessing, feature extraction, and model training. It rigorously evaluates parameters such as accuracy, efficiency, and practical applicability. The knowledge gained from this study aids in the development of efficient forest fire detection systems, enabling prompt responses and improving disaster management. Moreover, the correlation between SVM accuracy and the difficulties presented by high-dimensional datasets is carefully investigated, demonstrated through a revealing case study. The relationship between accuracy scores and the different resolutions used for resizing the training datasets has also been discussed in this article. These comprehensive studies result in a definitive overview of the difficulties faced and the potential sectors requiring further improvement and focus.

Enhanced Nature Inspired-Support Vector Machine for Glaucoma Detection

Glaucoma is a progressive eye disease that can lead to blindness if left untreated.Early detection is crucial to prevent vision loss,but current manual scanning methods are expensive,time-consuming,and require specialized expertise.This study presents a novel approach to Glaucoma detection using the Enhanced Grey Wolf Optimized Support Vector Machine(EGWO-SVM)method.The proposed method involves preprocessing steps such as removing image noise using the adaptive median filter(AMF)and feature extraction using the previously processed speeded-up robust feature(SURF),histogram of oriented gradients(HOG),and Global features.The enhanced Grey Wolf Optimization(GWO)technique is then employed with SVM for classification.To evaluate the proposed method,we used the online retinal images for glaucoma analysis(ORIGA)database,and it achieved high accuracy,sensitivity,and specificity rates of 94%,92%,and 92%,respectively.The results demonstrate that the proposed method outperforms other current algorithms in detecting the presence or absence of Glaucoma.This study provides a novel and effective approach to Glaucoma detection that can potentially improve the detection process and outcomes.

SCADA Data-Based Support Vector Machine for False Alarm Identification for Wind Turbine Management

Maintenance operations have a critical influence on power gen-eration by wind turbines(WT).Advanced algorithms must analyze large volume of data from condition monitoring systems(CMS)to determine the actual working conditions and avoid false alarms.This paper proposes different support vector machine(SVM)algorithms for the prediction and detection of false alarms.K-Fold cross-validation(CV)is applied to evaluate the classification reliability of these algorithms.Supervisory Control and Data Acquisition(SCADA)data from an operating WT are applied to test the proposed approach.The results from the quadratic SVM showed an accuracy rate of 98.6%.Misclassifications from the confusion matrix,alarm log and maintenance records are analyzed to obtain quantitative information and determine if it is a false alarm.The classifier reduces the number of false alarms called misclassifications by 25%.These results demonstrate that the proposed approach presents high reliability and accuracy in false alarm identification.

Decision tree support vector machine based on genetic algorithm for multi-class classification

To solve the multi-class fault diagnosis tasks, decision tree support vector machine （DTSVM）, which combines SVM and decision tree using the concept of dichotomy, is proposed. Since the classification performance of DTSVM highly depends on its structure, to cluster the multi-classes with maximum distance between the clustering centers of the two sub-classes, genetic algorithm is introduced into the formation of decision tree, so that the most separable classes would be separated at each node of decisions tree. Numerical simulations conducted on three datasets compared with ＂one-against-all＂ and ＂one-against-one＂ demonstrate the proposed method has better performance and higher generalization ability than the two conventional methods.

Approximate entropy and support vector machines for electroencephalogram signal classification

The automatic detection and identification of electroencephalogram waves play an important role in the prediction, diagnosis and treatment of epileptic seizures. In this study, a nonlinear dynamics index–approximate entropy and a support vector machine that has strong generalization ability were applied to classify electroencephalogram signals at epileptic interictal and ictal periods. Our aim was to verify whether approximate entropy waves can be effectively applied to the automatic real-time detection of epilepsy in the electroencephalogram, and to explore its generalization ability as a classifier trained using a nonlinear dynamics index. Four patients presenting with partial epileptic seizures were included in this study. They were all diagnosed with neocortex localized epilepsy and epileptic foci were clearly observed by electroencephalogram. The electroencephalogram data form the four involved patients were segmented and the characteristic values of each segment, that is, the approximate entropy, were extracted. The support vector machine classifier was constructed with the approximate entropy extracted from one epileptic case, and then electroencephalogram waves of the other three cases were classified, reaching a 93.33% accuracy rate. Our findings suggest that the use of approximate entropy allows the automatic real-time detection of electroencephalogram data in epileptic cases. The combination of approximate entropy and support vector machines shows good generalization ability for the classification of electroencephalogram signals for epilepsy.

Classification using wavelet packet decomposition and support vector machine for digital modulations

To make the modulation classification system more suitable for signals in a wide range of signal to noise rate （SNR）, a feature extraction method based on signal wavelet packet transform modulus maxima matrix （WPTMMM） and a novel support vector machine fuzzy network （SVMFN） classifier is presented. The WPTMMM feature extraction method has less computational complexity, more stability, and has the preferable advantage of robust with the time parallel moving and white noise. Further, the SVMFN uses a new definition of fuzzy density that incorporates accuracy and uncertainty of the classifiers to improve recognition reliability to classify nine digital modulation types （i.e. 2ASK, 2FSK, 2PSK, 4ASK, 4FSK, 4PSK, 16QAM, MSK, and OQPSK）. Computer simulation shows that the proposed scheme has the advantages of high accuracy and reliability （success rates are over 98% when SNR is not lower than 0dB）, and it adapts to engineering applications.

Weighted Proximal Support Vector Machines: Robust Classification

Despite of its great efficiency for pattern classification, proximal supportvector machines (PSVM), a new version of SVM proposed recently, is sensitive to noise and outliers.To overcome the drawback, this paper modifies PSVM by associating a weightvalue with each input dataof PSVM. The distance between each data point and the center of corresponding class is used tocalculate the weight value. In this way, the effect of noise is reduced. The experiments indicatethat new SVM, weighted proximal support vector machine (WPSVM), is much more robust to noise thanPSVM without loss of computationally attractive feature of PSVM.

Human Behavior Classification Using Geometrical Features of Skeleton and Support Vector Machines

Classification of human actions under video surveillance is gaining a lot of attention from computer vision researchers.In this paper,we have presented methodology to recognize human behavior in thin crowd which may be very helpful in surveillance.Research have mostly focused the problem of human detection in thin crowd,overall behavior of the crowd and actions of individuals in video sequences.Vision based Human behavior modeling is a complex task as it involves human detection,tracking,classifying normal and abnormal behavior.The proposed methodology takes input video and applies Gaussian based segmentation technique followed by post processing through presenting hole filling algorithm i.e.,fill hole inside objects algorithm.Human detection is performed by presenting human detection algorithm and then geometrical features from human skeleton are extracted using feature extraction algorithm.The classification task is achieved using binary and multi class support vector machines.The proposed technique is validated through accuracy,precision,recall and F-measure metrics.

Basic Tenets of Classification Algorithms K-Nearest-Neighbor, Support Vector Machine, Random Forest and Neural Network: A Review

In this paper, sixty-eight research articles published between 2000 and 2017 as well as textbooks which employed four classification algorithms: K-Nearest-Neighbor (KNN), Support Vector Machines (SVM), Random Forest (RF) and Neural Network (NN) as the main statistical tools were reviewed. The aim was to examine and compare these nonparametric classification methods on the following attributes: robustness to training data, sensitivity to changes, data fitting, stability, ability to handle large data sizes, sensitivity to noise, time invested in parameter tuning, and accuracy. The performances, strengths and shortcomings of each of the algorithms were examined, and finally, a conclusion was arrived at on which one has higher performance. It was evident from the literature reviewed that RF is too sensitive to small changes in the training dataset and is occasionally unstable and tends to overfit in the model. KNN is easy to implement and understand but has a major drawback of becoming significantly slow as the size of the data in use grows, while the ideal value of K for the KNN classifier is difficult to set. SVM and RF are insensitive to noise or overtraining, which shows their ability in dealing with unbalanced data. Larger input datasets will lengthen classification times for NN and KNN more than for SVM and RF. Among these nonparametric classification methods, NN has the potential to become a more widely used classification algorithm, but because of their time-consuming parameter tuning procedure, high level of complexity in computational processing, the numerous types of NN architectures to choose from and the high number of algorithms used for training, most researchers recommend SVM and RF as easier and wieldy used methods which repeatedly achieve results with high accuracies and are often faster to implement.

Mobile SMS Spam Filtering for Nepali Text Using Naive Bayesian and Support Vector Machine

Spam is a universal problem with which everyone is familiar. A number of approaches are used for Spam filtering. The most common filtering technique is content-based filtering which uses the actual text of message to determine whether it is Spam or not. The content is very dynamic and it is very challenging to represent all information in a mathematical model of classification. For instance, in content-based Spam filtering, the characteristics used by the filter to identify Spam message are constantly changing over time. Na?ve Bayes method represents the changing nature of message using probability theory and support vector machine (SVM) represents those using different features. These two methods of classification are efficient in different domains and the case of Nepali SMS or Text classification has not yet been in consideration;these two methods do not consider the issue and it is interesting to find out the performance of both the methods in the problem of Nepali Text classification. In this paper, the Na?ve Bayes and SVM-based classification techniques are implemented to classify the Nepali SMS as Spam and non-Spam. An empirical analysis for various text cases has been done to evaluate accuracy measure of the classification methodologies used in this study. And, it is found to be 87.15% accurate in SVM and 92.74% accurate in the case of Na?ve Bayes.

Classification using least squares support vector machine for reliability analysis

In order to improve the efficiency of the support vector machine （SVM） for classification to deal with a large amount of samples, the least squares support vector machine （LSSVM） for classification methods is introduced into the reliability analysis. To reduce the coraputational cost, the solution of the SVM is transformed from a quadratic programming to a group of linear equations. The numerical results indicate that the reliability method based on the LSSVM for classification has higher accuracy and requires less computational cost than the SVM method.

Multiple mental tasks classification based on nonlinear parameter of mean period using support vector machines

Mental task classification is one of the most important problems in Brain-computer interface.This paper studies the classification of five-class mental tasks.The nonlinear parameter of mean period obtained from frequency domain information was used as features for classification implemented by using the method of SVM(support vector machines).The averaged classification accuracy of 85.6% over 7 subjects was achieved for 2-second EEG segments.And the results for EEG segments of 0.5s and 5.0s compared favorably to those of Garrett's.The results indicate that the parameter of mean period represents mental tasks well for classification.Furthermore,the method of mean period is less computationally demanding,which indicates its potential use for online BCI systems.

Animal Classification System Based on Image Processing &Support Vector Machine

This project is mainly focused to develop system for animal researchers & wild life photographers to overcome so many challenges in their day life today. When they engage in such situation, they need to be patiently waiting for long hours, maybe several days in whatever location and under severe weather conditions until capturing what they are interested in. Also there is a big demand for rare wild life photo graphs. The proposed method makes the task automatically use microcontroller controlled camera, image processing and machine learning techniques. First with the aid of microcontroller and four passive IR sensors system will automatically detect the presence of animal and rotate the camera toward that direction. Then the motion detection algorithm will get the animal into middle of the frame and capture by high end auto focus web cam. Then the captured images send to the PC and are compared with photograph database to check whether the animal is exactly the same as the photographer choice. If that captured animal is the exactly one who need to capture then it will automatically capture more. Though there are several technologies available none of these are capable of recognizing what it captures. There is no detection of animal presence in different angles. Most of available equipment uses a set of PIR sensors and whatever it disturbs the IR field will automatically be captured and stored. Night time images are black and white and have less details and clarity due to infrared flash quality. If the infrared flash is designed for best image quality, range will be sacrificed. The photographer might be interested in a specific animal but there is no facility to recognize automatically whether captured animal is the photographer’s choice or not.

Named Entity Recognition for Nepali Text Using Support Vector Machines

Named Entity Recognition aims to identify and to classify rigid designators in text such as proper names, biological species, and temporal expressions into some predefined categories. There has been growing interest in this field of research since the early 1990s. Named Entity Recognition has a vital role in different fields of natural language processing such as Machine Translation, Information Extraction, Question Answering System and various other fields. In this paper, Named Entity Recognition for Nepali text, based on the Support Vector Machine (SVM) is presented which is one of machine learning approaches for the classification task. A set of features are extracted from training data set. Accuracy and efficiency of SVM classifier are analyzed in three different sizes of training data set. Recognition systems are tested with ten datasets for Nepali text. The strength of this work is the efficient feature extraction and the comprehensive recognition techniques. The Support Vector Machine based Named Entity Recognition is limited to use a certain set of features and it uses a small dictionary which affects its performance. The learning performance of recognition system is observed. It is found that system can learn well from the small set of training data and increase the rate of learning on the increment of training size.

Classification of power quality combined disturbances based on phase space reconstruction and support vector machines

Power Quality (PQ) combined disturbances become common along with ubiquity of voltage flickers and harmonics. This paper presents a novel approach to classify the different patterns of PQ combined disturbances. The classification system consists of two parts, namely the feature extraction and the automatic recognition. In the feature extraction stage, Phase Space Reconstruction (PSR), a time series analysis tool, is utilized to construct disturbance signal trajectories. For these trajectories, several indices are proposed to form the feature vectors. Support Vector Machines (SVMs) are then implemented to recognize the different patterns and to evaluate the efficiencies. The types of disturbances discussed include a combination of short-term dis-turbances (voltage sags, swells) and long-term disturbances (flickers, harmonics), as well as their homologous single ones. The feasibilities of the proposed approach are verified by simulation with thousands of PQ events. Comparison studies based on Wavelet Transform (WT) and Artificial Neural Network (ANN) are also reported to show its advantages.