Improved Twin Support Vector Machine Algorithm and Applications in Classification Problems

The distribution of data has a significant impact on the results of classification.When the distribution of one class is insignificant compared to the distribution of another class,data imbalance occurs.This will result in rising outlier values and noise.Therefore,the speed and performance of classification could be greatly affected.Given the above problems,this paper starts with the motivation and mathematical representing of classification,puts forward a new classification method based on the relationship between different classification formulations.Combined with the vector characteristics of the actual problem and the choice of matrix characteristics,we firstly analyze the orderly regression to introduce slack variables to solve the constraint problem of the lone point.Then we introduce the fuzzy factors to solve the problem of the gap between the isolated points on the basis of the support vector machine.We introduce the cost control to solve the problem of sample skew.Finally,based on the bi-boundary support vector machine,a twostep weight setting twin classifier is constructed.This can help to identify multitasks with feature-selected patterns without the need for additional optimizers,which solves the problem of large-scale classification that can’t deal effectively with the very low category distribution gap.

POSITIVE DEFINITE KERNEL IN SUPPORT VECTOR MACHINE(SVM)

The relationship among Mercer kernel, reproducing kernel and positive definite kernel in support vector machine （SVM） is proved and their roles in SVM are discussed. The quadratic form of the kernel matrix is used to confirm the positive definiteness and their construction. Based on the Bochner theorem, some translation invariant kernels are checked in their Fourier domain. Some rotation invariant radial kernels are inspected according to the Schoenberg theorem. Finally, the construction of discrete scaling and wavelet kernels, the kernel selection and the kernel parameter learning are discussed.

Endpoint Prediction of EAF Based on Multiple Support Vector Machines

The endpoint parameters are very important to the process of EAF steel-making, but their on-line measurement is difficult. The soft sensor technology is widely used for the prediction of endpoint parameters. Based on the analysis of the smelting process of EAF and the advantages of support vector machines, a soft sensor model for predicting the endpoint parameters was built using multiple support vector machines （MSVM）. In this model, the input space was divided by subtractive clustering and a sub-model based on LS-SVM was built in each sub-space. To decrease the correlation among the sub-models and to improve the accuracy and robustness of the model, the sub- models were combined by Principal Components Regression. The accuracy of the soft sensor model is perfectly improved. The simulation result demonstrates the practicability and efficiency of the MSVM model for the endpoint prediction of EAF.

Nonlinear Model Predictive Control Based on Support Vector Machine with Multi-kernel

Multi-kernel-based support vector machine （SVM） model structure of nonlinear systems and its specific identification method is proposed, which is composed of a SVM with linear kernel function followed in series by a SVM with spline kernel function. With the help of this model, nonlinear model predictive control can be transformed to linear model predictive control, and consequently a unified analytical solution of optimal input of multi-step-ahead predictive control is possible to derive. This algorithm does not require online iterative optimization in order to be suitable for real-time control with less calculation. The simulation results of pH neutralization process and CSTR reactor show the effectiveness and advantages of the presented algorithm.

Incremental support vector machine algorithm based on multi-kernel learning

A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set learning problem can be solved effectively. Furthermore, different punishments are adopted in allusion to the training subset and the acquired support vectors, which may help to improve the performance of SVM. Simulation results indicate that the proposed algorithm can not only solve the model selection problem in SVM incremental learning, but also improve the classification or prediction precision.

Chaotic time series prediction using fuzzy sigmoid kernel-based support vector machines

Support vector machines （SVM） have been widely used in chaotic time series predictions in recent years. In order to enhance the prediction efficiency of this method and implement it in hardware, the sigmoid kernel in SVM is drawn in a more natural way by using the fuzzy logic method proposed in this paper. This method provides easy hardware implementation and straightforward interpretability. Experiments on two typical chaotic time series predictions have been carried out and the obtained results show that the average CPU time can be reduced significantly at the cost of a small decrease in prediction accuracy, which is favourable for the hardware implementation for chaotic time series prediction.

A Multiple Model Approach to Modeling Based on Fuzzy Support Vector Machines

A new multiple models(MM) approach was proposed to model complex industrial process by using Fuzzy Support Vector Machines(F -SVMs). By applying the proposed approach to a pH neutralization titration experiment, F -SVMs MM not only provides satisfactory approximation and generalization property, but also achieves superior performance to USOCPN multiple modeling method and single modeling method based on standard SVMs.

Forecast Urban Air Pollution in Mexico City by Using Support Vector Machines: A Kernel Performance Approach

The development of forecasting models for pollution particles shows a nonlinear dynamic behavior;hence, implementation is a non-trivial process. In the literature, there have been multiple models of particulate pollutants, which use softcomputing techniques and machine learning such as: multilayer perceptrons, neural networks, support vector machines, kernel algorithms, and so on. This paper presents a prediction pollution model using support vector machines and kernel functions, which are: Gaussian, Polynomial and Spline. Finally, the prediction results of ozone (O3), particulate matter (PM10) and nitrogen dioxide (NO2) at Mexico City are presented as a case study using these techniques.

WAVELET KERNEL SUPPORT VECTOR MACHINES FOR SPARSE APPROXIMATION

Wavelet, a powerful tool for signal processing, can be used to approximate the target func-tion. For enhancing the sparse property of wavelet approximation, a new algorithm was proposed by using wavelet kernel Support Vector Machines (SVM), which can converge to minimum error with bet-ter sparsity. Here, wavelet functions would be firstly used to construct the admitted kernel for SVM according to Mercy theory; then new SVM with this kernel can be used to approximate the target fun-citon with better sparsity than wavelet approxiamtion itself. The results obtained by our simulation ex-periment show the feasibility and validity of wavelet kernel support vector machines.

Construction and application of pre-classified smooth semi-supervised twin support vector machine

In order to handle the semi-supervised problem quickly and efficiently in the twin support vector machine (TWSVM) field, a semi-supervised twin support vector machine (S2TSVM) is proposed by adding the original unlabeled samples. In S2TSVM, the addition of unlabeled samples can easily cause the classification hyper plane to deviate from the sample points. Then a centerdistance principle is proposed to pre-classify unlabeled samples, and a pre-classified S2TSVM (PS2TSVM) is proposed. Compared with S2TSVM, PS2TSVM not only improves the problem of the samples deviating from the classification hyper plane, but also improves the training speed. Then PS2TSVM is smoothed. After smoothing the model, the pre-classified smooth S2TSVM (PS3TSVM) is obtained, and its convergence is deduced. Finally, nine datasets are selected in the UCI machine learning database for comparison with other types of semi-supervised models. The experimental results show that the proposed PS3TSVM model has better classification results.

A Kernel Clustering Algorithm for Fast Training of Support Vector Machines

A new algorithm named kernel bisecting k-means and sample removal(KBK-SR) is proposed as sampling preprocessing for support vector machine(SVM) training to improve the efficiency.The proposed algorithm tends to quickly produce balanced clusters of similar sizes in the kernel feature space,which makes it efficient and effective for reducing training samples.Theoretical analysis and experimental results on three UCI real data benchmarks both show that,with very short sampling time,the proposed algorithm dramatically accelerates SVM sampling and training while maintaining high test accuracy.

Support vector machine with mixture of kernels for automatic image annotation

Automatic image annotation(AIA)has become an important and challenging problem in computer vision due to the existence of semantic gap.In this paper,a novel support vector machine with mixture of kernels(SVM-MK)for automatic image annotation is proposed.On one hand,the combined global and local block-based image features are extracted in order to reflect the intrinsic content of images as complete as possible.On the other hand,SVM-MK is constructed to shoot for better annotating performance.Experimental results on Corel dataset show that the proposed image feature representation method as well as automatic image annotation classifier,SVM-MK,can achieve higher annotating accuracy than SVM with any single kernel and mi-SVM for semantic image annotation.

Traffic Sign Recognition Based on CNN and Twin Support Vector Machine Hybrid Model

With the progress of deep learning research, convolutional neural networks have become the most important method in feature extraction. How to effectively classify and recognize the extracted features will directly affect the performance of the entire network. Traditional processing methods include classification models such as fully connected network models and support vector machines. In order to solve the problem that the traditional convolutional neural network is prone to over-fitting for the classification of small samples, a CNN-TWSVM hybrid model was proposed by fusing the twin support vector machine (TWSVM) with higher computational efficiency as the CNN classifier, and it was applied to the traffic sign recognition task. In order to improve the generalization ability of the model, the wavelet kernel function is introduced to deal with the nonlinear classification task. The method uses the network initialized from the ImageNet dataset to fine-tune the specific domain and intercept the inner layer of the network to extract the high abstract features of the traffic sign image. Finally, the TWSVM based on wavelet kernel function is used to identify the traffic signs, so as to effectively solve the over-fitting problem of traffic signs classification. On GTSRB and BELGIUMTS datasets, the validity and generalization ability of the improved model is verified by comparing with different kernel functions and different SVM classifiers.

A Novel Kernel for Least Squares Support Vector Machine

Extreme learning machine(ELM) has attracted much attention in recent years due to its fast convergence and good performance.Merging both ELM and support vector machine is an important trend,thus yielding an ELM kernel.ELM kernel based methods are able to solve the nonlinear problems by inducing an explicit mapping compared with the commonly-used kernels such as Gaussian kernel.In this paper,the ELM kernel is extended to the least squares support vector regression(LSSVR),so ELM-LSSVR was proposed.ELM-LSSVR can be used to reduce the training and test time simultaneously without extra techniques such as sequential minimal optimization and pruning mechanism.Moreover,the memory space for the training and test was relieved.To confirm the efficacy and feasibility of the proposed ELM-LSSVR,the experiments are reported to demonstrate that ELM-LSSVR takes the advantage of training and test time with comparable accuracy to other algorithms.

A stacked multiple kernel support vector machine for blast inducedflyrock prediction

As a widely used rock excavation method in civil and mining construction works, the blasting operations and theinduced side effects are always investigated by the existing studies. The occurrence of flyrock is regarded as one ofthe most important issues induced by blasting operations, since the accurate prediction of which is crucial fordelineating safety zone. For this purpose, this study developed a flyrock prediction model based on 234 sets ofblasting data collected from Sugun Copper Mine site. A stacked multiple kernel support vector machine (stackedMK-SVM) model was proposed for flyrock prediction. The proposed stacked structure can effectively improve themodel performance by addressing the importance level of different features. For comparison purpose, 6 othermachine learning models were developed, including SVM, MK-SVM, Lagragian Twin SVM (LTSVM), ArtificialNeural Network (ANN), Random Forest (RF) and M5 Tree. This study implemented a 5-fold cross validationprocess for hyperparameters tuning purpose. According to the evaluation results, the proposed stacked MK-SVMmodel achieved the best overall performance, with RMSE of 1.73 and 1.74, MAE of 0.58 and 1.08, VAF of 98.95and 99.25 in training and testing phase, respectively.

Temperature prediction control based on least squares support vector machines

A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controlled plant is built by LS-SVM with radial basis function (RBF) kernel. In the process of system running, the off-line model is linearized at each sampling instant, and the generalized prediction control (GPC) algorithm is employed to implement the prediction control for the controlled plant. The obtained algorithm is applied to a boiler temperature control system with complicated nonlinearity and large time delay. The results of the experiment verify the effectiveness and merit of the algorithm.

Support vector machine based nonlinear model multi-step-ahead optimizing predictive control

A support vector machine with guadratic polynomial kernel function based nonlinear model multi-step-ahead optimizing predictive controller was presented. A support vector machine based predictive model was established by black-box identification. And a quadratic objective function with receding horizon was selected to obtain the controller output. By solving a nonlinear optimization problem with equality constraint of model output and boundary constraint of controller output using Nelder-Mead simplex direct search method, a sub-optimal control law was achieved in feature space. The effect of the controller was demonstrated on a recognized benchmark problem and a continuous-stirred tank reactor. The simulation results show that the multi-step-ahead predictive controller can be well applied to nonlinear system, with better performance in following reference trajectory and disturbance-rejection.

TWIN SUPPORT TENSOR MACHINES FOR MCS DETECTION

Tensor representation is useful to reduce the overfitting problem in vector-based learning algorithm in pattern recognition.This is mainly because the structure information of objects in pattern analysis is a reasonable constraint to reduce the number of unknown parameters used to model a classifier.In this paper, we generalize the vector-based learning algorithm TWin Support Vector Machine(TWSVM) to the tensor-based method TWin Support Tensor Machines(TWSTM), which accepts general tensors as input.To examine the effectiveness of TWSTM, we implement the TWSTM method for Microcalcification Clusters(MCs) detection.In the tensor subspace domain, the MCs detection procedure is formulated as a supervised learning and classification problem, and TWSTM is used as a classifier to make decision for the presence of MCs or not.A large number of experiments were carried out to evaluate and compare the performance of the proposed MCs detection algorithm.By comparison with TWSVM, the tensor version reduces the overfitting problem.

New predictive control algorithms based on Least Squares Support Vector Machines

Used for industrial process with different degree of nonlinearity, the two predictive control algorithms presented in this paper are based on Least Squares Support Vector Machines (LS-SVM) model. For the weakly nonlinear system, the system model is built by using LS-SVM with linear kernel function, and then the obtained linear LS-SVM model is transformed into linear input-output relation of the controlled system. However, for the strongly nonlinear system, the off-line model of the controlled system is built by using LS-SVM with Radial Basis Function (RBF) kernel. The obtained nonlinear LS-SVM model is linearized at each sampling instant of system running, after which the on-line linear input-output model of the system is built. Based on the obtained linear input-output model, the Generalized Predictive Control (GPC) algorithm is employed to implement predictive control for the controlled plant in both algorithms. The simulation results after the presented algorithms were implemented in two different industrial processes model; respectively revealed the effectiveness and merit of both algorithms.

Mandarin Digits Speech Recognition Using Support Vector Machines

A method of applying support vector machine (SVM) in speech recognition was proposed, and a speech recognition system for mandarin digits was built up by SVMs. In the system, vectors were linearly extracted from speech feature sequence to make up time-aligned input patterns for SVM, and the decisions of several 2-class SVM classifiers were employed for constructing an N-class classifier. Four kinds of SVM kernel functions were compared in the experiments of speaker-independent speech recognition of mandarin digits. And the kernel of radial basis function has the highest accurate rate of 99.33%, which is better than that of the baseline system based on hidden Markov models (HMM) (97.08%). And the experiments also show that SVM can outperform HMM especially when the samples for learning were very limited.