Improved Twin Support Vector Machine Algorithm and Applications in Classification Problems

The distribution of data has a significant impact on the results of classification.When the distribution of one class is insignificant compared to the distribution of another class,data imbalance occurs.This will result in rising outlier values and noise.Therefore,the speed and performance of classification could be greatly affected.Given the above problems,this paper starts with the motivation and mathematical representing of classification,puts forward a new classification method based on the relationship between different classification formulations.Combined with the vector characteristics of the actual problem and the choice of matrix characteristics,we firstly analyze the orderly regression to introduce slack variables to solve the constraint problem of the lone point.Then we introduce the fuzzy factors to solve the problem of the gap between the isolated points on the basis of the support vector machine.We introduce the cost control to solve the problem of sample skew.Finally,based on the bi-boundary support vector machine,a twostep weight setting twin classifier is constructed.This can help to identify multitasks with feature-selected patterns without the need for additional optimizers,which solves the problem of large-scale classification that can’t deal effectively with the very low category distribution gap.

Forecasting model of residential load based on general regression neural network and PSO-Bayes least squares support vector machine

Firstly,general regression neural network(GRNN) was used for variable selection of key influencing factors of residential load(RL) forecasting.Secondly,the key influencing factors chosen by GRNN were used as the input and output terminals of urban and rural RL for simulating and learning.In addition,the suitable parameters of final model were obtained through applying the evidence theory to combine the optimization results which were calculated with the PSO method and the Bayes theory.Then,the model of PSO-Bayes least squares support vector machine(PSO-Bayes-LS-SVM) was established.A case study was then provided for the learning and testing.The empirical analysis results show that the mean square errors of urban and rural RL forecast are 0.02% and 0.04%,respectively.At last,taking a specific province RL in China as an example,the forecast results of RL from 2011 to 2015 were obtained.

Small-time scale network traffic prediction based on a local support vector machine regression model

In this paper we apply the nonlinear time series analysis method to small-time scale traffic measurement data. The prediction-based method is used to determine the embedding dimension of the traffic data. Based on the reconstructed phase space, the local support vector machine prediction method is used to predict the traffic measurement data, and the BIC-based neighbouring point selection method is used to choose the number of the nearest neighbouring points for the local support vector machine regression model. The experimental results show that the local support vector machine prediction method whose neighbouring points are optimized can effectively predict the small-time scale traffic measurement data and can reproduce the statistical features of real traffic measurements.

Optimization of support vector machine power load forecasting model based on data mining and Lyapunov exponents

According to the chaotic and non-linear characters of power load data,the time series matrix is established with the theory of phase-space reconstruction,and then Lyapunov exponents with chaotic time series are computed to determine the time delay and the embedding dimension.Due to different features of the data,data mining algorithm is conducted to classify the data into different groups.Redundant information is eliminated by the advantage of data mining technology,and the historical loads that have highly similar features with the forecasting day are searched by the system.As a result,the training data can be decreased and the computing speed can also be improved when constructing support vector machine(SVM) model.Then,SVM algorithm is used to predict power load with parameters that get in pretreatment.In order to prove the effectiveness of the new model,the calculation with data mining SVM algorithm is compared with that of single SVM and back propagation network.It can be seen that the new DSVM algorithm effectively improves the forecast accuracy by 0.75%,1.10% and 1.73% compared with SVM for two random dimensions of 11-dimension,14-dimension and BP network,respectively.This indicates that the DSVM gains perfect improvement effect in the short-term power load forecasting.

Forecasting regional economic growth using support vector machine model

Support vector machine(SVM)is a new technology in data mining.It is a new tool to solve machine learning problems with the help of optimization.Support vector machines belong to a new machine learning that extends from statistical learning theory.Its structure is relatively simple,with good generalization ability and global optimality.Support vector machine has provided a unified framework for solving finite sample learning problems,and there are many solutions proposed.It can deal with those more complex problems and introduce the characteristics of the support vector machine model.Aiming at the application of the model in economic forecasting,a method to improve the prediction accuracy of the model is proposed.The theoretical analysis and practical application verification are performed,which shows that this method can obtain more accurate prediction results.

Path Loss Modeling: A Machine Learning Based Approach Using Support Vector Regression and Radial Basis Function Models

Path loss prediction models are vital for accurate signal propagation in wireless channels. Empirical and deterministic models used in path loss predictions have not produced optimal results. In this paper, we introduced machine learning algorithms to path loss predictions because it offers a flexible network architecture and extensive data can be used. We introduced support vector regression (SVR) and radial basis function (RBF) models to path loss predictions in the investigated environments. The SVR model was able to process several input parameters without introducing complexity to the network architecture. The RBF on its part provides a good function approximation. Hyperparameter tuning of the machine learning models was carried out in order to achieve optimal results. The performances of the SVR and RBF models were compared and result validated using the root-mean squared error (RMSE). The two machine learning algorithms were also compared with the Cost-231, SUI, Egli, Freespace, Cost-231 W-I models. The analytical models overpredicted path loss. Overall, the machine learning models predicted path loss with greater accuracy than the empirical models. The SVR model performed best across all the indices with RMSE values of 1.378 dB, 1.4523 dB, 2.1568 dB in rural, suburban and urban settings respectively and should therefore be adopted for signal propagation in the investigated environments and beyond.

On-line forecasting model for zinc output based on self-tuning support vector regression and its application

An on-line forecasting model based on self-tuning support vectors regression for zinc output was put forward to maximize zinc output by adjusting operational parameters in the process of imperial smelting furnace. In this model, the mathematical model of support vector regression was converted into the same format as support vector machine for classification. Then a simplified sequential minimal optimization for classification was applied to train the regression coefficient vector α- α* and threshold b. Sequentially penalty parameter C was tuned dynamically through forecasting result during the training process. Finally, an on-line forecasting algorithm for zinc output was proposed. The simulation result shows that in spite of a relatively small industrial data set, the effective error is less than 10% with a remarkable performance of real time. The model was applied to the optimization operation and fault diagnosis system for imperial smelting furnace.

Combination Computing of Support Vector Machine, Support Vector Regression and Molecular Docking for Potential Cytochrome P450 1A2 Inhibitors

The computational approaches of support vector machine （SVM）, support vector regression （SVR） and molecular docking were widely utilized for the computation of active compounds. In this work, to improve the accuracy and reliability of prediction, the strategy of combining the above three computational approaches was applied to predict potential cytochrome P450 1A2 （CYP1A2） inhibitors. The accuracy of the optimal SVM qualitative model was 99.432%, 97.727%, and 91.667% for training set, internal test set and external test set, respectively, showing this model had high discrimination ability. The R2 and mean square error for the optimal SVR quantitative model were 0.763, 0.013 for training set, and 0.753, 0.056 for test set respectively, indicating that this SVR model has high predictive ability for the biolog-ical activities of compounds. According to the results of the SVM and SVR models, some types of descriptors were identi ed to be essential to bioactivity prediction of compounds, including the connectivity indices, constitutional descriptors and functional group counts. Moreover, molecular docking studies were used to reveal the binding poses and binding a n-ity of potential inhibitors interacting with CYP1A2. Wherein, the amino acids of THR124 and ASP320 could form key hydrogen bond interactions with active compounds. And the amino acids of ALA317 and GLY316 could form strong hydrophobic bond interactions with active compounds. The models obtained above were applied to discover potential CYP1A2 inhibitors from natural products, which could predict the CYPs-mediated drug-drug inter-actions and provide useful guidance and reference for rational drug combination therapy. A set of 20 potential CYP1A2 inhibitors were obtained. Part of the results was consistent with references, which further indicates the accuracy of these models and the reliability of this combinatorial computation strategy.

Comparative study of different machine learning models in landslide susceptibility assessment: A case study of Conghua District, Guangzhou, China

Machine learning is currently one of the research hotspots in the field of landslide prediction.To clarify and evaluate the differences in characteristics and prediction effects of different machine learning models,Conghua District,which is the most prone to landslide disasters in Guangzhou,was selected for landslide susceptibility evaluation.The evaluation factors were selected by using correlation analysis and variance expansion factor method.Applying four machine learning methods namely Logistic Regression(LR),Random Forest(RF),Support Vector Machines(SVM),and Extreme Gradient Boosting(XGB),landslide models were constructed.Comparative analysis and evaluation of the model were conducted through statistical indices and receiver operating characteristic(ROC)curves.The results showed that LR,RF,SVM,and XGB models have good predictive performance for landslide susceptibility,with the area under curve(AUC)values of 0.752,0.965,0.996,and 0.998,respectively.XGB model had the highest predictive ability,followed by RF model,SVM model,and LR model.The frequency ratio(FR)accuracy of LR,RF,SVM,and XGB models was 0.775,0.842,0.759,and 0.822,respectively.RF and XGB models were superior to LR and SVM models,indicating that the integrated algorithm has better predictive ability than a single classification algorithm in regional landslide classification problems.

Early Warning Model of Diamondback Moth Based on ε-Support Vector Regression

The model for predicting vegetable pest diamondback moth was established based on E-Support Vector Regression algorithms in the multiply occurrence season of diamondback moth. The experimental data of diamondback moth in Guangdong vegetable were analyzed, and the result showed that when penalty factor c was 43, kernel function parameter k was O. 2, the better prediction result could be obtained by the early warning model of E-Support Vector Regression algorithms.

Application of support vector machine algorithm for early differential diagnosis of prostate cancer

Prostate cancer(PCa)symptoms are commonly confused with benign prostate hyperplasia(BPH),particularly in the early stages due to similarities between symptoms,and in some instances,underdiagnoses.Clinical methods have been utilized to diagnose PCa;however,at the full-blown stage,clinical methods usually present high risks of complicated side effects.Therefore,we proposed the use of support vector machine for early differential diagnosis of PCa(SVM-PCa-EDD).SVM was used to classify persons with and without PCa.We used the PCa dataset from the Kaggle Healthcare repository to develop and validate SVM model for classification.The PCa dataset consisted of 250 features and one class of features.Attributes considered in this study were age,body mass index(BMI),race,family history,obesity,trouble urinating,urine stream force,blood in semen,bone pain,and erectile dysfunction.The SVM-PCa-EDD was used for preprocessing the PCa dataset,specifically dealing with class imbalance,and for dimensionality reduction.After eliminating class imbalance,the area under the receiver operating characteristic(ROC)curve(AUC)of the logistic regression(LR)model trained with the downsampled dataset was 58.4%,whereas that of the AUC-ROC of LR trained with the class imbalance dataset was 54.3%.The SVM-PCa-EDD achieved 90%accuracy,80%sensitivity,and 80%specificity.The validation of SVM-PCa-EDD using random forest and LR showed that SVM-PCa-EDD performed better in early differential diagnosis of PCa.The proposed model can assist medical experts in early diagnosis of PCa,particularly in resource-constrained healthcare settings and making further recommendations for PCa testing and treatment.

Nonlinear Model Predictive Control Based on Support Vector Machine with Multi-kernel

Multi-kernel-based support vector machine （SVM） model structure of nonlinear systems and its specific identification method is proposed, which is composed of a SVM with linear kernel function followed in series by a SVM with spline kernel function. With the help of this model, nonlinear model predictive control can be transformed to linear model predictive control, and consequently a unified analytical solution of optimal input of multi-step-ahead predictive control is possible to derive. This algorithm does not require online iterative optimization in order to be suitable for real-time control with less calculation. The simulation results of pH neutralization process and CSTR reactor show the effectiveness and advantages of the presented algorithm.

Support vector machine forecasting method improved by chaotic particle swarm optimization and its application

By adopting the chaotic searching to improve the global searching performance of the particle swarm optimization (PSO), and using the improved PSO to optimize the key parameters of the support vector machine (SVM) forecasting model, an improved SVM model named CPSO-SVM model was proposed. The new model was applied to predicting the short term load, and the improved effect of the new model was proved. The simulation results of the South China Power Market’s actual data show that the new method can effectively improve the forecast accuracy by 2.23% and 3.87%, respectively, compared with the PSO-SVM and SVM methods. Compared with that of the PSO-SVM and SVM methods, the time cost of the new model is only increased by 3.15 and 4.61 s, respectively, which indicates that the CPSO-SVM model gains significant improved effects.

Support vector machine based nonlinear model multi-step-ahead optimizing predictive control

A support vector machine with guadratic polynomial kernel function based nonlinear model multi-step-ahead optimizing predictive controller was presented. A support vector machine based predictive model was established by black-box identification. And a quadratic objective function with receding horizon was selected to obtain the controller output. By solving a nonlinear optimization problem with equality constraint of model output and boundary constraint of controller output using Nelder-Mead simplex direct search method, a sub-optimal control law was achieved in feature space. The effect of the controller was demonstrated on a recognized benchmark problem and a continuous-stirred tank reactor. The simulation results show that the multi-step-ahead predictive controller can be well applied to nonlinear system, with better performance in following reference trajectory and disturbance-rejection.

SOFT SENSING MODEL BASED ON SUPPORT VECTOR MACHINE AND ITS APPLICATION

Soft sensor is widely used in industrial process control. It plays animportant role to improve the quality of product and assure safety in production. The core of softsensor is to construct soft sensing model. A new soft sensing modeling method based on supportvector machine (SVM) is proposed. SVM is a new machine learning method based on statistical learningtheory and is powerful for the problem characterized by small sample, nonlinearity, high dimensionand local minima. The proposed methods are applied to the estimation of frozen point of light dieseloil in distillation column. The estimated outputs of soft sensing model based on SVM match the realvalues of frozen point and follow varying trend of frozen point very well. Experiment results showthat SVM provides a new effective method for soft sensing modeling and has promising application inindustrial process applications.

Support Vector Machine active learning for 3D model retrieval

In this paper, we present a novel Support Vector Machine active learning algorithm for effective 3D model retrieval using the concept of relevance feedback. The proposed method learns from the most informative objects which are marked by the user, and then creates a boundary separating the relevant models from irrelevant ones. What it needs is only a small number of 3D models labelled by the user. It can grasp the user＇s semantic knowledge rapidly and accurately. Experimental results showed that the proposed algorithm significantly improves the retrieval effectiveness. Compared with four state-of-the-art query refinement schemes for 3D model retrieval, it provides superior retrieval performance after no more than two rounds of relevance feedback.

Semi-supervised Support Vector Regression Model for Remote Sensing Water Quality Retrieving

This paper proposed a semi-supervised regression model with co-training algorithm based on support vector machine, which was used for retrieving water quality variables from SPOT 5 remote sensing data. The model consisted of two support vector regressors (SVRs). Nonlinear relationship between water quality variables and SPOT 5 spectrum was described by the two SVRs, and semi-supervised co-training algorithm for the SVRs was es-tablished. The model was used for retrieving concentrations of four representative pollution indicators―permangan- ate index (CODmn), ammonia nitrogen (NH3-N), chemical oxygen demand (COD) and dissolved oxygen (DO) of the Weihe River in Shaanxi Province, China. The spatial distribution map for those variables over a part of the Weihe River was also produced. SVR can be used to implement any nonlinear mapping readily, and semi-supervis- ed learning can make use of both labeled and unlabeled samples. By integrating the two SVRs and using semi-supervised learning, we provide an operational method when paired samples are limited. The results show that it is much better than the multiple statistical regression method, and can provide the whole water pollution condi-tions for management fast and can be extended to hyperspectral remote sensing applications.

A Geometric Approach to Support Vector Regression and Its Application to Fermentation Process Fast Modeling

Support vector machine(SVM) has shown great potential in pattern recognition and regressive estima-tion.Due to the industrial development demands,such as the fermentation process modeling,improving the training performance on increasingly large sample sets is an important problem.However,solving a large optimization problem is computationally intensive and memory intensive.In this paper,a geometric interpretation of SVM re-gression(SVR) is derived,and μ-SVM is extended for both L1-norm and L2-norm penalty SVR.Further,Gilbert al-gorithm,a well-known geometric algorithm,is modified to solve SVR problems.Theoretical analysis indicates that the presented SVR training geometric algorithms have the same convergence and almost identical cost of computa-tion as their corresponding algorithms for SVM classification.Experimental results show that the geometric meth-ods are more efficient than conventional methods using quadratic programming and require much less memory.

Robustly stable model predictive control based on parallel support vector machines with linear kernel

Robustly stable multi-step-ahead model predictive control (MPC) based on parallel support vector machines (SVMs) with linear kernel was proposed. First, an analytical solution of optimal control laws of parallel SVMs based MPC was derived, and then the necessary and sufficient stability condition for MPC closed loop was given according to SVM model, and finally a method of judging the discrepancy between SVM model and the actual plant was presented, and consequently the constraint sets, which can guarantee that the stability condition is still robust for model/plant mismatch within some given bounds, were obtained by applying small-gain theorem. Simulation experiments show the proposed stability condition and robust constraint sets can provide a convenient way of adjusting controller parameters to ensure a closed-loop with larger stable margin.

A Multiple Model Approach to Modeling Based on Fuzzy Support Vector Machines

A new multiple models(MM) approach was proposed to model complex industrial process by using Fuzzy Support Vector Machines(F -SVMs). By applying the proposed approach to a pH neutralization titration experiment, F -SVMs MM not only provides satisfactory approximation and generalization property, but also achieves superior performance to USOCPN multiple modeling method and single modeling method based on standard SVMs.