Application of Least Squares Support Vector Machine for Regression to Reliability Analysis

In order to deal with the issue of huge computational cost very well in direct numerical simulation, the traditional response surface method (RSM) as a classical regression algorithm is used to approximate a functional relationship between the state variable and basic variables in reliability design. The algorithm has treated successfully some problems of implicit performance function in reliability analysis. However, its theoretical basis of empirical risk minimization narrows its range of applications for...

Least Squares One-Class Support Tensor Machine

One-class classification problem has become a popular problem in many fields, with a wide range of applications in anomaly detection, fault diagnosis, and face recognition. We investigate the one-class classification problem for second-order tensor data. Traditional vector-based one-class classification methods such as one-class support vector machine (OCSVM) and least squares one-class support vector machine (LSOCSVM) have limitations when tensor is used as input data, so we propose a new tensor one-class classification method, LSOCSTM, which directly uses tensor as input data. On one hand, using tensor as input data not only enables to classify tensor data, but also for vector data, classifying it after high dimensionalizing it into tensor still improves the classification accuracy and overcomes the over-fitting problem. On the other hand, different from one-class support tensor machine (OCSTM), we use squared loss instead of the original loss function so that we solve a series of linear equations instead of quadratic programming problems. Therefore, we use the distance to the hyperplane as a metric for classification, and the proposed method is more accurate and faster compared to existing methods. The experimental results show the high efficiency of the proposed method compared with several state-of-the-art methods.

NOVEL WEIGHTED LEAST SQUARES SUPPORT VECTOR REGRESSION FOR THRUST ESTIMATION ON PERFORMANCE DETERIORATION OF AERO-ENGINE

A thrust estimator with high precision and excellent real-time performance is needed to mitigate perfor- mance deterioration for future aero-engines. A weight least squares support vector regression is proposed using a novel weighting strategy. Then a thrust estimator based on the proposed regression is designed for the perfor- mance deterioration. Compared with the existing weighting strategy, the novel one not only satisfies the require- ment of precision but also enhances the real-time performance. Finally, numerical experiments demonstrate the effectiveness and feasibility of the proposed weighted least squares support vector regression for thrust estimator. Key words ： intelligent engine control; least squares ; support vector machine ; performance deterioration

Improved Twin Support Vector Machine Algorithm and Applications in Classification Problems

The distribution of data has a significant impact on the results of classification.When the distribution of one class is insignificant compared to the distribution of another class,data imbalance occurs.This will result in rising outlier values and noise.Therefore,the speed and performance of classification could be greatly affected.Given the above problems,this paper starts with the motivation and mathematical representing of classification,puts forward a new classification method based on the relationship between different classification formulations.Combined with the vector characteristics of the actual problem and the choice of matrix characteristics,we firstly analyze the orderly regression to introduce slack variables to solve the constraint problem of the lone point.Then we introduce the fuzzy factors to solve the problem of the gap between the isolated points on the basis of the support vector machine.We introduce the cost control to solve the problem of sample skew.Finally,based on the bi-boundary support vector machine,a twostep weight setting twin classifier is constructed.This can help to identify multitasks with feature-selected patterns without the need for additional optimizers,which solves the problem of large-scale classification that can’t deal effectively with the very low category distribution gap.

Development of a least squares support vector machine model for prediction of natural gas hydrate formation temperature

Hydrates always are considered as a threat to petroleum industry due to the operational problems it can cause.These problems could result in reducing production performance or even production stoppage for a long time.In this paper, we were intended to develop a LSSVM algorithm for prognosticating hydrate formation temperature(HFT) in a wide range of natural gas mixtures. A total number of 279 experimental data points were extracted from open literature to develop the LSSVM. The input parameters were chosen based on the hydrate structure that each gas species form. The modeling resulted in a robust algorithm with the squared correlation coefficients(R^2) of 0.9918. Aside from the excellent statistical parameters of the model, comparing proposed LSSVM with some of conventional correlations showed its supremacy, particularly in the case of sour gases with high H_2S concentrations, where the model surpasses all correlations and existing thermodynamic models. For detection of the probable doubtful experimental data, and applicability of the model, the Leverage statistical approach was performed on the data sets. This algorithm showed that the proposed LSSVM model is statistically valid for HFT prediction and almost all the data points are in the applicability domain of the model.

Improved adaptive pruning algorithm for least squares support vector regression

As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satis- factory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which in- volves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generali- zation performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.

A sparse algorithm for adaptive pruning least square support vector regression machine based on global representative point ranking

Least square support vector regression(LSSVR)is a method for function approximation,whose solutions are typically non-sparse,which limits its application especially in some occasions of fast prediction.In this paper,a sparse algorithm for adaptive pruning LSSVR algorithm based on global representative point ranking(GRPR-AP-LSSVR)is proposed.At first,the global representative point ranking(GRPR)algorithm is given,and relevant data analysis experiment is implemented which depicts the importance ranking of data points.Furthermore,the pruning strategy of removing two samples in the decremental learning procedure is designed to accelerate the training speed and ensure the sparsity.The removed data points are utilized to test the temporary learning model which ensures the regression accuracy.Finally,the proposed algorithm is verified on artificial datasets and UCI regression datasets,and experimental results indicate that,compared with several benchmark algorithms,the GRPR-AP-LSSVR algorithm has excellent sparsity and prediction speed without impairing the generalization performance.

Least Squares Support Vector Machine Based Real-Time Fault Diagnosis Model for Gas Path Parameters of Aero Engines

Least squares support vector machine （LS-SVM） is applied in gas path fault diagnosis for aero engines. Firstly, the deviation data of engine cruise are analyzed. Then, model selection is conducted using pattern search method. Finally, by decoding aircraft communication addressing and reporting system （ACARS） report, a real-time cruise data set is acquired, and the diagnosis model is adopted to process data. In contrast to the radial basis function （RBF） neutral network, LS-SVM is more suitable for real-time diagnosis of gas turbine engine.

ONLINE PARSIMONIOUS LEAST SQUARES SUPPORT VECTOR REGRESSION AND ITS APPLICATION

A simple and effective mechanism is proposed to realize the parsimoniousness of the online least squares support vector regression （LS-SVR）, and the approach is called the OPLS-SVR for short. Hence, the response time is curtailed. Besides, an OPLS-SVR based analytical redundancy technique is presented to cope with the sensor failure and drift problems to guarantee that the provided signals for the aeroengine controller are correct and acceptable. Experiments on the sensor failure and drift show the effectiveness and the validity of the proposed analytical redundancy.

Combination Computing of Support Vector Machine, Support Vector Regression and Molecular Docking for Potential Cytochrome P450 1A2 Inhibitors

The computational approaches of support vector machine （SVM）, support vector regression （SVR） and molecular docking were widely utilized for the computation of active compounds. In this work, to improve the accuracy and reliability of prediction, the strategy of combining the above three computational approaches was applied to predict potential cytochrome P450 1A2 （CYP1A2） inhibitors. The accuracy of the optimal SVM qualitative model was 99.432%, 97.727%, and 91.667% for training set, internal test set and external test set, respectively, showing this model had high discrimination ability. The R2 and mean square error for the optimal SVR quantitative model were 0.763, 0.013 for training set, and 0.753, 0.056 for test set respectively, indicating that this SVR model has high predictive ability for the biolog-ical activities of compounds. According to the results of the SVM and SVR models, some types of descriptors were identi ed to be essential to bioactivity prediction of compounds, including the connectivity indices, constitutional descriptors and functional group counts. Moreover, molecular docking studies were used to reveal the binding poses and binding a n-ity of potential inhibitors interacting with CYP1A2. Wherein, the amino acids of THR124 and ASP320 could form key hydrogen bond interactions with active compounds. And the amino acids of ALA317 and GLY316 could form strong hydrophobic bond interactions with active compounds. The models obtained above were applied to discover potential CYP1A2 inhibitors from natural products, which could predict the CYPs-mediated drug-drug inter-actions and provide useful guidance and reference for rational drug combination therapy. A set of 20 potential CYP1A2 inhibitors were obtained. Part of the results was consistent with references, which further indicates the accuracy of these models and the reliability of this combinatorial computation strategy.

BOOSTING SPARSE LEAST SQUARES SUPPORT VECTOR REGRESSION (BSLSSVR) AND ITS APPLICATION TO THRUST ESTIMATION

In order to realize direct thrust control instead of conventional sensors-based control for aero-engine, a thrust estimator with high accuracy is designed by using the boosting technique to improve the performance of least squares support vector regression （LSSVR）. There exist two distinct features compared with the conven- tional boosting technique： （1） Sampling without replacement is used to avoid numerical instability for modeling LSSVR. （2） To realize the sparseness of LSSVR and reduce the computational complexity, only a subset of the training samples is used to construct LSSVR. Thus, this boosting method for LSSVR is called the boosting sparse LSSVR （BSLSSVR）. Finally, simulation results show that BSLSSVR-based thrust estimator can satisfy the requirement of direct thrust control, i.e. , maximum absolute value of relative error of thrust estimation is not more than 5‰.

Semi-supervised least squares support vector machine algorithm:application to offshore oil reservoir

At the early stages of deep-water oil exploration and development, fewer and further apart wells are drilled than in onshore oilfields. Supervised least squares support vector machine algorithms are used to predict the reservoir parameters but the prediction accuracy is low. We combined the least squares support vector machine （LSSVM） algorithm with semi-supervised learning and established a semi-supervised regression model, which we call the semi-supervised least squares support vector machine （SLSSVM） model. The iterative matrix inversion is also introduced to improve the training ability and training time of the model. We use the UCI data to test the generalization of a semi-supervised and a supervised LSSVM models. The test results suggest that the generalization performance of the LSSVM model greatly improves and with decreasing training samples the generalization performance is better. Moreover, for small-sample models, the SLSSVM method has higher precision than the semi-supervised K-nearest neighbor （SKNN） method. The new semi- supervised LSSVM algorithm was used to predict the distribution of porosity and sandstone in the Jingzhou study area.

Classification using least squares support vector machine for reliability analysis

In order to improve the efficiency of the support vector machine （SVM） for classification to deal with a large amount of samples, the least squares support vector machine （LSSVM） for classification methods is introduced into the reliability analysis. To reduce the coraputational cost, the solution of the SVM is transformed from a quadratic programming to a group of linear equations. The numerical results indicate that the reliability method based on the LSSVM for classification has higher accuracy and requires less computational cost than the SVM method.

Improved Scheme for Fast Approximation to Least Squares Support Vector Regression

The solution of normal least squares support vector regression(LSSVR)is lack of sparseness,which limits the real-time and hampers the wide applications to a certain degree.To overcome this obstacle,a scheme,named I2FSA-LSSVR,is proposed.Compared with the previously approximate algorithms,it not only adopts the partial reduction strategy but considers the influence between the previously selected support vectors and the willselected support vector during the process of computing the supporting weights.As a result,I2FSA-LSSVR reduces the number of support vectors and enhances the real-time.To confirm the feasibility and effectiveness of the proposed algorithm,experiments on benchmark data sets are conducted,whose results support the presented I2FSA-LSSVR.

Seasonal Least Squares Support Vector Machine with Fruit Fly Optimization Algorithm in Electricity Consumption Forecasting

Electricity is the guarantee of economic development and daily life. Thus, accurate monthly electricity consumption forecasting can provide reliable guidance for power construction planning. In this paper, a hybrid model in combination of least squares support vector machine(LSSVM) model with fruit fly optimization algorithm(FOA) and the seasonal index adjustment is constructed to predict monthly electricity consumption. The monthly electricity consumption demonstrates a nonlinear characteristic and seasonal tendency. The LSSVM has a good fit for nonlinear data, so it has been widely applied to handling nonlinear time series prediction. However, there is no unified selection method for key parameters and no unified method to deal with the effect of seasonal tendency. Therefore, the FOA was hybridized with the LSSVM and the seasonal index adjustment to solve this problem. In order to evaluate the forecasting performance of hybrid model, two samples of monthly electricity consumption of China and the United States were employed, besides several different models were applied to forecast the two empirical time series. The results of the two samples all show that, for seasonal data, the adjusted model with seasonal indexes has better forecasting performance. The forecasting performance is better than the models without seasonal indexes. The fruit fly optimized LSSVM model outperforms other alternative models. In other words, the proposed hybrid model is a feasible method for the electricity consumption forecasting.

Thrust estimator design based on least squares support vector regression machine

In order to realize direct thrust control instead of traditional sensor-based control for aero-engines,it is indispensable to design a thrust estimator with high accuracy,so a scheme for thrust estimator design based on the least square support vector regression machine is proposed to solve this problem. Furthermore,numerical simulations confirm the effectiveness of our presented scheme. During the process of estimator design,a wrapper criterion that can not only reduce the computational complexity but also enhance the generalization performance is proposed to select variables as input variables for estimator.

Modeling of Isomerization of C_8 Aromatics by Online Least Squares Support Vector Machine

The least squares support vector regression (LS-SVR) is usually used for the modeling of single output system, but it is not well suitable for the actual multi-input-multi-output system. The paper aims at the modeling of multi-output systems by LS-SVR. The multi-output LS-SVR is derived in detail. To avoid the inversion of large matrix, the recursive algorithm of the parameters is given, which makes the online algorithm of LS-SVR practical. Since the computing time increases with the number of training samples, the sparseness is studied based on the pro-jection of online LS-SVR. The residual of projection less than a threshold is omitted, so that a lot of samples are kept out of the training set and the sparseness is obtained. The standard LS-SVR, nonsparse online LS-SVR and sparse online LS-SVR with different threshold are used for modeling the isomerization of C8 aromatics. The root-mean-square-error (RMSE), number of support vectors and running time of three algorithms are compared and the result indicates that the performance of sparse online LS-SVR is more favorable.

Temperature prediction control based on least squares support vector machines

A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controlled plant is built by LS-SVM with radial basis function (RBF) kernel. In the process of system running, the off-line model is linearized at each sampling instant, and the generalized prediction control (GPC) algorithm is employed to implement the prediction control for the controlled plant. The obtained algorithm is applied to a boiler temperature control system with complicated nonlinearity and large time delay. The results of the experiment verify the effectiveness and merit of the algorithm.

Generalized Predictive Control with Online Least Squares Support Vector Machines

这份报纸基于能有效地处理非线性的系统的联机最少的广场支持向量机器(LS-SVM ) 建议一个实际概括预兆的控制(GPC ) 算法。在每个采样时期，算法递归地由增加新数据对并且在实时性质上从考虑删除最不重要的修改模型。删除的数据对被 lagrange 的绝对值从最后一个采样时期更多样地决定。当增加新数据对并且删除存在的时，纸给模型参数的递归的算法分别地，一个大矩阵的倒置被避免，存储器能被算法完全控制。非线性的 LS-SVM 模型在每个采样时期在 GPC 算法被使用。抵销过程的 pH 上的概括预兆的控制的实验显示出建议算法的有效性和实物。