A sparse algorithm for adaptive pruning least square support vector regression machine based on global representative point ranking

Least square support vector regression(LSSVR)is a method for function approximation,whose solutions are typically non-sparse,which limits its application especially in some occasions of fast prediction.In this paper,a sparse algorithm for adaptive pruning LSSVR algorithm based on global representative point ranking(GRPR-AP-LSSVR)is proposed.At first,the global representative point ranking(GRPR)algorithm is given,and relevant data analysis experiment is implemented which depicts the importance ranking of data points.Furthermore,the pruning strategy of removing two samples in the decremental learning procedure is designed to accelerate the training speed and ensure the sparsity.The removed data points are utilized to test the temporary learning model which ensures the regression accuracy.Finally,the proposed algorithm is verified on artificial datasets and UCI regression datasets,and experimental results indicate that,compared with several benchmark algorithms,the GRPR-AP-LSSVR algorithm has excellent sparsity and prediction speed without impairing the generalization performance.

ONLINE PARSIMONIOUS LEAST SQUARES SUPPORT VECTOR REGRESSION AND ITS APPLICATION

A simple and effective mechanism is proposed to realize the parsimoniousness of the online least squares support vector regression （LS-SVR）, and the approach is called the OPLS-SVR for short. Hence, the response time is curtailed. Besides, an OPLS-SVR based analytical redundancy technique is presented to cope with the sensor failure and drift problems to guarantee that the provided signals for the aeroengine controller are correct and acceptable. Experiments on the sensor failure and drift show the effectiveness and the validity of the proposed analytical redundancy.

NOVEL WEIGHTED LEAST SQUARES SUPPORT VECTOR REGRESSION FOR THRUST ESTIMATION ON PERFORMANCE DETERIORATION OF AERO-ENGINE

A thrust estimator with high precision and excellent real-time performance is needed to mitigate perfor- mance deterioration for future aero-engines. A weight least squares support vector regression is proposed using a novel weighting strategy. Then a thrust estimator based on the proposed regression is designed for the perfor- mance deterioration. Compared with the existing weighting strategy, the novel one not only satisfies the require- ment of precision but also enhances the real-time performance. Finally, numerical experiments demonstrate the effectiveness and feasibility of the proposed weighted least squares support vector regression for thrust estimator. Key words ： intelligent engine control; least squares ; support vector machine ; performance deterioration

BOOSTING SPARSE LEAST SQUARES SUPPORT VECTOR REGRESSION (BSLSSVR) AND ITS APPLICATION TO THRUST ESTIMATION

In order to realize direct thrust control instead of conventional sensors-based control for aero-engine, a thrust estimator with high accuracy is designed by using the boosting technique to improve the performance of least squares support vector regression （LSSVR）. There exist two distinct features compared with the conven- tional boosting technique： （1） Sampling without replacement is used to avoid numerical instability for modeling LSSVR. （2） To realize the sparseness of LSSVR and reduce the computational complexity, only a subset of the training samples is used to construct LSSVR. Thus, this boosting method for LSSVR is called the boosting sparse LSSVR （BSLSSVR）. Finally, simulation results show that BSLSSVR-based thrust estimator can satisfy the requirement of direct thrust control, i.e. , maximum absolute value of relative error of thrust estimation is not more than 5‰.

Thrust estimator design based on least squares support vector regression machine

In order to realize direct thrust control instead of traditional sensor-based control for aero-engines,it is indispensable to design a thrust estimator with high accuracy,so a scheme for thrust estimator design based on the least square support vector regression machine is proposed to solve this problem. Furthermore,numerical simulations confirm the effectiveness of our presented scheme. During the process of estimator design,a wrapper criterion that can not only reduce the computational complexity but also enhance the generalization performance is proposed to select variables as input variables for estimator.

Improved adaptive pruning algorithm for least squares support vector regression

As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satis- factory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which in- volves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generali- zation performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.

Temperature prediction control based on least squares support vector machines

A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controlled plant is built by LS-SVM with radial basis function (RBF) kernel. In the process of system running, the off-line model is linearized at each sampling instant, and the generalized prediction control (GPC) algorithm is employed to implement the prediction control for the controlled plant. The obtained algorithm is applied to a boiler temperature control system with complicated nonlinearity and large time delay. The results of the experiment verify the effectiveness and merit of the algorithm.

Generalized Predictive Control with Online Least Squares Support Vector Machines

这份报纸基于能有效地处理非线性的系统的联机最少的广场支持向量机器(LS-SVM ) 建议一个实际概括预兆的控制(GPC ) 算法。在每个采样时期，算法递归地由增加新数据对并且在实时性质上从考虑删除最不重要的修改模型。删除的数据对被 lagrange 的绝对值从最后一个采样时期更多样地决定。当增加新数据对并且删除存在的时，纸给模型参数的递归的算法分别地，一个大矩阵的倒置被避免，存储器能被算法完全控制。非线性的 LS-SVM 模型在每个采样时期在 GPC 算法被使用。抵销过程的 pH 上的概括预兆的控制的实验显示出建议算法的有效性和实物。

Image Denoising Using Local Adaptive Least Squares Support Vector Regression

Rather than attempting to separate signal from noise in the spatial domain, it is often advantageous to work in a transform domain. Building on previous work, a novel denoising method based on local adaptive least squares support vector regression is proposed. Investigation on real images contaminated by Gaussian noise has demonstrated that the proposed method can achieve an acceptable trade off between the noise removal and smoothing of the edges and details.

Prediction of chaotic systems with multidimensional recurrent least squares support vector machines

In this paper, we propose a multidimensional version of recurrent least squares support vector machines （MDRLS- SVM） to solve the problem about the prediction of chaotic system. To acquire better prediction performance, the high-dimensional space, which provides more information on the system than the scalar time series, is first reconstructed utilizing Takens＇s embedding theorem. Then the MDRLS-SVM instead of traditional RLS-SVM is used in the high- dimensional space, and the prediction performance can be improved from the point of view of reconstructed embedding phase space. In addition, the MDRLS-SVM algorithm is analysed in the context of noise, and we also find that the MDRLS-SVM has lower sensitivity to noise than the RLS-SVM.

Sparse representation based on projection method in online least squares support vector machines

A sparse approximation algorithm based on projection is presented in this paper in order to overcome the limitation of the non-sparsity of least squares support vector machines （LS-SVM）. The new inputs are projected into the subspace spanned by previous basis vectors （BV） and those inputs whose squared distance from the subspace is higher than a threshold are added in the BV set, while others are rejected. This consequently results in the sparse approximation. In addition, a recursive approach to deleting an exiting vector in the BV set is proposed. Then the online LS-SVM, sparse approximation and BV removal are combined to produce the sparse online LS-SVM algorithm that can control the size of memory irrespective of the processed data size. The suggested algorithm is applied in the online modeling of a pH neutralizing process and the isomerization plant of a refinery, respectively. The detailed comparison of computing time and precision is also given between the suggested algorithm and the nonsparse one. The results show that the proposed algorithm greatly improves the sparsity just with little cost of precision.

Flatness intelligent control via improved least squares support vector regression algorithm

To overcome the disadvantage that the standard least squares support vector regression(LS-SVR) algorithm is not suitable to multiple-input multiple-output(MIMO) system modelling directly,an improved LS-SVR algorithm which was defined as multi-output least squares support vector regression(MLSSVR) was put forward by adding samples' absolute errors in objective function and applied to flatness intelligent control.To solve the poor-precision problem of the control scheme based on effective matrix in flatness control,the predictive control was introduced into the control system and the effective matrix-predictive flatness control method was proposed by combining the merits of the two methods.Simulation experiment was conducted on 900HC reversible cold roll.The performance of effective matrix method and the effective matrix-predictive control method were compared,and the results demonstrate the validity of the effective matrix-predictive control method.

New predictive control algorithms based on Least Squares Support Vector Machines

Used for industrial process with different degree of nonlinearity, the two predictive control algorithms presented in this paper are based on Least Squares Support Vector Machines (LS-SVM) model. For the weakly nonlinear system, the system model is built by using LS-SVM with linear kernel function, and then the obtained linear LS-SVM model is transformed into linear input-output relation of the controlled system. However, for the strongly nonlinear system, the off-line model of the controlled system is built by using LS-SVM with Radial Basis Function (RBF) kernel. The obtained nonlinear LS-SVM model is linearized at each sampling instant of system running, after which the on-line linear input-output model of the system is built. Based on the obtained linear input-output model, the Generalized Predictive Control (GPC) algorithm is employed to implement predictive control for the controlled plant in both algorithms. The simulation results after the presented algorithms were implemented in two different industrial processes model; respectively revealed the effectiveness and merit of both algorithms.

Support Vector Machines for Regression: A Succinct Review of Large-Scale and Linear Programming Formulations

Support Vector-based learning methods are an important part of Computational Intelligence techniques. Recent efforts have been dealing with the problem of learning from very large datasets. This paper reviews the most commonly used formulations of support vector machines for regression (SVRs) aiming to emphasize its usability on large-scale applications. We review the general concept of support vector machines (SVMs), address the state-of-the-art on training methods SVMs, and explain the fundamental principle of SVRs. The most common learning methods for SVRs are introduced and linear programming-based SVR formulations are explained emphasizing its suitability for large-scale learning. Finally, this paper also discusses some open problems and current trends.

Fault diagnosis of power-shift steering transmission based on multiple outputs least squares support vector regression

A method of multiple outputs least squares support vector regression （LS-SVR） was developed and described in detail, with the radial basis function （RBF） as the kernel function. The method was applied to predict the future state of the power-shift steering transmission （PSST）. A prediction model of PSST was gotten with multiple outputs LS-SVR. The model performance was greatly influenced by the penalty parameter γ and kernel parameter σ2 which were optimized using cross validation method. The training and prediction of the model were done with spectrometric oil analysis data. The predictive and actual values were compared and a fault in the second PSST was found. The research proved that this method had good accuracy in PSST fault prediction, and any possible problem in PSST could be found through a comparative analysis.

Combination Computing of Support Vector Machine, Support Vector Regression and Molecular Docking for Potential Cytochrome P450 1A2 Inhibitors

The computational approaches of support vector machine （SVM）, support vector regression （SVR） and molecular docking were widely utilized for the computation of active compounds. In this work, to improve the accuracy and reliability of prediction, the strategy of combining the above three computational approaches was applied to predict potential cytochrome P450 1A2 （CYP1A2） inhibitors. The accuracy of the optimal SVM qualitative model was 99.432%, 97.727%, and 91.667% for training set, internal test set and external test set, respectively, showing this model had high discrimination ability. The R2 and mean square error for the optimal SVR quantitative model were 0.763, 0.013 for training set, and 0.753, 0.056 for test set respectively, indicating that this SVR model has high predictive ability for the biolog-ical activities of compounds. According to the results of the SVM and SVR models, some types of descriptors were identi ed to be essential to bioactivity prediction of compounds, including the connectivity indices, constitutional descriptors and functional group counts. Moreover, molecular docking studies were used to reveal the binding poses and binding a n-ity of potential inhibitors interacting with CYP1A2. Wherein, the amino acids of THR124 and ASP320 could form key hydrogen bond interactions with active compounds. And the amino acids of ALA317 and GLY316 could form strong hydrophobic bond interactions with active compounds. The models obtained above were applied to discover potential CYP1A2 inhibitors from natural products, which could predict the CYPs-mediated drug-drug inter-actions and provide useful guidance and reference for rational drug combination therapy. A set of 20 potential CYP1A2 inhibitors were obtained. Part of the results was consistent with references, which further indicates the accuracy of these models and the reliability of this combinatorial computation strategy.

Semi-supervised least squares support vector machine algorithm:application to offshore oil reservoir

At the early stages of deep-water oil exploration and development, fewer and further apart wells are drilled than in onshore oilfields. Supervised least squares support vector machine algorithms are used to predict the reservoir parameters but the prediction accuracy is low. We combined the least squares support vector machine （LSSVM） algorithm with semi-supervised learning and established a semi-supervised regression model, which we call the semi-supervised least squares support vector machine （SLSSVM） model. The iterative matrix inversion is also introduced to improve the training ability and training time of the model. We use the UCI data to test the generalization of a semi-supervised and a supervised LSSVM models. The test results suggest that the generalization performance of the LSSVM model greatly improves and with decreasing training samples the generalization performance is better. Moreover, for small-sample models, the SLSSVM method has higher precision than the semi-supervised K-nearest neighbor （SKNN） method. The new semi- supervised LSSVM algorithm was used to predict the distribution of porosity and sandstone in the Jingzhou study area.

Improved Scheme for Fast Approximation to Least Squares Support Vector Regression

The solution of normal least squares support vector regression(LSSVR)is lack of sparseness,which limits the real-time and hampers the wide applications to a certain degree.To overcome this obstacle,a scheme,named I2FSA-LSSVR,is proposed.Compared with the previously approximate algorithms,it not only adopts the partial reduction strategy but considers the influence between the previously selected support vectors and the willselected support vector during the process of computing the supporting weights.As a result,I2FSA-LSSVR reduces the number of support vectors and enhances the real-time.To confirm the feasibility and effectiveness of the proposed algorithm,experiments on benchmark data sets are conducted,whose results support the presented I2FSA-LSSVR.

Improved scheme to accelerate sparse least squares support vector regression

The pruning algorithms for sparse least squares support vector regression machine are common methods, and easily com- prehensible, but the computational burden in the training phase is heavy due to the retraining in performing the pruning process, which is not favorable for their applications. To this end, an im- proved scheme is proposed to accelerate sparse least squares support vector regression machine. A major advantage of this new scheme is based on the iterative methodology, which uses the previous training results instead of retraining, and its feasibility is strictly verified theoretically. Finally, experiments on bench- mark data sets corroborate a significant saving of the training time with the same number of support vectors and predictive accuracy compared with the original pruning algorithms, and this speedup scheme is also extended to classification problem.

The seam offset identification based on support vector regression machines

The principle of the support vector regression machine（SVR） is first analysed. Then the new data-dependent kernel function is constructed from information geometry perspective. The current waveforms change regularly in accordance with the different horizontal offset when the rotational frequency of the high speed rotational arc sensor is in the range from 15 Hz to 30 Hz. The welding current data is pretreated by wavelet filtering, mean filtering and normalization treatment. The SVR model is constructed by making use of the evolvement laws, the decision function can be achieved by training the SVR and the seam offset can be identified. The experimental results show that the precision of the offset identification can be greatly improved by modifying the SVR and applying mean filteringfrom the longitudinal direction.