Improved adaptive pruning algorithm for least squares support vector regression

As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satis- factory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which in- volves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generali- zation performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.

Flatness intelligent control via improved least squares support vector regression algorithm

To overcome the disadvantage that the standard least squares support vector regression(LS-SVR) algorithm is not suitable to multiple-input multiple-output(MIMO) system modelling directly,an improved LS-SVR algorithm which was defined as multi-output least squares support vector regression(MLSSVR) was put forward by adding samples' absolute errors in objective function and applied to flatness intelligent control.To solve the poor-precision problem of the control scheme based on effective matrix in flatness control,the predictive control was introduced into the control system and the effective matrix-predictive flatness control method was proposed by combining the merits of the two methods.Simulation experiment was conducted on 900HC reversible cold roll.The performance of effective matrix method and the effective matrix-predictive control method were compared,and the results demonstrate the validity of the effective matrix-predictive control method.

Fault diagnosis of power-shift steering transmission based on multiple outputs least squares support vector regression

A method of multiple outputs least squares support vector regression （LS-SVR） was developed and described in detail, with the radial basis function （RBF） as the kernel function. The method was applied to predict the future state of the power-shift steering transmission （PSST）. A prediction model of PSST was gotten with multiple outputs LS-SVR. The model performance was greatly influenced by the penalty parameter γ and kernel parameter σ2 which were optimized using cross validation method. The training and prediction of the model were done with spectrometric oil analysis data. The predictive and actual values were compared and a fault in the second PSST was found. The research proved that this method had good accuracy in PSST fault prediction, and any possible problem in PSST could be found through a comparative analysis.

Improved Scheme for Fast Approximation to Least Squares Support Vector Regression

The solution of normal least squares support vector regression(LSSVR)is lack of sparseness,which limits the real-time and hampers the wide applications to a certain degree.To overcome this obstacle,a scheme,named I2FSA-LSSVR,is proposed.Compared with the previously approximate algorithms,it not only adopts the partial reduction strategy but considers the influence between the previously selected support vectors and the willselected support vector during the process of computing the supporting weights.As a result,I2FSA-LSSVR reduces the number of support vectors and enhances the real-time.To confirm the feasibility and effectiveness of the proposed algorithm,experiments on benchmark data sets are conducted,whose results support the presented I2FSA-LSSVR.

Improved scheme to accelerate sparse least squares support vector regression

The pruning algorithms for sparse least squares support vector regression machine are common methods, and easily com- prehensible, but the computational burden in the training phase is heavy due to the retraining in performing the pruning process, which is not favorable for their applications. To this end, an im- proved scheme is proposed to accelerate sparse least squares support vector regression machine. A major advantage of this new scheme is based on the iterative methodology, which uses the previous training results instead of retraining, and its feasibility is strictly verified theoretically. Finally, experiments on bench- mark data sets corroborate a significant saving of the training time with the same number of support vectors and predictive accuracy compared with the original pruning algorithms, and this speedup scheme is also extended to classification problem.

Improved Twin Support Vector Machine Algorithm and Applications in Classification Problems

The distribution of data has a significant impact on the results of classification.When the distribution of one class is insignificant compared to the distribution of another class,data imbalance occurs.This will result in rising outlier values and noise.Therefore,the speed and performance of classification could be greatly affected.Given the above problems,this paper starts with the motivation and mathematical representing of classification,puts forward a new classification method based on the relationship between different classification formulations.Combined with the vector characteristics of the actual problem and the choice of matrix characteristics,we firstly analyze the orderly regression to introduce slack variables to solve the constraint problem of the lone point.Then we introduce the fuzzy factors to solve the problem of the gap between the isolated points on the basis of the support vector machine.We introduce the cost control to solve the problem of sample skew.Finally,based on the bi-boundary support vector machine,a twostep weight setting twin classifier is constructed.This can help to identify multitasks with feature-selected patterns without the need for additional optimizers,which solves the problem of large-scale classification that can’t deal effectively with the very low category distribution gap.

A sparse algorithm for adaptive pruning least square support vector regression machine based on global representative point ranking

Least square support vector regression(LSSVR)is a method for function approximation,whose solutions are typically non-sparse,which limits its application especially in some occasions of fast prediction.In this paper,a sparse algorithm for adaptive pruning LSSVR algorithm based on global representative point ranking(GRPR-AP-LSSVR)is proposed.At first,the global representative point ranking(GRPR)algorithm is given,and relevant data analysis experiment is implemented which depicts the importance ranking of data points.Furthermore,the pruning strategy of removing two samples in the decremental learning procedure is designed to accelerate the training speed and ensure the sparsity.The removed data points are utilized to test the temporary learning model which ensures the regression accuracy.Finally,the proposed algorithm is verified on artificial datasets and UCI regression datasets,and experimental results indicate that,compared with several benchmark algorithms,the GRPR-AP-LSSVR algorithm has excellent sparsity and prediction speed without impairing the generalization performance.

Robust least squares projection twin SVM and its sparse solution

Least squares projection twin support vector machine(LSPTSVM)has faster computing speed than classical least squares support vector machine(LSSVM).However,LSPTSVM is sensitive to outliers and its solution lacks sparsity.Therefore,it is difficult for LSPTSVM to process large-scale datasets with outliers.In this paper,we propose a robust LSPTSVM model(called R-LSPTSVM)by applying truncated least squares loss function.The robustness of R-LSPTSVM is proved from a weighted perspective.Furthermore,we obtain the sparse solution of R-LSPTSVM by using the pivoting Cholesky factorization method in primal space.Finally,the sparse R-LSPTSVM algorithm(SR-LSPTSVM)is proposed.Experimental results show that SR-LSPTSVM is insensitive to outliers and can deal with large-scale datasets fastly.

Primal least squares twin support vector regression

The training algorithm of classical twin support vector regression (TSVR) can be attributed to the solution of a pair of quadratic programming problems (QPPs) with inequality constraints in the dual space.However,this solution is affected by time and memory constraints when dealing with large datasets.In this paper,we present a least squares version for TSVR in the primal space,termed primal least squares TSVR (PLSTSVR).By introducing the least squares method,the inequality constraints of TSVR are transformed into equality constraints.Furthermore,we attempt to directly solve the two QPPs with equality constraints in the primal space instead of the dual space;thus,we need only to solve two systems of linear equations instead of two QPPs.Experimental results on artificial and benchmark datasets show that PLSTSVR has comparable accuracy to TSVR but with considerably less computational time.We further investigate its validity in predicting the opening price of stock.

Application of Least Squares Support Vector Machine for Regression to Reliability Analysis

In order to deal with the issue of huge computational cost very well in direct numerical simulation, the traditional response surface method (RSM) as a classical regression algorithm is used to approximate a functional relationship between the state variable and basic variables in reliability design. The algorithm has treated successfully some problems of implicit performance function in reliability analysis. However, its theoretical basis of empirical risk minimization narrows its range of applications for...

A Novel Method for Flatness Pattern Recognition via Least Squares Support Vector Regression

To adapt to the new requirement of the developing flatness control theory and technology, cubic patterns were introduced on the basis of the traditional linear, quadratic and quartic flatness basic patterns. Linear, quadratic, cubic and quartic Legendre orthogonal polynomials were adopted to express the flatness basic patterns. In order to over- come the defects live in the existent recognition methods based on fuzzy, neural network and support vector regres- sion （SVR） theory, a novel flatness pattern recognition method based on least squares support vector regression （LS-SVR） was proposed. On this basis, for the purpose of determining the hyper-parameters of LS-SVR effectively and enhan- cing the recognition accuracy and generalization performance of the model, particle swarm optimization algorithm with leave-one-out （LOO） error as fitness function was adopted. To overcome the disadvantage of high computational complexity of naive cross-validation algorithm, a novel fast cross-validation algorithm was introduced to calculate the LOO error of LDSVR. Results of experiments on flatness data calculated by theory and a 900HC cold-rolling mill practically measured flatness signals demonstrate that the proposed approach can distinguish the types and define the magnitudes of the flatness defects effectively with high accuracy, high speed and strong generalization ability.

Least squares twin support vector machine with asymmetric squared loss

For classification problems,the traditional least squares twin support vector machine(LSTSVM)generates two nonparallel hyperplanes directly by solving two systems of linear equations instead of a pair of quadratic programming problems(QPPs),which makes LSTSVM much faster than the original TSVM.But the standard LSTSVM adopting quadratic loss measured by the minimal distance is sensitive to noise and unstable to re-sampling.To overcome this problem,the expectile distance is taken into consideration to measure the margin between classes and LSTSVM with asymmetric squared loss(aLSTSVM)is proposed.Compared to the original LSTSVM with the quadratic loss,the proposed aLSTSVM not only has comparable computational accuracy,but also performs good properties such as noise insensitivity,scatter minimization and re-sampling stability.Numerical experiments on synthetic datasets,normally distributed clustered(NDC)datasets and University of California,Irvine(UCI)datasets with different noises confirm the great performance and validity of our proposed algorithm.

基于LSTSVR模型的边缘计算预测变压器平均油温及绕组热点温度

变压器绕组的热点温度过高,会导致变压器绝缘脆解、裂化甚至击穿短路。因此及时、准确地预测出变压器绕组的热点温度,对提高变压器运行的安全可靠性至关重要。利用最小二乘双支持向量回归机(LSTSVR)作为边缘计算模型,将变压器油中气体色谱分析数据信息与变压器负载电流、环境温度、顶层油温、上死角温度等变压器运行信息结合,构建监测系统架构,预测变压器的平均油温,并计算出绕组热点温度。将所提方法得到的数据与实测数据进行对比,结果利用LSTSVR模型实现了变压器平均油温及绕组热点温度的准确预测,且该模型的预测精度优于最小二乘支持向量回归机模型,有效地提高了绕组热点温度测量的精度。现场实例也证明了所提方法的有效性和可靠性。

基于LSTSVR模型预测STATCOM晶闸管阀组本体温度

STATCOM晶闸管阀组本体温度过高,会导致其失效。因此及时、准确地预测出STATCOM晶闸管阀组本体温度对提高STATCOM运行的可靠性至关重要。本文利用最小二乘双支持向量回归机(least square twin support vector regression,LSTSVR)算法,将STATCOM进水温度、回水温度、进水流量、IGBT模块散热材料的导热系数、STATCOM输出电压、STATCOM输出电流、晶闸管阀组的集电极电流共7个量作为输入量,构建了STATCOM晶闸管阀组本体温度预测模型。与现场实测数据对比的结果表明,利用LSTSVR模型实现了STATCOM晶闸管阀组本体温度的高精度预测,且模型的预测精度优于最小二乘支持向量回归机(least square support vector regression,LSSVR)模型。应用实例也验证了该方法的准确性和有效性。

基于QCSSA-LSTSVR的氧化铝质量预测模型

针对氧化铝焙烧过程氧化铝质量指标检测滞后的问题,提出融合最小二乘孪生支持向量机(LSTSVR)与量子混沌樽海鞘算法(QCSSA)方法,建立一种氧化铝焙烧过程的质量指标预测模型。首先,利用LSTSVR建立氧化铝质量指标预测模型;其次,针对LSTSVR模型中核宽度系数和惩罚因子选取困难的问题,采用QCSSA进行LSTSVR模型结构参数寻优,利用Logistic混沌策略和量子局部搜索策略来提高SSA的全局寻优能力;最后,利用实际生产数据对所提方法进行实验验证。仿真结果表明,QCSSA优化LSTSVR的方法具有较好的预测效果。

基于LSTSVR的路基沉降组合预测模型

鉴于路基沉降各种单相预测模型均有其适用范围,总体预测波动性较大,精度较低,提出基于最小二乘双支持向量回归机(LSTSVR,least square twin support vector regression)的路基沉降组合预测模型。该模型的核心是根据路基沉降的发展规律及其沉降曲线的特点,选择具有S型特点的成长曲线特征的单相预测模型;以各单项预测模型预测结果作为最小二乘双支持向量回归机的输入向量,构建路基沉降组合预测模型。对比试验表明:提出方法具有更好的预测精度和稳定性。

Quadratic Kernel-Free Least Square Twin Support Vector Machine for Binary Classification Problems

In this paper,a new quadratic kernel-free least square twin support vector machine(QLSTSVM)is proposed for binary classification problems.The advantage of QLSTSVM is that there is no need to select the kernel function and related parameters for nonlinear classification problems.After using consensus technique,we adopt alternating direction method of multipliers to solve the reformulated consensus QLSTSVM directly.To reduce CPU time,the Karush-Kuhn-Tucker(KKT)conditions is also used to solve the QLSTSVM.The performance of QLSTSVM is tested on two artificial datasets and several University of California Irvine(UCI)benchmark datasets.Numerical results indicate that the QLSTSVM may outperform several existing methods for solving twin support vector machine with Gaussian kernel in terms of the classification accuracy and operation time.

End-point dynamic control of basic oxygen furnace steelmaking based on improved unconstrained twin support vector regression

In order to improve the end-point hit rate of basic oxygen furnace steelmaking,a novel dynamic control model was proposed based on an improved twin support vector regression algorithm.The controlled objects were the end-point carbon content and temperature.The proposed control model was established by using the low carbon steel samples collected from a steel plant,which consists of two prediction models,a preprocess model,two regulation units,a controller and a basic oxygen furnace.The test results of 100 heats show that the prediction models can achieve a double hit rate of 90%within the error bound of 0.005 wt.%C and 15℃.The preprocess model was used to predict an initial end-blow oxygen volume.However,the double hit rate of the carbon con tent and temperature only achieves 65%.Then,the oxygen volume and coolant additi ons were adjusted by the regulation units to improve the hit rate.Finally,the double hit rate after the regulation is reached up to 90%.The results indicate that the proposed dynamic control model is efficient to guide the real production for low carbon steel,and the modeling method is also suitable for the applications of other steel grades.

Short Term Electric Load Prediction by Incorporation of Kernel into Features Extraction Regression Technique

Accurate load prediction plays an important role in smart power management system, either for planning, facing the increasing of load demand, maintenance issues, or power distribution system. In order to achieve a reasonable prediction, authors have applied and compared two features extraction technique presented by kernel partial least square regression and kernel principal component regression, and both of them are carried out by polynomial and Gaussian kernels to map the original features’ to high dimension features’ space, and then draw new predictor variables known as scores and loadings, while kernel principal component regression draws the predictor features to construct new predictor variables without any consideration to response vector. In contrast, kernel partial least square regression does take the response vector into consideration. Models are simulated by three different cities’ electric load data, which used historical load data in addition to weekends and holidays as common predictor features for all models. On the other hand temperature has been used for only one data as a comparative study to measure its effect. Models’ results evaluated by three statistic measurements, show that Gaussian Kernel Partial Least Square Regression offers the more powerful features and significantly can improve the load prediction performance than other presented models.

基于高光谱成像技术的涌泉蜜桔糖度最优检测位置

本文旨在探索涌泉蜜桔糖度的最优检测位置和最佳预测模型,以便为蜜桔糖度检测分级提供理论依据。本文利用波长为390.2~981.3 nm的高光谱成像系统对涌泉蜜桔糖度最佳检测位置进行研究,将涌泉蜜桔的花萼、果茎、赤道和全局的光谱信息与其对应部位的糖度结合,建立其预测模型。使用标准正态变量变换(SNV)、多元散射校正(MSC)、基线校准(Baseline)和SG平滑(Savitzkv-Golay)4种预处理方法对不同部位的原始光谱进行预处理,用预处理后的光谱数据建立偏最小二乘回归(PLSR)和最小二乘支持向量机(LSSVM)模型。找出蜜桔不同部位的最佳预处理方式,对经过最佳预处理后的光谱数据采用竞争性自适应重加权算法(CARS)和无信息变量消除法(UVE)进行特征波长筛选。最后,用筛选后的光谱数据建立PLSR和LSSVM模型并进行分析比较。研究结果表明,全局的MSC-CARS-LSSVM模型预测效果最佳,其预测集相关系数Rp=0.955,均方根误差RMSEP=0.395,其次是蜜桔赤道部位的SNV-PLSR模型,其预测集相关系数Rp=0.936,均方根误差RMSEP=0.37。两者预测集相关系数相近,因此可将赤道位置作为蜜桔糖度的最优检测位置。本研究表明根据蜜桔不同部位建立的糖度预测模型的预测效果有所差异,研究最优检测位置和最佳预测模型可以为蜜桔进行糖度检测分级提供理论依据。