Improved adaptive pruning algorithm for least squares support vector regression

As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satis- factory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which in- volves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generali- zation performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.

Fault diagnosis of power-shift steering transmission based on multiple outputs least squares support vector regression

A method of multiple outputs least squares support vector regression （LS-SVR） was developed and described in detail, with the radial basis function （RBF） as the kernel function. The method was applied to predict the future state of the power-shift steering transmission （PSST）. A prediction model of PSST was gotten with multiple outputs LS-SVR. The model performance was greatly influenced by the penalty parameter γ and kernel parameter σ2 which were optimized using cross validation method. The training and prediction of the model were done with spectrometric oil analysis data. The predictive and actual values were compared and a fault in the second PSST was found. The research proved that this method had good accuracy in PSST fault prediction, and any possible problem in PSST could be found through a comparative analysis.

Flatness intelligent control via improved least squares support vector regression algorithm

To overcome the disadvantage that the standard least squares support vector regression(LS-SVR) algorithm is not suitable to multiple-input multiple-output(MIMO) system modelling directly,an improved LS-SVR algorithm which was defined as multi-output least squares support vector regression(MLSSVR) was put forward by adding samples' absolute errors in objective function and applied to flatness intelligent control.To solve the poor-precision problem of the control scheme based on effective matrix in flatness control,the predictive control was introduced into the control system and the effective matrix-predictive flatness control method was proposed by combining the merits of the two methods.Simulation experiment was conducted on 900HC reversible cold roll.The performance of effective matrix method and the effective matrix-predictive control method were compared,and the results demonstrate the validity of the effective matrix-predictive control method.

Improved Scheme for Fast Approximation to Least Squares Support Vector Regression

The solution of normal least squares support vector regression(LSSVR)is lack of sparseness,which limits the real-time and hampers the wide applications to a certain degree.To overcome this obstacle,a scheme,named I2FSA-LSSVR,is proposed.Compared with the previously approximate algorithms,it not only adopts the partial reduction strategy but considers the influence between the previously selected support vectors and the willselected support vector during the process of computing the supporting weights.As a result,I2FSA-LSSVR reduces the number of support vectors and enhances the real-time.To confirm the feasibility and effectiveness of the proposed algorithm,experiments on benchmark data sets are conducted,whose results support the presented I2FSA-LSSVR.

Improved scheme to accelerate sparse least squares support vector regression

The pruning algorithms for sparse least squares support vector regression machine are common methods, and easily com- prehensible, but the computational burden in the training phase is heavy due to the retraining in performing the pruning process, which is not favorable for their applications. To this end, an im- proved scheme is proposed to accelerate sparse least squares support vector regression machine. A major advantage of this new scheme is based on the iterative methodology, which uses the previous training results instead of retraining, and its feasibility is strictly verified theoretically. Finally, experiments on bench- mark data sets corroborate a significant saving of the training time with the same number of support vectors and predictive accuracy compared with the original pruning algorithms, and this speedup scheme is also extended to classification problem.

Cloud removal of remote sensing image based on multi-output support vector regression

Removal of cloud cover on the satellite remote sensing image can effectively improve the availability of remote sensing images. For thin cloud cover, support vector value contourlet transform is used to achieve multi-scale decomposition of the area of thin cloud cover on remote sensing images. Through enhancing coefficients of high frequency and suppressing coefficients of low frequency, the thin cloud is removed. For thick cloud cover, if the areas of thick cloud cover on multi-source or multi-temporal remote sensing images do not overlap, the multi-output support vector regression learning method is used to remove this kind of thick clouds. If the thick cloud cover areas overlap, by using the multi-output learning of the surrounding areas to predict the surface features of the overlapped thick cloud cover areas, this kind of thick cloud is removed. Experimental results show that the proposed cloud removal method can effectively solve the problems of the cloud overlapping and radiation difference among multi-source images. The cloud removal image is clear and smooth.

A sparse algorithm for adaptive pruning least square support vector regression machine based on global representative point ranking

Least square support vector regression(LSSVR)is a method for function approximation,whose solutions are typically non-sparse,which limits its application especially in some occasions of fast prediction.In this paper,a sparse algorithm for adaptive pruning LSSVR algorithm based on global representative point ranking(GRPR-AP-LSSVR)is proposed.At first,the global representative point ranking(GRPR)algorithm is given,and relevant data analysis experiment is implemented which depicts the importance ranking of data points.Furthermore,the pruning strategy of removing two samples in the decremental learning procedure is designed to accelerate the training speed and ensure the sparsity.The removed data points are utilized to test the temporary learning model which ensures the regression accuracy.Finally,the proposed algorithm is verified on artificial datasets and UCI regression datasets,and experimental results indicate that,compared with several benchmark algorithms,the GRPR-AP-LSSVR algorithm has excellent sparsity and prediction speed without impairing the generalization performance.

Primal least squares twin support vector regression

The training algorithm of classical twin support vector regression (TSVR) can be attributed to the solution of a pair of quadratic programming problems (QPPs) with inequality constraints in the dual space.However,this solution is affected by time and memory constraints when dealing with large datasets.In this paper,we present a least squares version for TSVR in the primal space,termed primal least squares TSVR (PLSTSVR).By introducing the least squares method,the inequality constraints of TSVR are transformed into equality constraints.Furthermore,we attempt to directly solve the two QPPs with equality constraints in the primal space instead of the dual space;thus,we need only to solve two systems of linear equations instead of two QPPs.Experimental results on artificial and benchmark datasets show that PLSTSVR has comparable accuracy to TSVR but with considerably less computational time.We further investigate its validity in predicting the opening price of stock.

A Novel Method for Flatness Pattern Recognition via Least Squares Support Vector Regression

To adapt to the new requirement of the developing flatness control theory and technology, cubic patterns were introduced on the basis of the traditional linear, quadratic and quartic flatness basic patterns. Linear, quadratic, cubic and quartic Legendre orthogonal polynomials were adopted to express the flatness basic patterns. In order to over- come the defects live in the existent recognition methods based on fuzzy, neural network and support vector regres- sion （SVR） theory, a novel flatness pattern recognition method based on least squares support vector regression （LS-SVR） was proposed. On this basis, for the purpose of determining the hyper-parameters of LS-SVR effectively and enhan- cing the recognition accuracy and generalization performance of the model, particle swarm optimization algorithm with leave-one-out （LOO） error as fitness function was adopted. To overcome the disadvantage of high computational complexity of naive cross-validation algorithm, a novel fast cross-validation algorithm was introduced to calculate the LOO error of LDSVR. Results of experiments on flatness data calculated by theory and a 900HC cold-rolling mill practically measured flatness signals demonstrate that the proposed approach can distinguish the types and define the magnitudes of the flatness defects effectively with high accuracy, high speed and strong generalization ability.

Short Term Electric Load Prediction by Incorporation of Kernel into Features Extraction Regression Technique

Accurate load prediction plays an important role in smart power management system, either for planning, facing the increasing of load demand, maintenance issues, or power distribution system. In order to achieve a reasonable prediction, authors have applied and compared two features extraction technique presented by kernel partial least square regression and kernel principal component regression, and both of them are carried out by polynomial and Gaussian kernels to map the original features’ to high dimension features’ space, and then draw new predictor variables known as scores and loadings, while kernel principal component regression draws the predictor features to construct new predictor variables without any consideration to response vector. In contrast, kernel partial least square regression does take the response vector into consideration. Models are simulated by three different cities’ electric load data, which used historical load data in addition to weekends and holidays as common predictor features for all models. On the other hand temperature has been used for only one data as a comparative study to measure its effect. Models’ results evaluated by three statistic measurements, show that Gaussian Kernel Partial Least Square Regression offers the more powerful features and significantly can improve the load prediction performance than other presented models.

基于高光谱成像技术的涌泉蜜桔糖度最优检测位置

本文旨在探索涌泉蜜桔糖度的最优检测位置和最佳预测模型,以便为蜜桔糖度检测分级提供理论依据。本文利用波长为390.2~981.3 nm的高光谱成像系统对涌泉蜜桔糖度最佳检测位置进行研究,将涌泉蜜桔的花萼、果茎、赤道和全局的光谱信息与其对应部位的糖度结合,建立其预测模型。使用标准正态变量变换(SNV)、多元散射校正(MSC)、基线校准(Baseline)和SG平滑(Savitzkv-Golay)4种预处理方法对不同部位的原始光谱进行预处理,用预处理后的光谱数据建立偏最小二乘回归(PLSR)和最小二乘支持向量机(LSSVM)模型。找出蜜桔不同部位的最佳预处理方式,对经过最佳预处理后的光谱数据采用竞争性自适应重加权算法(CARS)和无信息变量消除法(UVE)进行特征波长筛选。最后,用筛选后的光谱数据建立PLSR和LSSVM模型并进行分析比较。研究结果表明,全局的MSC-CARS-LSSVM模型预测效果最佳,其预测集相关系数Rp=0.955,均方根误差RMSEP=0.395,其次是蜜桔赤道部位的SNV-PLSR模型,其预测集相关系数Rp=0.936,均方根误差RMSEP=0.37。两者预测集相关系数相近,因此可将赤道位置作为蜜桔糖度的最优检测位置。本研究表明根据蜜桔不同部位建立的糖度预测模型的预测效果有所差异,研究最优检测位置和最佳预测模型可以为蜜桔进行糖度检测分级提供理论依据。

基于近红外光谱技术结合ARO-LSSVR的天麻中有效成分含量快速检测

为实现对天麻中天麻素和对羟基苯甲醇含量的快速、无损检测,以云南昭通乌天麻为实验对象,采集900~1 700 nm波长范围内的光谱数据。首先,采用卷积平滑和标准正态变量变换进行光谱数据预处理,其次通过竞争性自适应重加权采样法(competitive adapative reweighted sampling,CARS)与迭代保留信息变量算法进行特征波长的提取,根据基于特征波长建立最小二乘支持向量回归(least squares support vector machine,LSSVR)模型的结果,选择最佳特征波长提取方法。为了提高模型的准确率,本研究引入人工兔智能算法对LSSVR中的正则化参数γ和核函数密度σ2进行优化,并与粒子群优化算法(particle swarm optimization,PSO)、灰狼优化算法(grey wolf optimizer,GWO)进行对比,评估人工兔优化算法(artificial rabbits optimization,ARO)的优越性。结果表明,ARO算法在寻优速度、寻优能力上优于PSO、GWO;天麻素、对羟基苯甲醇的最佳预测模型均为CARS-AROLSSVR,其Rp2分别为0.969 6和0.957 7,预测均方根误差分别为0.014和0.020。综上,近红外光谱可用于天麻中有效成分的定量检测,本研究可为天麻快速检测装置的研发提供理论依据。

基于PSO-LSSVR的机器人磨抛材料去除模型

为了建立磨抛工艺参数与材料去除深度的关系,建立一种基于最小二乘法支持向量回归机(LSSVR)的材料去除深度预测模型,并引入粒子群优化(PSO)算法来优化LSSVR的超参数,可提高LSSVR模型的预测准确性和全局优寻能力。搭建叶片机器人砂带磨抛实验平台,设计并进行多工艺参数实验,考虑工艺参数:砂带粒度、砂带转速、进给速度、接触力和叶片表面曲率半径,获得叶片表面的材料去除深度,最终利用实验数据建立了PSO-LSSVR叶片材料去除深度预测模型。结果表明,PSO-LSSVR模型的预测准确率为95.37%,平均预测误差为0.003463,说明PSO-LSSVR模型具有较高的预测精度,并结合实际加工情况进行实验验证可行性,证明PSO-LSSVR模型可以有效合理地建立工艺参数与材料去除深度的关系。

可见-近红外与中红外光谱预测土壤养分的比较研究

对土壤养分的快速和准确测定有助于适时指导施肥。为进一步研究可见-近红外(350~2500 nm)与中红外光谱(4000~650 cm^(–1))对土壤养分的预测能力,以贵州省500个土样为例,对光谱进行Savitzky-Golay(SG)平滑去噪处理,再用标准正态化(SNV)方法进行基线校正,然后分别应用偏最小二乘回归(PLSR)和支持向量机(SVM)两种方法进行建模,探讨了可见-近红外和中红外光谱对土壤全氮(TN)、全磷(TP)、全钾(TK)和碱解氮(AN)、有效磷(AP)、速效钾(AK)共六种土壤养分的预测效果。结果表明:(1)无论基于可见-近红外光谱还是中红外光谱,PLSR模型的预测精度整体均优于SVM模型。(2)中红外光谱对TN、TK和AN的预测精度均显著高于可见-近红外光谱,可见-近红外和中红外光谱均可以可靠地预测TN和TK(性能与四分位间隔距离的比率(RPIQ)大于2.10),中红外光谱可相对较可靠地预测AN(RPIQ=1.87);但两类光谱对TP、AP和AK的预测效果均较差(RPIQ<1.34)。(3)当变量投影重要性得分(VIP)大于1.5时,PLSR模型在中红外光谱区域预测TN和TK的重要波段多于可见-近红外光谱区域,TN的重要波段主要集中于可见-近红外光谱区域的1910和2207 nm附近,中红外光谱区域的1120、1000、960、910、770和668 cm^(–1)附近;TK的重要波段主要集中于可见-近红外光谱区域的540、2176、2225和2268 nm附近,中红外光谱区域的1040、960、910、776、720和668 cm^(–1)附近。因此,中红外光谱技术结合PLSR模型对土壤养分预测效果较好,可快速准确预测土壤TN和TK,可为指导适时施肥提供技术支撑。

基于慢特征分析与最小二乘支持向量回归集成的草酸钴合成过程粒度预报

草酸钴合成过程是钴湿法冶炼的关键单元操作,其粒度分布是重要的质量指标,然而难以在线实时测量。同时,草酸钴合成过程通常存在非线性、多约束和慢时变特征。因此,提出一种集成慢特征分析(slow feature analysis,SFA)与最小二乘支持向量回归(least square support vector regression,LSSVR)的草酸钴粒度预报模型对草酸钴合成过程质量指标实现在线测量。在该方法中,首先,SFA方法可以有效地捕获过程的慢特征向量,解决慢时变问题;然后,利用LSSVR方法建立慢特征与粒度之间的非线性关系模型,进而实现质量指标在线预报。最后,应用非线性的数值案例以及草酸钴合成过程数据,验证该方法的有效性。实验结果显示:相较于单一的径向基函数神经网络(radial basis function neural network,RBFNN)、LSSVR预测模型以及SFA与NN相结合的预报模型,所提方法在数值案例中的预测精度分别提升了13.31%、2.26%、1.72%;在草酸钴合成过程中的预测精度分别提升了13.27%、9.96%、8.92%。

基于MLSSVR数据驱动的潮流非线性回归及其灵敏度解析

潮流计算及其灵敏度分析是电力系统稳态分析与控制的基础。传统基于模型驱动的潮流计算是在电网拓扑和模型参数完备条件下,通过构建节点功率非线性方程并采用迭代方式进行求解的,灵敏度则由潮流雅可比矩阵求逆获取。模型及参数的准确性和迭代求解的时效性是影响潮流计算精度和速度的重要因素。该文提出一种数据驱动的潮流非线性回归及灵敏度解析计算方法,以实现不依赖于电网物理模型的潮流快速计算与分析。首先,利用电网潮流量测数据,构建基于改进的多输出最小二乘支持向量回归(multi-output least-squares support vector regression,MLSSVR)的潮流显式回归模型;其次,通过矩阵快速递归求逆,提出MLSSVR在线学习方法,增强对电网运行场景变化的适应性;最后,对潮流回归模型进行泰勒展开,提出潮流灵敏度解析计算方法。所提方法在多个IEEE标准系统和某实际省级电网进行仿真,验证了所提方法可有效得到高准确度的潮流解及其灵敏度。

基于高光谱技术的场地土壤重金属污染快速调查研究

为了准确预测场地土壤重金属分布状况,实现土壤重金属污染快速调查,以某废弃助剂厂填埋区土壤为研究对象,基于高光谱数据,利用单变量回归模型、偏最小二乘回归模型和支持向量机模型估算土壤重金属Cr、Ni、Cu、Zn、Cd、Pb、As和Hg的含量。结果表明,土壤光谱反射率与各重金属含量均呈负相关;偏最小二乘回归模型和支持向量机模型对8种重金属的预测精度均优于单变量回归模型,偏最小二乘回归模型为Cd、Pb、Cr和Ni的最佳估算模型,支持向量机模型为Cu、As、Zn和Hg的最佳预测模型;研究区土壤重金属反演结果在趋势上与实验室分析结果基本一致,高值区和极值点分布亦较为吻合,能够圈定存在重金属污染风险的区域,同时提供技术支撑,实现场地土壤重金属污染的快速调查。

利用近红外光谱技术检测固定化脂肪酶酶蛋白含量

利用近红外光谱技术检测固定化脂肪酶的酶蛋白含量,在获得近红外光谱数据后,分别用一阶导数、二阶导数、标准正态变量变换和多元散射校正4种不同预处理方法进行处理,获得最佳的光谱处理方法为标准正态变量变换。利用连续投影算法、竞争性自适应重加权算法、无信息变量消除算法3种不同的波长筛选算法获得特征波长,在此基础上分别用偏最小二乘法、支持向量机回归和BP神经网络3种建模方法建立近红外光谱模型,最终获得最佳的建模方法为标准正态变量变换-支持向量机回归,在此建模条件下,模型决定系数为0.9894,模型均方根误差为0.3178,模型交叉验证决定系数为0.9764,模型交叉验证均方根误差为0.3816,相对分析误差可达6.51。

基于改进哈里斯鹰优化算法的光谱特征波段选择模型研究

特征波段选择是近红外光谱分析的关键步骤之一,有效的特征波段选择能提高建模效率与模型性能。传统的特征波段选择算法存在运行时间长、选择特征冗余的缺陷,在实际工程应用中难以达到期望的效果。哈里斯鹰优化(HHO)算法具有原理简单、参数少的优点,但同时也存在收敛精度低且易陷入局部最优的不足。在HHO算法的基础上提出了一种基于改进哈里斯鹰优化(IHHO)算法的近红外光谱特征波段选择模型。针对HHO算法只能用于求解连续空间的优化问题,采用离散化策略对HHO算法进行修正,使其能求解离散形式的特征波段选择问题;考虑到HHO算法初始种群的质量差,使用混沌映射、反向学习提高初始种群的质量,以增强算法的全局探索能力;由于HHO算法在局部搜索时的收敛精度低,提出了新的猎物能量衰减模型与跳跃策略,以进一步增强算法在局部搜索时的寻优能力;为避免算法在寻优过程中落入局部最优,借鉴了遗传算法的变异方式对HHO算法进行扰动。使用竞争性自适应重加权采样法(CARS)、连续投影算法(SPA)、粒子群优化(PSO)算法、遗传算法(GA)、 HHO算法与IHHO算法进行比较,并以4个定性分析近红外光谱数据集与2个定量分析近红外光谱数据集分别建立了支持向量机(SVM)识别模型和偏最小二乘回归(PLSR)模型。在定性分析实验中,IHHO算法得到的平均准确率相对于全波段时分别提高了0.83%、 9.55%、 17.65%以及0%,平均特征波段数仅占全波段的9.97%、 2.59%、 1.36%以及0.59%。在定量分析实验中,IHHO算法得到的平均决定系数分别较全波段提高了10.57%、 1.47%、 4.41%、 3.66%以及3.06%,平均均方根误差分别较全波段较低了0.162、 1.266 3、 1.868、 1.869 4以及0.408 4,平均特征波段数仅占全波段的9.24%、 10.53%、 6.54%、 6.91%以及7.14%。实验结果表明,IHHO算法在选择特征波段时能够去冗余,针对性选择最重要的特征波段,其性能均优于比较的几种算法。IHHO算法具有良好的应用前景。

基于相似日聚类及模态分解的短期光伏发电功率组合预测研究

短期光伏发电功率预测是电站制定发电计划并进行调度的重要组成部分,有助于电力系统的动态稳定。针对光伏时序预测中存在的噪声干扰及单一模型预测效果不稳定等问题,该文提出一种基于改进型自适应白噪声的完全集合经验模态分解(improved complete ensemble empirical mode decomposition with adaptive noise,ICEEMDAN)的组合预测模型。首先,利用相关系数提取重要气象特征,并采用模糊C均值聚类(fuzzy C-means clustering,FCM)将原始数据集划分为晴天、晴转多云和雨天。其次,每种相似日以最后一天为待预测日,其余为历史训练集;利用ICEEMDAN将历史训练集分解成若干个较为规律的子序列,并用排列熵(permutation entropy,PE)对各子序列进行重构。最后,高频项采用由卷积神经网络(convolutional neural network,CNN)、(bidirectional gated recurrent unit,Bi GRU)双向门控循环单元与注意力机制组合而成的CNN-BiGRU-ATTENTION神经网络预测,低频项和趋势项采用最小二乘支持向量回归机(least squares support vector regression,LSSVR)进行预测,将预测结果叠加得到最终光伏发电功率预测值。通过实例验证:该文组合模型在不同天气条件下,可以解决单一模型预测精度低、预测效果不稳定等问题;相比其他模态分解,能够更精确地预测波动较大的局部特征。