Learning Rates for Least Square Regressions with Coefficient Regularization

We analyze the learning rates for the least square regression with data dependent hy- pothesis spaces and coefficient regularization algorithms based on general kernels. Under a very mild regularity condition on the regression function, we obtain a bound for the approximation error by esti- mating the corresponding K：-functional. Combining this estimate with the previous result of the sample error, we derive a dimensional free learning rate by the proper choice of the regularization parameter.

Spatter Rate Estimation of GMAW-S based on Partial Least Square Regression

This paper analyzes the drop transfer process in gas metal arc welding in short-circuit transfer mode （GMAW-S） in order to develop an optimized spatter rate model that can be used on line. According to thermodynamic characters and practical behavior, a complete arcing process is divided into three sub-processes： arc re-ignition, energy output and shorting preparation. Shorting process is then divided as drop spread, bridge sustention and bridge destabilization. Nine process variables and their distribution are analyzed based on welding experiments with high-speed photos and synchronous current and voltage signals. Method of variation coefficient is used to reflect process consistency and to design characteristic parameters. Partial least square regression （PLSR） is utilized to set up spatter rate model because of severe correlativity among the above characteristic parameters. PLSR is a new multivariate statistical analysis method, in which regression modeling, data simplification and relativity analysis are included in a single algorithm. Experiment results show that the regression equation based on PLSR is effective for on-line predicting spatter rate of its corresponding welding condition.

Discrimination of Transgenic Rice Based on Near Infrared Reflectance Spectroscopy and Partial Least Squares Regression Discriminant Analysis

Near infrared reflectance spectroscopy （NIRS）, a non-destructive measurement technique, was combined with partial least squares regression discrimiant analysis （PLS-DA） to discriminate the transgenic （TCTP and mi166） and wild type （Zhonghua 11） rice. Furthermore, rice lines transformed with protein gene （OsTCTP） and regulation gene （Osmi166） were also discriminated by the NIRS method. The performances of PLS-DA in spectral ranges of 4 000-8 000 cm-1 and 4 000-10 000 cm-1 were compared to obtain the optimal spectral range. As a result, the transgenic and wild type rice were distinguished from each other in the range of 4 000-10 000 cm-1, and the correct classification rate was 100.0% in the validation test. The transgenic rice TCTP and mi166 were also distinguished from each other in the range of 4 000-10 000 cm-1, and the correct classification rate was also 100.0%. In conclusion, NIRS combined with PLS-DA can be used for the discrimination of transgenic rice.

Estimating Wheat Grain Protein Content Using Multi-Temporal Remote Sensing Data Based on Partial Least Squares Regression

Estimating wheat grain protein content by remote sensing is important for assessing wheat quality at maturity and making grains harvest and purchase policies. However, spatial variability of soil condition, temperature, and precipitation will affect grain protein contents and these factors usually cannot be monitored accurately by remote sensing data from single image. In this research, the relationships between wheat protein content at maturity and wheat agronomic parameters at different growing stages were analyzed and multi-temporal images of Landsat TM were used to estimate grain protein content by partial least squares regression. Experiment data were acquired in the suburb of Beijing during a 2-yr experiment in the period from 2003 to 2004. Determination coefficient, average deviation of self-modeling, and deviation of cross- validation were employed to assess the estimation accuracy of wheat grain protein content. Their values were 0.88, 1.30%, 3.81% and 0.72, 5.22%, 12.36% for 2003 and 2004, respectively. The research laid an agronomic foundation for GPC （grain protein content） estimation by multi-temporal remote sensing. The results showed that it is feasible to estimate GPC of wheat from multi-temporal remote sensing data in large area.

Improved adaptive pruning algorithm for least squares support vector regression

As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satis- factory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which in- volves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generali- zation performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.

Flatness intelligent control via improved least squares support vector regression algorithm

To overcome the disadvantage that the standard least squares support vector regression(LS-SVR) algorithm is not suitable to multiple-input multiple-output(MIMO) system modelling directly,an improved LS-SVR algorithm which was defined as multi-output least squares support vector regression(MLSSVR) was put forward by adding samples' absolute errors in objective function and applied to flatness intelligent control.To solve the poor-precision problem of the control scheme based on effective matrix in flatness control,the predictive control was introduced into the control system and the effective matrix-predictive flatness control method was proposed by combining the merits of the two methods.Simulation experiment was conducted on 900HC reversible cold roll.The performance of effective matrix method and the effective matrix-predictive control method were compared,and the results demonstrate the validity of the effective matrix-predictive control method.

Partial Least Squares Regression Model to Predict Water Quality in Urban Water Distribution Systems

The water distribution system of one residential district in Tianjin is taken as an example to analyze the changes of water quality.Partial least squares(PLS) regression model,in which the turbidity and Fe are regarded as control objectives,is used to establish the statistical model.The experimental results indicate that the PLS regression model has good predicted results of water quality compared with the monitored data.The percentages of absolute relative error(below 15%,20%,30%) are 44.4%,66.7%,100%(turbidity) and 33.3%,44.4%,77.8%(Fe) on the 4th sampling point;77.8%,88.9%,88.9%(turbidity) and 44.4%,55.6%,66.7%(Fe) on the 5th sampling point.

Fault diagnosis of power-shift steering transmission based on multiple outputs least squares support vector regression

A method of multiple outputs least squares support vector regression （LS-SVR） was developed and described in detail, with the radial basis function （RBF） as the kernel function. The method was applied to predict the future state of the power-shift steering transmission （PSST）. A prediction model of PSST was gotten with multiple outputs LS-SVR. The model performance was greatly influenced by the penalty parameter γ and kernel parameter σ2 which were optimized using cross validation method. The training and prediction of the model were done with spectrometric oil analysis data. The predictive and actual values were compared and a fault in the second PSST was found. The research proved that this method had good accuracy in PSST fault prediction, and any possible problem in PSST could be found through a comparative analysis.

A sparse algorithm for adaptive pruning least square support vector regression machine based on global representative point ranking

Least square support vector regression(LSSVR)is a method for function approximation,whose solutions are typically non-sparse,which limits its application especially in some occasions of fast prediction.In this paper,a sparse algorithm for adaptive pruning LSSVR algorithm based on global representative point ranking(GRPR-AP-LSSVR)is proposed.At first,the global representative point ranking(GRPR)algorithm is given,and relevant data analysis experiment is implemented which depicts the importance ranking of data points.Furthermore,the pruning strategy of removing two samples in the decremental learning procedure is designed to accelerate the training speed and ensure the sparsity.The removed data points are utilized to test the temporary learning model which ensures the regression accuracy.Finally,the proposed algorithm is verified on artificial datasets and UCI regression datasets,and experimental results indicate that,compared with several benchmark algorithms,the GRPR-AP-LSSVR algorithm has excellent sparsity and prediction speed without impairing the generalization performance.

Simultaneous Spectrophotometric Determination of Three Components Including Deoxyschizandrin by Partial Least Squares Regression

The computer auxiliary partial least squares is introduced to simultaneously determine the contents of Deoxyschizandin, Schisandrin, r-Schisandrin in the extracted solution of wuweizi. Regression analysis of the experimental results shows that the average recovery of each component is all in the range from 98.9% to 110.3% , which means the partial least squares regression spectrophotometry can circumvent the overlappirtg of absorption spectrums of mlulti-components, so that sctisfactory results can be obtained without any scrapple pre-separation.

Quantum partial least squares regression algorithm for multiple correlation problem

Partial least squares(PLS) regression is an important linear regression method that efficiently addresses the multiple correlation problem by combining principal component analysis and multiple regression. In this paper, we present a quantum partial least squares(QPLS) regression algorithm. To solve the high time complexity of the PLS regression, we design a quantum eigenvector search method to speed up principal components and regression parameters construction. Meanwhile, we give a density matrix product method to avoid multiple access to quantum random access memory(QRAM)during building residual matrices. The time and space complexities of the QPLS regression are logarithmic in the independent variable dimension n, the dependent variable dimension w, and the number of variables m. This algorithm achieves exponential speed-ups over the PLS regression on n, m, and w. In addition, the QPLS regression inspires us to explore more potential quantum machine learning applications in future works.

Improved scheme to accelerate sparse least squares support vector regression

The pruning algorithms for sparse least squares support vector regression machine are common methods, and easily com- prehensible, but the computational burden in the training phase is heavy due to the retraining in performing the pruning process, which is not favorable for their applications. To this end, an im- proved scheme is proposed to accelerate sparse least squares support vector regression machine. A major advantage of this new scheme is based on the iterative methodology, which uses the previous training results instead of retraining, and its feasibility is strictly verified theoretically. Finally, experiments on bench- mark data sets corroborate a significant saving of the training time with the same number of support vectors and predictive accuracy compared with the original pruning algorithms, and this speedup scheme is also extended to classification problem.

Based on Partial Least-squares Regression to Build up and Analyze the Model of Rice Evapotranspiration

During the course of calculating the rice evapotranspiration using weather factors,we often find that some independent variables have multiple correlation.The phenomena can lead to the traditional multivariate regression model which based on least square method distortion.And the stability of the model will be lost.The model will be built based on partial least square regression in the paper,through applying the idea of main component analyze and typical correlation analyze,the writer picks up some component from original material.Thus,the writer builds up the model of rice evapotranspiration to solve the multiple correlation among the independent variables (some weather factors).At last,the writer analyses the model in some parts,and gains the satisfied result.

Estimating canopy closure density and above-ground tree biomass using partial least square methods in Chinese boreal forests

Boreal forests play an important role in global environment systems. Understanding boreal forest ecosystem structure and function requires accurate monitoring and estimating of forest canopy and biomass. We used partial least square regression （PLSR） models to relate forest parameters, i.e. canopy closure density and above ground tree biomass, to Landsat ETM＋ data. The established models were optimized according to the variable importance for projection （VIP） criterion and the bootstrap method, and their performance was compared using several statistical indices. All variables selected by the VIP criterion passed the bootstrap test （p〈0.05）. The simplified models without insignificant variables （VIP 〈1） performed as well as the full model but with less computation time. The relative root mean square error （RMSE%） was 29% for canopy closure density, and 58% for above ground tree biomass. We conclude that PLSR can be an effective method for estimating canopy closure density and above ground biomass.

Incorporating empirical knowledge into data-driven variable selection for quantitative analysis of coal ash content by laser-induced breakdown spectroscopy

Laser-induced breakdown spectroscopy(LIBS)has become a widely used atomic spectroscopic technique for rapid coal analysis.However,the vast amount of spectral information in LIBS contains signal uncertainty,which can affect its quantification performance.In this work,we propose a hybrid variable selection method to improve the performance of LIBS quantification.Important variables are first identified using Pearson's correlation coefficient,mutual information,least absolute shrinkage and selection operator(LASSO)and random forest,and then filtered and combined with empirical variables related to fingerprint elements of coal ash content.Subsequently,these variables are fed into a partial least squares regression(PLSR).Additionally,in some models,certain variables unrelated to ash content are removed manually to study the impact of variable deselection on model performance.The proposed hybrid strategy was tested on three LIBS datasets for quantitative analysis of coal ash content and compared with the corresponding data-driven baseline method.It is significantly better than the variable selection only method based on empirical knowledge and in most cases outperforms the baseline method.The results showed that on all three datasets the hybrid strategy for variable selection combining empirical knowledge and data-driven algorithms achieved the lowest root mean square error of prediction(RMSEP)values of 1.605,3.478 and 1.647,respectively,which were significantly lower than those obtained from multiple linear regression using only 12 empirical variables,which are 1.959,3.718 and 2.181,respectively.The LASSO-PLSR model with empirical support and 20 selected variables exhibited a significantly improved performance after variable deselection,with RMSEP values dropping from 1.635,3.962 and 1.647 to 1.483,3.086 and 1.567,respectively.Such results demonstrate that using empirical knowledge as a support for datadriven variable selection can be a viable approach to improve the accuracy and reliability of LIBS quantification.

NIR Hyperspectral Imaging Measurement of Sugar Content in Peach Using PLS Regression

Near infrared （NIR） hyperspectral imaging measurement of sugar content in peach was introauced. NIR spectral images （650～1 000 nm, resolution： 2 nm） of peach samples were captured with developed hyperspectral imaging setup. Partial least square （PLS） regression prediction model was developed to estimate the sugar content in peach; step-wise backward method was utilized to determine optimal wavelength subsets. Experimental results show that the calibration model with optimal wavelength subsets has a correlation coefficient of prediction of 0.97 and a standard error of prediction of 0.19, the prediction accuracy is higher than the calibration model applied over the whole wavelength, which proves that variable selection plays an important role in improving the prediction accuracy of PLS regression model.

Modeling Solid Waste Minimization Performance at Source in Dar es Salaam City, Tanzania

Municipal solid waste generation is strongly linked to rising human population and expanding urban areas, with significant implications on urban metabolism as well as space and place values redefinition. Effective management performance of municipal solid waste management underscores the interdisciplinarity strategies. Such knowledge and skills are paramount to uncover the sources of waste generation as well as means of waste storage, collection, recycling, transportation, handling/treatment, disposal, and monitoring. This study was conducted in Dar es Salaam city. Driven by the curiosity model of the solid waste minimization performance at source, study data was collected using focus group discussion techniques to ward-level local government officers, which was triangulated with literature and documentary review. The main themes of the FGD were situational factors (SFA) and local government by-laws (LGBY). In the FGD session, sub-themes of SFA tricked to understand how MSW minimization is related to the presence and effect of services such as land use planning, availability of landfills, solid waste transfer stations, material recovery facilities, incinerators, solid waste collection bins, solid waste trucks, solid waste management budget and solid waste collection agents. Similarly, FGD on LGBY was extended by sub-themes such as contents of the by-law, community awareness of the by-law, and by-law enforcement mechanisms. While data preparation applied an analytical hierarchy process, data analysis applied an ordinary least square (OLS) regression model for sub-criteria that explain SFA and LGBY;and OLS standard residues as variables into geographically weighted regression with a resolution of 241 × 241 meter in ArcMap v10.5. Results showed that situational factors and local government by-laws have a strong relationship with the rate of minimizing solid waste dumping in water bodies (local R square = 0.94).

Process Characterization of the Transesterification of Rapeseed Oil to Biodiesel Using Design of Experiments and Infrared Spectroscopy

For optimization of production processes and product quality,often knowledge of the factors influencing the process outcome is compulsory.Thus,process analytical technology(PAT)that allows deeper insight into the process and results in a mathematical description of the process behavior as a simple function based on the most important process factors can help to achieve higher production efficiency and quality.The present study aims at characterizing a well-known industrial process,the transesterification reaction of rapeseed oil with methanol to produce fatty acid methyl esters(FAME)for usage as biodiesel in a continuous micro reactor set-up.To this end,a design of experiment approach is applied,where the effects of two process factors,the molar ratio and the total flow rate of the reactants,are investigated.The optimized process target response is the FAME mass fraction in the purified nonpolar phase of the product as a measure of reaction yield.The quantification is performed using attenuated total reflection infrared spectroscopy in combination with partial least squares regression.The data retrieved during the conduction of the DoE experimental plan were used for statistical analysis.A non-linear model indicating a synergistic interaction between the studied factors describes the reactor behavior with a high coefficient of determination(R^(2))of 0.9608.Thus,we applied a PAT approach to generate further insight into this established industrial process.

A Comparison of CNN and PLSR for Glucose Monitoring Using Mid-Infrared Absorption Spectroscopy

With the development of mid-infrared (MIR) photoelectric devices, mid-infrared spectroscopy has become one of the important methods for non-invasive detection of blood glucose. The mid-infrared region (4000 - 400 cm-1) has the well-known fingerprint region (1200 - 800 cm-1) of glucose, which has clearer characteristic absorption peaks and better specificity. There is a lot of molecular information about glucose in the MIR. The non-invasive detection of blood glucose by mid-infrared spectroscopy needs to achieve certain accuracy, and the quantitative model is an important factor affecting the accuracy of glucose detection. In this paper, the samples of imitation solution containing only glucose and the samples of imitation mixed solution are taken as the research objects, and the mid-infrared spectral data of the samples are collected. The full spectrum partial least squares Regression (PLSR) model, SNV + Ctr-PLSR model, MSC + Ctr-PLSR model, and convolutional neural networks (CNN) model of 3000 - 900 cm-1 band were constructed. Full spectrum PLS model and CNN model of 1200 - 900 cm-1 band were constructed. The experimental results show that the optimal model of the two bands is CNN, then the correlation coefficient of prediction set (Rp) of 3000 - 900 cm-1 band is 0.95, and the root mean square error of pre-diction set (RMSEP) value is 22.10. The Rp of 1200 - 900 cm-1 band is 0.95, and the RMSEP value is 22.54. The research results show that CNN is a promising method, which has higher accuracy than PLSR, and is especially suitable for modeling human complex environment. In addition, the study provides a theoretical and practical basis for CNN in feature selection and model interpretation.

Multi-View & Transfer Learning for Epilepsy Recognition Based on EEG Signals

Epilepsy is a central nervous system disorder in which brain activity becomes abnormal.Electroencephalogram(EEG)signals,as recordings of brain activity,have been widely used for epilepsy recognition.To study epilep-tic EEG signals and develop artificial intelligence(AI)-assist recognition,a multi-view transfer learning(MVTL-LSR)algorithm based on least squares regression is proposed in this study.Compared with most existing multi-view transfer learning algorithms,MVTL-LSR has two merits:(1)Since traditional transfer learning algorithms leverage knowledge from different sources,which poses a significant risk to data privacy.Therefore,we develop a knowledge transfer mechanism that can protect the security of source domain data while guaranteeing performance.(2)When utilizing multi-view data,we embed view weighting and manifold regularization into the transfer framework to measure the views’strengths and weaknesses and improve generalization ability.In the experimental studies,12 different simulated multi-view&transfer scenarios are constructed from epileptic EEG signals licensed and provided by the Uni-versity of Bonn,Germany.Extensive experimental results show that MVTL-LSR outperforms baselines.The source code will be available on https://github.com/didid5/MVTL-LSR.