Revisiting Akaike’s Final Prediction Error and the Generalized Cross Validation Criteria in Regression from the Same Perspective: From Least Squares to Ridge Regression and Smoothing Splines

In regression, despite being both aimed at estimating the Mean Squared Prediction Error (MSPE), Akaike’s Final Prediction Error (FPE) and the Generalized Cross Validation (GCV) selection criteria are usually derived from two quite different perspectives. Here, settling on the most commonly accepted definition of the MSPE as the expectation of the squared prediction error loss, we provide theoretical expressions for it, valid for any linear model (LM) fitter, be it under random or non random designs. Specializing these MSPE expressions for each of them, we are able to derive closed formulas of the MSPE for some of the most popular LM fitters: Ordinary Least Squares (OLS), with or without a full column rank design matrix;Ordinary and Generalized Ridge regression, the latter embedding smoothing splines fitting. For each of these LM fitters, we then deduce a computable estimate of the MSPE which turns out to coincide with Akaike’s FPE. Using a slight variation, we similarly get a class of MSPE estimates coinciding with the classical GCV formula for those same LM fitters.

Fast cross validation for regularized extreme learning machine

A method for fast 1-fold cross validation is proposed for the regularized extreme learning machine （RELM）. The computational time of fast l-fold cross validation increases as the fold number decreases, which is opposite to that of naive 1-fold cross validation. As opposed to naive l-fold cross validation, fast l-fold cross validation takes the advantage in terms of computational time, especially for the large fold number such as l 〉 20. To corroborate the efficacy and feasibility of fast l-fold cross validation, experiments on five benchmark regression data sets are evaluated.

Efficient strategies for leave-one-out cross validation for genomic best linear unbiased prediction

Background： A random multiple-regression model that simultaneously fit all allele substitution effects for additive markers or haplotypes as uncorrelated random effects was proposed for Best Linear Unbiased Prediction, using whole-genome data. Leave-one-out cross validation can be used to quantify the predictive ability of a statistical model.Methods： Naive application of Leave-one-out cross validation is computationally intensive because the training and validation analyses need to be repeated n times, once for each observation. Efficient Leave-one-out cross validation strategies are presented here, requiring little more effort than a single analysis.Results： Efficient Leave-one-out cross validation strategies is 786 times faster than the naive application for a simulated dataset with 1,000 observations and 10,000 markers and 99 times faster with 1,000 observations and 100 markers. These efficiencies relative to the naive approach using the same model will increase with increases in the number of observations.Conclusions： Efficient Leave-one-out cross validation strategies are presented here, requiring little more effort than a single analysis.

Subsampling bias and the best-discrepancy systematic cross validation

Statistical machine learning models should be evaluated and validated before putting to work.Conventional k-fold Monte Carlo cross-validation(MCCV)procedure uses a pseudo-random sequence to partition instances into k subsets,which usually causes subsampling bias,inflates generalization errors and jeopardizes the reliability and effectiveness of cross-validation.Based on ordered systematic sampling theory in statistics and low-discrepancy sequence theory in number theory,we propose a new k-fold cross-validation procedure by replacing a pseudo-random sequence with a best-discrepancy sequence,which ensures low subsampling bias and leads to more precise expected-prediction-error(EPE)estimates.Experiments with 156 benchmark datasets and three classifiers(logistic regression,decision tree and na?ve bayes)show that in general,our cross-validation procedure can extrude subsampling bias in the MCCV by lowering the EPE around 7.18%and the variances around 26.73%.In comparison,the stratified MCCV can reduce the EPE and variances of the MCCV around 1.58%and 11.85%,respectively.The leave-one-out(LOO)can lower the EPE around 2.50%but its variances are much higher than the any other cross-validation(CV)procedure.The computational time of our cross-validation procedure is just 8.64%of the MCCV,8.67%of the stratified MCCV and 16.72%of the LOO.Experiments also show that our approach is more beneficial for datasets characterized by relatively small size and large aspect ratio.This makes our approach particularly pertinent when solving bioscience classification problems.Our proposed systematic subsampling technique could be generalized to other machine learning algorithms that involve random subsampling mechanism.

Least Squares Model Averaging Based on Generalized Cross Validation

Frequentist model averaging has received much attention from econometricians and statisticians in recent years.A key problem with frequentist model average estimators is the choice of weights.This paper develops a new approach of choosing weights based on an approximation of generalized cross validation.The resultant least squares model average estimators are proved to be asymptotically optimal in the sense of achieving the lowest possible squared errors.Especially,the optimality is built under both discrete and continuous weigh sets.Compared with the existing approach based on Mallows criterion,the conditions required for the asymptotic optimality of the proposed method are more reasonable.Simulation studies and real data application show good performance of the proposed estimators.

非参数回归的L_1-Cross-Validation最近邻中位数估计的强相合性

考虑非参数回归模型Yi=g（xi）+ei,i≥1,其中g（x）是待估计的连续函数,{xi,i≥1}是非随机的,{ei,i≥1}是iid随机误差,在本文中,我们讨论最近邻中位数估计（x）=m(Y（i（1））,…,Yi（h*）=Yi（1）,…,Yi（h*）之中位数,其中h*利用L1—Cross—Validation方法选择,在一定条件下,建立了L1—Cross—Validation最近邻中位数估计的强相合性。

Intelligent prediction of slope stability based on visual exploratory data analysis of 77 in situ cases

Slope stability prediction research is a complex non-linear system problem.In carrying out slope stability prediction work,it often encounters low accuracy of prediction models and blind data preprocessing.Based on 77 field cases,5 quantitative indicators are selected to improve the accuracy of prediction models for slope stability.These indicators include slope angle,slope height,internal friction angle,cohesion and unit weight of rock and soil.Potential data aggregation in the prediction of slope stability is analyzed and visualized based on Six-dimension reduction methods,namely principal components analysis(PCA),Kernel PCA,factor analysis(FA),independent component analysis(ICA),non-negative matrix factorization(NMF)and t-SNE(stochastic neighbor embedding).Combined with classic machine learning methods,7 prediction models for slope stability are established and their reliabilities are examined by random cross validation.Besides,the significance of each indicator in the prediction of slope stability is discussed using the coefficient of variation method.The research results show that dimension reduction is unnecessary for the data processing of prediction models established in this paper of slope stability.Random forest(RF),support vector machine(SVM)and k-nearest neighbour(KNN)achieve the best prediction accuracy,which is higher than 90%.The decision tree(DT)has better accuracy which is 86%.The most important factor influencing slope stability is slope height,while unit weight of rock and soil is the least significant.RF and SVM models have the best accuracy and superiority in slope stability prediction.The results provide a new approach toward slope stability prediction in geotechnical engineering.

Seasonal Prediction of Tropical Cyclones and Storms over the Southwestern Indian Ocean Region Using the Generalized Linear Models

Tropical cyclones (TCs) and storms (TSs) are among the devastating events in the world and southwestern Indian Ocean (SWIO) in particular. The seasonal forecasting TCs and TSs for December to March (DJFM) and November to May (NM) over SWIO were conducted. Dynamic parameters including vertical wind shear, mean zonal steering wind and vorticity at 850 mb were derived from NOAA (NCEP-NCAR) reanalysis 1 wind fields. Thermodynamic parameters including monthly and daily mean Sea Surface Temperature (SST), Outgoing Longwave Radiation (OLR) and equatorial Standard Oscillation Index (SOI) were used. Three types of Poison regression models (i.e. dynamic, thermodynamic and combined models) were developed and validated using the Leave One Out Cross Validation (LOOCV). Moreover, 2 × 2 square matrix contingency tables for model verification were used. The results revealed that, the observed and cross validated DJFM and NM TCs and TSs strongly correlated with each other (p ≤ 0.02) for all model types, with correlations (r) ranging from 0.62 - 0.86 for TCs and 0.52 - 0.87 for TSs, indicating great association between these variables. Assessment of the model skill for all model types of DJFM and NM TCs and TSs frequency revealed high skill scores ranging from 38% - 70% for TCs and 26% - 72% for TSs frequency, respectively. Moreover, results indicated that the dynamic and combined models had higher skill scores than the thermodynamic models. The DJFM and NM selected predictors explained the TCs and TSs variability by the range of 0.45 - 0.65 and 0.37 - 0.66, respectively. However, verification analysis revealed that all models were adequate for predicting the seasonal TCs and TSs, with high bias values ranging from 0.85 - 0.94. Conclusively, the study calls for more studies in TCs and TSs frequency and strengths for enhancing the performance of the March to May (MAM) and December to October (OND) seasonal rainfalls in the East African (EA) and Tanzania in particular.

农田土壤微量元素的空间变异及Kriging估值

采用差分式GPS定位的方法,在山东省龙口市北马镇面积为10 km2的科技示范区内采用200 m×170 m网格式设置205个土壤采样点。用EDTA提取-原子吸收分光光度计测定法对示范区土壤微量元素铜、锌、铁、锰、硼的有效态含量进行了测定分析。在此基础上,选择Arcinfo地理信息系统软件,采用GeostatisticsAnalyst地统计分析模块中的Kriging插值方法对微量元素的空间变异性进行研究,该工作的关键是Kriging插值模型的选择及Cross-Validation交叉验证法的检验与系数修正。结果表明,龙口市北马镇0～20 cm耕层土壤中铜、锌、铁、锰、硼微量元素含量空间相关性中等,其中锌空间变异性相对较强,铜空间变异性相对弱一些。

A GCM-Based Forecasting Model for the Landfall of Tropical Cyclones in China

A statistical dynamic model for forecasting Chinese landfall of tropical cyclones （CLTCs） was developed based on the empirical relationship between the observed CLTC variability and the hindcast atmospheric circulations from the Pusan National University coupled general circulation model （PNU-CGCM）.In the last 31 years,CLTCs have shown strong year-to-year variability,with a maximum frequency in 1994 and a minimum frequency in 1987.Such features were well forecasted by the model.A cross-validation test showed that the correlation between the observed index and the forecasted CLTC index was high,with a coefficient of 0.71.The relative error percentage （16.3%） and root-mean-square error （1.07） were low.Therefore the coupled model performs well in terms of forecasting CLTCs;the model has potential for dynamic forecasting of landfall of tropical cyclones.

Morphology cluster and prediction of growth of human brain pyramidal neurons

Predicting neuron growth is valuable to understand the morphology of neurons, thus it is helpful in the research of neuron classification. This study sought to propose a new method of predicting the growth of human neurons using 1 907 sets of data in human brain pyramidal neurons obtained from the website of NeuroMorpho.Org. First, we analyzed neurons in a morphology field and used an expectation-maximization algorithm to specify the neurons into six clusters. Second, naive Bayes classifier was used to verify the accuracy of the expectation-maximization algorithm. Experiment results proved that the cluster groups here were efficient and feasible. Finally, a new method to rank the six expectation-maximization algorithm clustered classes was used in predicting the growth of human pyramidal neurons.

A Bregman adaptive sparse-spike deconvolution method in the frequency domain

To improve the anti-noise performance of the time-domain Bregman iterative algorithm,an adaptive frequency-domain Bregman sparse-spike deconvolution algorithm is proposed.By solving the Bregman algorithm in the frequency domain,the influence of Gaussian as well as outlier noise on the convergence of the algorithm is effectively avoided.In other words,the proposed algorithm avoids data noise effects by implementing the calculations in the frequency domain.Moreover,the computational efficiency is greatly improved compared with the conventional method.Generalized cross validation is introduced in the solving process to optimize the regularization parameter and thus the algorithm is equipped with strong self-adaptation.Different theoretical models are built and solved using the algorithms in both time and frequency domains.Finally,the proposed and the conventional methods are both used to process actual seismic data.The comparison of the results confirms the superiority of the proposed algorithm due to its noise resistance and self-adaptation capability.

SVD-LSSVM and its application in chemical pattern classification

Pattern classification is an important field in machine learning; least squares support vector machine (LSSVM) is a powerful tool for pattern classification. A new version of LSSVM, SVD-LSSVM, to save time of selecting hyper parameters for LSSVM is proposed. SVD-LSSVM is trained through singular value decomposition (SVD) of kernel matrix. Cross validation time of selecting hyper parameters can be saved because a new hyper parameter, singular value contribution rate (SVCR), replaces the penalty factor of LSSVM. Several UCI benchmarking data and the Olive classification problem were used to test SVD-LSSVM. The result showed that SVD-LSSVM has good performance in classification and saves time for cross validation.

基于K-CV参数优化的SVR煤炭含碳量预测

煤炭的含碳量是衡量煤质的重要指标,传统的检测方法操作复杂、成本高,现有的预测模型精度有待进一步提高,为解决上述问题,提出一种基于K-fold Cross Validation(K-CV)参数优化的支持向量回归(SVR)预测模型。以煤炭质量检测中心提供的80组原始数据作为实验对象,选取其中的50组作为训练集,剩余的30组作为测试集。以训练集作为K-CV方法的样本数据寻找最优参数,以最优参数为基础建立SVR预测模型,并通过测试集对模型进行验证,结果表明含碳量预测的平均相对误差达到0.38%,该模型预测精度较高,具有良好的泛化性能。

Inverse Load Identification in Stiffened Plate Structure Based on in situ Strain Measurement

For practical engineering structures,it is usually difficult to measure external load distribution in a direct manner,which makes inverse load identification important.Specifically,load identification is a typical inverse problem,for which the models(e.g.,response matrix)are often ill-posed,resulting in degraded accuracy and impaired noise immunity of load identification.This study aims at identifying external loads in a stiffened plate structure,through comparing the effectiveness of different methods for parameter selection in regulation problems,including the Generalized Cross Validation(GCV)method,the Ordinary Cross Validation method and the truncated singular value decomposition method.With demonstrated high accuracy,the GCV method is used to identify concentrated loads in three different directions(e.g.,vertical,lateral and longitudinal)exerted on a stiffened plate.The results show that the GCV method is able to effectively identify multi-source static loads,with relative errors less than 5%.Moreover,under the situation of swept frequency excitation,when the excitation frequency is near the natural frequency of the structure,the GCV method can achieve much higher accuracy compared with direct inversion.At other excitation frequencies,the average recognition error of the GCV method load identification less than 10%.

Optimization of Shanghai marine environment monitoring sites by integrating spatial correlation and stratified heterogeneity

The water quality grades of phosphate（PO4-P） and dissolved inorganic nitrogen（DIN） are integrated by spatial partitioning to fit the global and local semi-variograms of these nutrients. Leave-one-out cross validation is used to determine the statistical inference method. To minimize absolute average errors and error mean squares,stratified Kriging（SK） interpolation is applied to DIN and ordinary Kriging（OK） interpolation is applied to PO4-P.Ten percent of the sites is adjusted by considering their impact on the change in deviations in DIN and PO4-P interpolation and the resultant effect on areas with different water quality grades. Thus, seven redundant historical sites are removed. Seven historical sites are distributed in areas with water quality poorer than Grade IV at the north and south branches of the Changjiang（Yangtze River） Estuary and at the coastal region north of the Hangzhou Bay. Numerous sites are installed in these regions. The contents of various elements in the waters are not remarkably changed, and the waters are mixed well. Seven sites that have been optimized and removed are set to water with quality Grades III and IV. Optimization and adjustment of unrestricted areas show that the optimized and adjusted sites are mainly distributed in regions where the water quality grade undergoes transition.Therefore, key sites for adjustment and optimization are located at the boundaries of areas with different water quality grades and seawater.

Support Vector Machine Cost Estimation Model for Road Projects

A cost estimate is one of the most important steps in road project management. There are ranges of factors that mostly affect the final project cost. Many approaches were used to estimate project cost, which took into consideration probable project performance and risks. The aim is to improve the ability of construction managers to predict a parametric cost estimate for road projects using SVM （support vector machine）. The work is based on collecting historical road executed cases. The 12 factors were identified to be the most important factors affecting the cost-estimating model. A total of 70 case studies from historical data were divided randomly into three sets： training set includes 60 cases, cross validation set includes three cases and testing set includes seven cases. The built model was successfully able to predict project cost to the AP （accuracy performance） of 95%.

半参数TBS模型的局部线性估计

研究了半参数多元回归模型E(Y|X)=μ(XTβ),其中X是维数为p的列向量,μ,β是未知参数.由于此模型不容易满足误差方差齐性和误差分布正态性,作者对该半参数回归模型两边同时应用含参数λ的Cox-Cox变换,使得变换后的回归模型满足误差方差齐性和误差分布正态性条件,然后应用局部线性技术及极大似然方法,通过两步迭代,对未知参数β,λ及未知函数μ(.)进行了估计.

An Anonymous Payment Protocol withMobile Agents in Hostile Environments

By using Pedersen's verifiable secret sharing scheme and the theory of crossvalidation, we propose an a-nonymous payment protocol which have following features: protecting theconfidentiality of sensitive payment information from spying by malicioushosts; using a trustedthird party in a minimal way; verifying the validity of the share by the merchant; allowing agent toverify that the product which it is a-bout to receive is the one it is paying for; keeping thecustomer anonymous.

Classification and Diagnosis of Lymphoma’s Histopathological Images Using Transfer Learning

Current cancer diagnosis procedure requires expert knowledge and is time-consuming,which raises the need to build an accurate diagnosis support system for lymphoma identification and classification.Many studies have shown promising results using Machine Learning and,recently,Deep Learning to detect malignancy in cancer cells.However,the diversity and complexity of the morphological structure of lymphoma make it a challenging classification problem.In literature,many attempts were made to classify up to four simple types of lymphoma.This paper presents an approach using a reliable model capable of diagnosing seven different categories of rare and aggressive lymphoma.These Lymphoma types are Classical Hodgkin Lymphoma,Nodular Lymphoma Predominant,Burkitt Lymphoma,Follicular Lymphoma,Mantle Lymphoma,Large B-Cell Lymphoma,and T-Cell Lymphoma.Our proposed approach uses Residual Neural Networks,ResNet50,with a Transfer Learning for lymphoma’s detection and classification.The model used results are validated according to the performance evaluation metrics:Accuracy,precision,recall,F-score,and kappa score for the seven multi-classes.Our algorithms are tested,and the results are validated on 323 images of 224×224 pixels resolution.The results are promising and show that our used model can classify and predict the correct lymphoma subtype with an accuracy of 91.6%.