Multivariate adaptive regression splines and neural network models for prediction of pile drivability

Piles are long, slender structural elements used to transfer the loads from the superstructure through weak strata onto stiffer soils or rocks. For driven piles, the impact of the piling hammer induces compression and tension stresses in the piles. Hence, an important design consideration is to check that the strength of the pile is sufficient to resist the stresses caused by the impact of the pile hammer. Due to its complexity, pile drivability lacks a precise analytical solution with regard to the phenomena involved.In situations where measured data or numerical hypothetical results are available, neural networks stand out in mapping the nonlinear interactions and relationships between the system’s predictors and dependent responses. In addition, unlike most computational tools, no mathematical relationship assumption between the dependent and independent variables has to be made. Nevertheless, neural networks have been criticized for their long trial-and-error training process since the optimal configuration is not known a priori. This paper investigates the use of a fairly simple nonparametric regression algorithm known as multivariate adaptive regression splines（MARS）, as an alternative to neural networks, to approximate the relationship between the inputs and dependent response, and to mathematically interpret the relationship between the various parameters. In this paper, the Back propagation neural network（BPNN） and MARS models are developed for assessing pile drivability in relation to the prediction of the Maximum compressive stresses（MCS）, Maximum tensile stresses（MTS）, and Blow per foot（BPF）. A database of more than four thousand piles is utilized for model development and comparative performance between BPNN and MARS predictions.

Advanced reliability analysis of slopes in spatially variable soils using multivariate adaptive regression splines

This study aims to extend the multivariate adaptive regression splines(MARS)-Monte Carlo simulation(MCS) method for reliability analysis of slopes in spatially variable soils. This approach is used to explore the influences of the multiscale spatial variability of soil properties on the probability of failure(P_f) of the slopes. In the proposed approach, the relationship between the factor of safety and the soil strength parameters characterized with spatial variability is approximated by the MARS, with the aid of Karhunen-Loeve expansion. MCS is subsequently performed on the established MARS model to evaluate Pf.Finally, a nominally homogeneous cohesive-frictional slope and a heterogeneous cohesive slope, which are both characterized with different spatial variabilities, are utilized to illustrate the proposed approach.Results showed that the proposed approach can estimate the P_f of the slopes efficiently in spatially variable soils with sufficient accuracy. Moreover, the approach is relatively robust to the influence of different statistics of soil properties, thereby making it an effective and practical tool for addressing slope reliability problems concerning time-consuming deterministic stability models with low levels of P_f.Furthermore, disregarding the multiscale spatial variability of soil properties can overestimate or underestimate the P_f. Although the difference is small in general, the multiscale spatial variability of the soil properties must still be considered in the reliability analysis of heterogeneous slopes, especially for those highly related to cost effective and accurate designs.

Using multivariate adaptive regression splines to develop relationship between rock quality designation and permeability

The assessment of in situ permeability of rock mass is challenging for large-scale projects such as reservoirs created by dams,where water tightness issues are of prime importance.The in situ permeability is strongly related to the frequency and distribution of discontinuities in the rock mass and quantified by rock quality designation(RQD).This paper analyzes the data of hydraulic conductivity and discontinuities sampled at different depths during the borehole investigations in the limestone and sandstone formations for the construction of hydraulic structures in Oman.Cores recovered from boreholes provide RQD data,and in situ Lugeon tests elucidate the permeability.A modern technique of multivariate adaptive regression splines(MARS)assisted in correlating permeability and RQD along with the depth.In situ permeability shows a declining trend with increasing RQD,and the depth of investigation is within 50 m.This type of relationship can be developed based on detailed initial investigations at the site where the hydraulic conductivity of discontinuous rocks is required to be delineated.The relationship can approximate the permeability by only measuring the RQD in later investigations on the same site,thus saving the time and cost of the site investigations.The applicability of the relationship developed in this study to another location requires a lithological similarity of the rock mass that can be verified through preliminary investigation at the site.

Mountain permafrost distribution modeling using Multivariate Adaptive Regression Spline (MARS) in the Wenquan area over the Qinghai-Tibet Plateau

In high mountainous areas, the development and distribution of alpine permafrost is greatly affected by macro- and mi- cro-topographic factors. The effects of latitude, altitude, slope, and aspect on the distribution of permafrost were studied to under- stand the dislribution patterns of permafrost in Wenquan on the Qinghai-Tibet Plateau. Cluster and correlation analysis were per- formed based on 30 m Global Digital Elevation Model （GDEM） data and field data obtained using geophysical exploration and borehole drilling methods. A Multivariate Adaptive Regression Spline model （MARS） was developed to simulate permafrost spa- tial distribution over the studied area. A validation was followed by comparing to 201 geophysical exploration sites, as well as by comparing to two other models, i.e., a binary logistic regression model and the Mean Annual Ground Temperature model （IVlAGT）. The MARS model provides a better simulation than the other two models. Besides the control effect of elevation on permafrost distribution, the MARS model also takes into account the impact of direct solar radiation on permafrost distribution.

Multivariate adaptive regression splines based simulation optimization using move-limit strategy

This paper makes an approach to the approximate optimum in structural design,which combines the global response surface（GRS） based multivariate adaptive regression splines（MARS） with Move-Limit strategy（MLS）.MARS is an adaptive regression process,which fits in with the multidimensional problems.It adopts a modified recursive partitioning strategy to simplify high-dimensional problems into smaller highly accurate models.MLS for moving and resizing the search sub-regions is employed in the space of design variables.The quality of the approximation functions and the convergence history of the optimization process are reflected in MLS.The disadvantages of the conventional response surface method（RSM） have been avoided,specifically,highly nonlinear high-dimensional problems.The GRS/MARS with MLS is applied to a high-dimensional test function and an engineering problem to demonstrate its feasibility and convergence,and compared with quadratic response surface（QRS） models in terms of computational efficiency and accuracy.

Prediction Model of Compressive Strength of Fly Ash-Slag Concrete Based on Multiple Adaptive Regression Splines

Accurate prediction of compressive strength of concrete is one of the key issues in the concrete industry. In this paper, a prediction method of fly ash-slag concrete compressive strength based on multiple adaptive regression splines (MARS) is proposed, and the model analysis process is determined by analyzing the principle of this algorithm. Based on the Concrete Compressive Strength dataset of UCI, the MARS model for compressive strength prediction was constructed with cement content, blast furnace slag powder content, fly ash content, water content, reducing agent content, coarse aggregate content, fine aggregate content and age as independent variables. The prediction results of artificial neural network (BP), random forest (RF), support vector machine (SVM), extreme learning machine (ELM), and multiple nonlinear regression (MnLR) were compared and analyzed, and the prediction accuracy and model stability of MARS and RF models had obvious advantages, and the comprehensive performance of MARS model was slightly better than that of RF model. Finally, the explicit expression of the MARS model for compressive strength is given, which provides an effective method to achieve the prediction of compressive strength of fly ash-slag concrete.

Spline-based multi-regime traffic stream models

In order to develop optimal multi-regime traffic stream models, a new method that integrates cluster analysis and B-spline regression is presented. First, for identifying the proper number of regimes, the K-means and the fuzzy c-means methods are applied in cluster analysis to actual traffic data, which suggests that dividing the traffic flow into two or three clusters can best reflect intrinsic patterns of traffic flows. Such information is then taken as guidance in spline regression, thus significantly reducing the computational burden of estimating spline models. Spline regression is used to estimate the locations of knots and the coefficients of the model so that the global error can be minimized. Model analysis results demonstrate that the proposed spline models have better fitting and generalization capability than the conventional models. In addition, the new method is more flexible in terms of data fitting and can provide smoother traffic stream models.

Data-driven intelligent modeling framework for the steam cracking process

Steam cracking is the dominant technology for producing light olefins,which are believed to be the foundation of the chemical industry.Predictive models of the cracking process can boost production efficiency and profit margin.Rapid advancements in machine learning research have recently enabled data-driven solutions to usher in a new era of process modeling.Meanwhile,its practical application to steam cracking is still hindered by the trade-off between prediction accuracy and computational speed.This research presents a framework for data-driven intelligent modeling of the steam cracking process.Industrial data preparation and feature engineering techniques provide computational-ready datasets for the framework,and feedstock similarities are exploited using k-means clustering.We propose LArge-Residuals-Deletion Multivariate Adaptive Regression Spline(LARD-MARS),a modeling approach that explicitly generates output formulas and eliminates potentially outlying instances.The framework is validated further by the presentation of clustering results,the explanation of variable importance,and the testing and comparison of model performance.

Physics-based and data-driven modeling for stability evaluation of buried structures in natural clays

This study presents a hybrid framework to predict stability solutions of buried structures under active trapdoor conditions in natural clays with anisotropy and heterogeneity by combining physics-based and data-driven modeling.Finite-element limit analysis(FELA)with a newly developed anisotropic undrained shear(AUS)failure criterion is used to identify the underlying active failure mechanisms as well as to develop a numerical(physics-based)database of stability numbers for both planar and circular trapdoors.Practical considerations are given for natural clays to three linearly increasing shear strengths in compression,extension,and direct simple shear in the AUS material model.The obtained numerical solutions are compared and validated with published solutions in the literature.A multivariate adaptive regression splines(MARS)algorithm is further utilized to learn the numerical solutions to act as fast FELA data-driven surrogates for stability evaluation.The current MARS-based modeling provides both relative importance index and accurate design equations that can be used with confidence by practitioners.

Simulation of Daily Diffuse Solar Radiation Based on Three Machine Learning Models

Solar radiation is an important parameter in the fields of computer modeling,engineering technology and energy development.This paper evaluated the ability of three machine learning models,i.e.,Extreme Gradient Boosting(XGBoost),Support Vector Machine(SVM)and Multivariate Adaptive Regression Splines(MARS),to estimate the daily diffuse solar radiation(Rd).The regular meteorological data of 1966-2015 at five stations in China were taken as the input parameters(including mean average temperature(Ta),theoretical sunshine duration(N),actual sunshine duration(n),daily average air relative humidity(RH),and extra-terrestrial solar radiation(Ra)).And their estimation accuracies were subjected to comparative analysis.The three models were first trained using meteorological data from 1966 to 2000.Then,the 2001-2015 data was used to test the trained machine learning model.The results show that the XGBoost had better accuracy than the other two models in coefficient of determination(R2),root mean square error(RMSE),mean bias error(MBE)and normalized root mean square error(NRMSE).The MARS performed better in the training phase than the testing phase,but became less accurate in the testing phase,with the R2 value falling by 2.7-16.9%on average.By contrast,the R2 values of SVM and XGBoost increased by 2.9-12.2%and 1.9-14.3%,respectively.Despite trailing slightly behind the SVM at the Beijing station,the XGBoost showed good performance at the rest of the stations in the two phases.In the training phase,the accuracy growth is small but observable.In addition,the XGBoost had a slightly lower RMSE than the SVM,a signal of its edge in stability.Therefore,the three machine learning models can estimate the daily Rd based on local inputs and the XGBoost stands out for its excellent performance and stability.

Prediction of Compressive Strength of Self-Compacting Concrete Using Intelligent Computational Modeling

In the present scenario,computational modeling has gained much importance for the prediction of the properties of concrete.This paper depicts that how computational intelligence can be applied for the prediction of compressive strength of Self Compacting Concrete(SCC).Three models,namely,Extreme Learning Machine(ELM),Adaptive Neuro Fuzzy Inference System(ANFIS)and Multi Adaptive Regression Spline(MARS)have been employed in the present study for the prediction of compressive strength of self compacting concrete.The contents of cement(c),sand(s),coarse aggregate(a),fly ash(f),water/powder(w/p)ratio and superplasticizer(sp)dosage have been taken as inputs and 28 days compressive strength(fck)as output for ELM,ANFIS and MARS models.A relatively large set of data including 80 normalized data available in the literature has been taken for the study.A comparison is made between the results obtained from all the above-mentioned models and the model which provides best fit is established.The experimental results demonstrate that proposed models are robust for determination of compressive strength of self-compacting concrete.

Prediction of interfaces of geological formations using the multivariate adaptive regression spline method

The design and construction of underground structures are significantly affected by the distribution of geological formations.Prediction of the geological interfaces using limited data has been a difficult task.A multivariate adaptive regression spline(MARS)method capable of modeling nonlinearities automatically was used in this study to spatially predict the elevations of geological interfaces.Borehole data from two sites in Singapore were used to evaluate the capability of the MARS method for predicting geological interfaces.By comparing the predicted values with the borehole data,it is shown that the MARS method has a mean of root mean square error of 4.4 m for the predicted elevations of the Kallang Formation–Old Alluvium interface.In addition,the MARS method is able to produce reasonable prediction intervals in the sense that the percentage of testing data covered by 95% prediction intervals was close to the associated confidence level,95%.More importantly,the prediction interval evaluated by the MARS method had a non-constant width that appropriately reflected the data density and geological complexity.

A nonparametric regression method for multiple longitudinal phenotypes using multivariate adaptive splines

In genetic studies of complex diseases, particularly mental illnesses, and behavior disorders, two distinct characteristics have emerged in some data sets. First, genetic data sets are collected with a large number of phenotypes that are potentially related to the complex disease under study. Second, each phenotype is collected from the same subject repeatedly over time. In this study, we present a nonparametric regression approach to study multivariate and time-repeated phenotypes together by using the technique of the multivariate adaptive regression splines for analysis of longitudinal data （MASAL）, which makes it possible to identify genes, gene-gene and gene-environment, including time, interactions associated with the phenotypes of interest. Furthermore, we propose a permutation test to assess the associations between the phenotypes and selected markers. Through simulation, we demonstrate that our proposed approach has advantages over the existing methods that examine each longitudinal phenotype separately or analyze the summarized values of phenotypes by compressing them into one-time-point phenotypes. Application of the proposed method to the Framingham Heart Study illustrates that the use of multivariate longitudinal phenotypes enhanced the significance of the association test.

Downscaling Landsat-8 land surface temperature maps in diverse urban landscapes using multivariate adaptive regression splines and very high resolution auxiliary data

We propose a method for spatial downscaling of Landsat 8-derived LST maps from 100(30 m)resolution down to 2–4 m with the use of the Multiple Adaptive Regression Splines(MARS)models coupled with very high resolution auxiliary data derived from hyperspectral aerial imagery and large-scale topographic maps.We applied the method to four Landsat 8 scenes,two collected in summer and two in winter,for three British towns collectively representing a variety of urban form.We used several spectral indices as well as fractional coverage of water and paved surfaces as LST predictors,and applied a novel method for the correction of temporal mismatch between spectral indices derived from aerial and satellite imagery captured at different dates,allowing for the application of the downscaling method for multiple dates without the need for repeating the aerial survey.Our results suggest that the method performed well for the summer dates,achieving RMSE of 1.40–1.83 K prior to and 0.76–1.21 K after correction for residuals.We conclude that the MARS models,by addressing the non-linear relationship of LST at coarse and fine spatial resolutions,can be successfully applied to produce high resolution LST maps suitable for studies of urban thermal environment at local scales.

Interactive image segmentation with a regression based ensemble learning paradigm

To achieve fine segmentation of complex natural images, people often resort to an interactive segmentation paradigm, since fully automatic methods often fail to obtain a result consistent with the ground truth. However, when the foreground and background share some similar areas in color, the fine segmentation result of conventional interactive methods usually relies on the increase o f manual labels. This paper presents a novel interactive image segmentation method via a regression-based ensemble model with semi-supervised learning. The task is formulated as a non-linear problem integrating two complementary spline regressors and strengthening the robustness of each regressor via semi-supervised learning. First, two spline regressors with a complementary nature are constructed based on multivariate adaptive regression splines （MARS） and smooth thin plate spline regression （TPSR）. Then, a regressor boosting method based on a clustering hypothesis and semi-supervised learning is proposed to assist the training of MARS and TPSR by using the region segmentation information contained in unlabeled pixels. Next, a support vector regression （SVR） based decision fusion model is adopted to integrate the results of MARS and TPSR. Finally, the GraphCut is introduced and combined with the SVR ensemble results to achieve image segmentation. Extensive experimental results on benchmark datasets of BSDS500 and Pascal VOC have demonstrated the effectiveness of our method, and the com- parison with experiment results has validated that the proposed method is comparable with the state-of-the-art methods for in- teractive natural image segmentation.

EFFICIENT ESTIMATION OF SEEMINGLY UNRELATED ADDITIVE NONPARAMETRIC REGRESSION MODELS

This paper is concerned with the estimating problem of seemingly unrelated(SU)nonparametric additive regression models.A polynomial spline based two-stage efficient approach is proposed to estimate the nonparametric components,which takes both of the additive structure and correlation between equations into account.The asymptotic normality of the derived estimators are established.The authors also show they own some advantages,including they are asymptotically more efficient than those based on only the individual regression equation and have an oracle property,which is the asymptotic distribution of each additive component is the same as it would be if the other components were known with certainty.Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedure.Applying the proposed procedure to a real data set is also made.

Soft computing approach for prediction of surface settlement induced by earth pressure balance shield tunneling

Estimating surface settlement induced by excavation construction is an indispensable task in tunneling,particularly for earth pressure balance(EPB)shield machines.In this study,predictive models for assessing surface settlement caused by EPB tunneling were established based on extreme gradient boosting(XGBoost),artificial neural network,support vector machine,and multivariate adaptive regression spline.Datasets from three tunnel construction projects in Singapore were used,with main input parameters of cover depth,advance rate,earth pressure,mean standard penetration test(SPT)value above crown level,mean tunnel SPT value,mean moisture content,mean soil elastic modulus,and grout pressure.The performances of these soft computing models were evaluated by comparing predicted deformation with measured values.Results demonstrate the acceptable accuracy of the model in predicting ground settlement,while XGBoost demonstrates a slightly higher accuracy.In addition,the ensemble method of XGBoost is more computationally efficient and can be used as a reliable alternative in solving multivariate nonlinear geo-engineering problems.

Prediction of lining response for twin tunnels constructed in anisotropic clay using machine learning techniques

Excessive structural forces generated inside tunnel linings could affect the safety and serviceability of tunnels,emphasizing the need to accurately predict the forces acting on tunnel linings during the preliminary design phase.In this study,an anisotropic soil model devel-oped by Norwegian Geotechnical Institute(NGI)based on the Active-Direct shear-Passive concept(NGI-ADP model)was adopted to conduct finite element(FE)analyses.A total of 682 cases were modeled to analyze the effects of five key parameters on twin-tunnel struc-tural forces;these parameters included twin-tunnel arrangements and subsurface soil properties:burial depth H,tunnel center-to-center distance D,soil strength s_(u)^(A),stiffness ratio G_(u)=s_(u)^(A),and degree of anisotropy ss_(u)^(P)=s_(u)^(A).The significant factors contributing to the bending moment and thrust force of the linings were the tunnel distance and overlying soil depth,respectively.The degree of anisotropy of the surrounding soil was found to be extremely important in simulating the twin-tunnel construction,and severe design errors could be made if the soil anisotropy is ignored.A cutting-edge application of machine learning in the construction of twin tunnels is presented;multivariate adaptive regression splines and decision tree regressor methods are developed to predict the maximum bending moment within the first tunnel’s linings based on the constructed FE cases.The developed prediction model can enable engineers to estimate the structural response of twin tunnels more accurately in order to meet the specific target reliability indices of projects.

Undrained stability of pit-in-pit braced excavations under hydraulic uplift

Pit-in-pit(PIP)excavations in an aquifer–aquitard system likely undergo catastrophic failures under the hydraulic uplift,the associated undrained stability problem,however,has not been well analyzed in the past.To this end,a hypothetical model of PIP braced excavation in typical soil layers of Shanghai,China is developed using the finite element limit analysis(FELA)tool.The FELA solutions of safety factors(FSs)against hydraulic uplift are verified with the results from the finite element analysis with strength reduction technique(SRFEA)and existing design approaches.Subsequently,FELA is employed to identify the triggering and failure mechanisms of PIP braced excavations subjected to hydraulic uplift.A series of parametric studies considering the various geometric configurations of the PIP excavation,undrained shear strengths of aquitard,and artesian pressures are carried out.The sensitivities of relevant design parameters are further assessed using a multivariate adaptive regression splines(MARS)model that is capable of accurately capturing the nonlinear relationships between a set of input variables and output variables in multi-dimensions.A MARS-based design equation used for predicting FS is finally presented using the artificial dataset from FELA for practical design uses.

A COMPARATIVE STUDY OF DATA MINING METHODS IN CONSUMER LOANS CREDIT SCORING MANAGEMENT

Credit scoring has become a critical and challenging management science issue as the credit industry has been facing stiffer competition in recent years. Many classification methods have been suggested to tackle this problem in the literature. In this paper, we investigate the performance of various credit scoring models and the corresponding credit risk cost for three real-life credit scoring data sets. Besides the well-known classification algorithms （e.g. linear discriminant analysis, logistic regression, neural networks and k-nearest neighbor）, we also investigate the suitability and performance of some recently proposed, advanced data mining techniques such as support vector machines （SVMs）, classification and regression tree （CART）, and multivariate adaptive regression splines （MARS）. The performance is assessed by using the classification accuracy and cost of credit scoring errors. The experiment results show that SVM, MARS, logistic regression and neural networks yield a very good performance. However, CART and MARS＇s explanatory capability outperforms the other methods.