Stress-assisted corrosion mechanism of 3Ni steel by using gradient boosting decision tree machining learning method

Traditional 3Ni weathering steel cannot completely meet the requirements for offshore engineering development,resulting in the design of novel 3Ni steel with the addition of microalloy elements such as Mn or Nb for strength enhancement becoming a trend.The stress-assisted corrosion behavior of a novel designed high-strength 3Ni steel was investigated in the current study using the corrosion big data method.The information on the corrosion process was recorded using the galvanic corrosion current monitoring method.The gradi-ent boosting decision tree(GBDT)machine learning method was used to mine the corrosion mechanism,and the importance of the struc-ture factor was investigated.Field exposure tests were conducted to verify the calculated results using the GBDT method.Results indic-ated that the GBDT method can be effectively used to study the influence of structural factors on the corrosion process of 3Ni steel.Dif-ferent mechanisms for the addition of Mn and Cu to the stress-assisted corrosion of 3Ni steel suggested that Mn and Cu have no obvious effect on the corrosion rate of non-stressed 3Ni steel during the early stage of corrosion.When the corrosion reached a stable state,the in-crease in Mn element content increased the corrosion rate of 3Ni steel,while Cu reduced this rate.In the presence of stress,the increase in Mn element content and Cu addition can inhibit the corrosion process.The corrosion law of outdoor-exposed 3Ni steel is consistent with the law based on corrosion big data technology,verifying the reliability of the big data evaluation method and data prediction model selection.

Modeling of Total Dissolved Solids (TDS) and Sodium Absorption Ratio (SAR) in the Edwards-Trinity Plateau and Ogallala Aquifers in the Midland-Odessa Region Using Random Forest Regression and eXtreme Gradient Boosting

Efficient water quality monitoring and ensuring the safety of drinking water by government agencies in areas where the resource is constantly depleted due to anthropogenic or natural factors cannot be overemphasized. The above statement holds for West Texas, Midland, and Odessa Precisely. Two machine learning regression algorithms (Random Forest and XGBoost) were employed to develop models for the prediction of total dissolved solids (TDS) and sodium absorption ratio (SAR) for efficient water quality monitoring of two vital aquifers: Edward-Trinity (plateau), and Ogallala aquifers. These two aquifers have contributed immensely to providing water for different uses ranging from domestic, agricultural, industrial, etc. The data was obtained from the Texas Water Development Board (TWDB). The XGBoost and Random Forest models used in this study gave an accurate prediction of observed data (TDS and SAR) for both the Edward-Trinity (plateau) and Ogallala aquifers with the R2 values consistently greater than 0.83. The Random Forest model gave a better prediction of TDS and SAR concentration with an average R, MAE, RMSE and MSE of 0.977, 0.015, 0.029 and 0.00, respectively. For the XGBoost, an average R, MAE, RMSE, and MSE of 0.953, 0.016, 0.037 and 0.00, respectively, were achieved. The overall performance of the models produced was impressive. From this study, we can clearly understand that Random Forest and XGBoost are appropriate for water quality prediction and monitoring in an area of high hydrocarbon activities like Midland and Odessa and West Texas at large.

基于贝叶斯超参数优化的Gradient Boosting方法的导弹气动特性预测

在导弹设计与研发的初期阶段,需要寻求高效且低成本的导弹气动力特性的分析方法。然而,气动性能分析过程中往往存在试验成本高、周期长、局限性大等问题。因此,本文采用基于提升(Boosting)的机器学习集成算法进行导弹气动特性预测,通过输入导弹的气动外形参数、马赫数和迎角,对于导弹气动力系数实现快速预测。结果表明,Boosting能够对导弹气动力系数进行精准高效预测。为进一步提升预测精度,与传统的机器学习参数调整方法相比,采用贝叶斯优化方法对梯度提升(Gradient Boosting)算法超参数进行优化,调优后的Gradient Boosting方法预测的导弹气动力系数与实际值吻合度得到提升,并将贝叶斯优化的Gradient Boosting方法与XGBoost、LightGBM、Adaboost方法进行了对比,贝叶斯优化的Gradient Boosting方法预测精度优于其他Boosting方法,证明了优化方法的可行性与有效性。

Light gradient boosting machine with optimized hyperparameters for identification of malicious access in IoT network

In this paper,an advanced and optimized Light Gradient Boosting Machine(LGBM)technique is proposed to identify the intrusive activities in the Internet of Things(IoT)network.The followings are the major contributions:i)An optimized LGBM model has been developed for the identification of malicious IoT activities in the IoT network;ii)An efficient evolutionary optimization approach has been adopted for finding the optimal set of hyper-parameters of LGBM for the projected problem.Here,a Genetic Algorithm(GA)with k-way tournament selection and uniform crossover operation is used for efficient exploration of hyper-parameter search space;iii)Finally,the performance of the proposed model is evaluated using state-of-the-art ensemble learning and machine learning-based model to achieve overall generalized performance and efficiency.Simulation outcomes reveal that the proposed approach is superior to other considered methods and proves to be a robust approach to intrusion detection in an IoT environment.

A Data-Driven Oil Production Prediction Method Based on the Gradient Boosting Decision Tree Regression

Accurate prediction ofmonthly oil and gas production is essential for oil enterprises tomake reasonable production plans,avoid blind investment and realize sustainable development.Traditional oil well production trend prediction methods are based on years of oil field production experience and expertise,and the application conditions are very demanding.With the rapid development of artificial intelligence technology,big data analysis methods are gradually applied in various sub-fields of the oil and gas reservoir development.Based on the data-driven artificial intelligence algorithmGradient BoostingDecision Tree(GBDT),this paper predicts the initial single-layer production by considering geological data,fluid PVT data and well data.The results show that the GBDT algorithm prediction model has great accuracy,significantly improving efficiency and strong universal applicability.The GBDTmethod trained in this paper can predict production,which is helpful for well site optimization,perforation layer optimization and engineering parameter optimization and has guiding significance for oilfield development.

Object-Based Burned Area Mapping with Extreme Gradient Boosting Using Sentinel-2 Imagery

The Sentinel-2 satellites are providing an unparalleled wealth of high-resolution remotely sensed information with a short revisit cycle, which is ideal for mapping burned areas both accurately and timely. This paper proposes an automated methodology for mapping burn scars using pairs of Sentinel-2 imagery, exploiting the state-of-the-art eXtreme Gradient Boosting (XGB) machine learning framework. A large database of 64 reference wildfire perimeters in Greece from 2016 to 2019 is used to train the classifier. An empirical methodology for appropriately sampling the training patterns from this database is formulated, which guarantees the effectiveness of the approach and its computational efficiency. A difference (pre-fire minus post-fire) spectral index is used for this purpose, upon which we appropriately identify the clear and fuzzy value ranges. To reduce the data volume, a super-pixel segmentation of the images is also employed, implemented via the QuickShift algorithm. The cross-validation results showcase the effectiveness of the proposed algorithm, with the average commission and omission errors being 9% and 2%, respectively, and the average Matthews correlation coefficient (MCC) equal to 0.93.

A Hybrid Ensemble Learning Approach Utilizing Light Gradient Boosting Machine and Category Boosting Model for Lifestyle-Based Prediction of Type-II Diabetes Mellitus

Addressing classification and prediction challenges, tree ensemble models have gained significant importance. Boosting ensemble techniques are commonly employed for forecasting Type-II diabetes mellitus. Light Gradient Boosting Machine (LightGBM) is a widely used algorithm known for its leaf growth strategy, loss reduction, and enhanced training precision. However, LightGBM is prone to overfitting. In contrast, CatBoost utilizes balanced base predictors known as decision tables, which mitigate overfitting risks and significantly improve testing time efficiency. CatBoost’s algorithm structure counteracts gradient boosting biases and incorporates an overfitting detector to stop training early. This study focuses on developing a hybrid model that combines LightGBM and CatBoost to minimize overfitting and improve accuracy by reducing variance. For the purpose of finding the best hyperparameters to use with the underlying learners, the Bayesian hyperparameter optimization method is used. By fine-tuning the regularization parameter values, the hybrid model effectively reduces variance (overfitting). Comparative evaluation against LightGBM, CatBoost, XGBoost, Decision Tree, Random Forest, AdaBoost, and GBM algorithms demonstrates that the hybrid model has the best F1-score (99.37%), recall (99.25%), and accuracy (99.37%). Consequently, the proposed framework holds promise for early diabetes prediction in the healthcare industry and exhibits potential applicability to other datasets sharing similarities with diabetes.

Prediction of undrained shear strength using extreme gradient boosting and random forest based on Bayesian optimization

Accurate assessment of undrained shear strength(USS)for soft sensitive clays is a great concern in geotechnical engineering practice.This study applies novel data-driven extreme gradient boosting(XGBoost)and random forest(RF)ensemble learning methods for capturing the relationships between the USS and various basic soil parameters.Based on the soil data sets from TC304 database,a general approach is developed to predict the USS of soft clays using the two machine learning methods above,where five feature variables including the preconsolidation stress(PS),vertical effective stress(VES),liquid limit(LL),plastic limit(PL)and natural water content(W)are adopted.To reduce the dependence on the rule of thumb and inefficient brute-force search,the Bayesian optimization method is applied to determine the appropriate model hyper-parameters of both XGBoost and RF.The developed models are comprehensively compared with three comparison machine learning methods and two transformation models with respect to predictive accuracy and robustness under 5-fold cross-validation(CV).It is shown that XGBoost-based and RF-based methods outperform these approaches.Besides,the XGBoostbased model provides feature importance ranks,which makes it a promising tool in the prediction of geotechnical parameters and enhances the interpretability of model.

Forecasting Multi-Step Ahead Monthly Reference Evapotranspiration Using Hybrid Extreme Gradient Boosting with Grey Wolf Optimization Algorithm

It is important for regional water resources management to know the agricultural water consumption information several months in advance.Forecasting reference evapotranspiration(ET_(0))in the next few months is important for irrigation and reservoir management.Studies on forecasting of multiple-month ahead ET_(0) using machine learning models have not been reported yet.Besides,machine learning models such as the XGBoost model has multiple parameters that need to be tuned,and traditional methods can get stuck in a regional optimal solution and fail to obtain a global optimal solution.This study investigated the performance of the hybrid extreme gradient boosting(XGBoost)model coupled with the Grey Wolf Optimizer(GWO)algorithm for forecasting multi-step ahead ET_(0)(1-3 months ahead),compared with three conventional machine learning models,i.e.,standalone XGBoost,multi-layer perceptron(MLP)and M5 model tree(M5)models in the subtropical zone of China.The results showed that theGWO-XGB model generally performed better than the other three machine learning models in forecasting 1-3 months ahead ET_(0),followed by the XGB,M5 and MLP models with very small differences among the three models.The GWO-XGB model performed best in autumn,while the MLP model performed slightly better than the other three models in summer.It is thus suggested to apply the MLP model for ET_(0) forecasting in summer but use the GWO-XGB model in other seasons.

Predicted Oil Recovery Scaling-Law Using Stochastic Gradient Boosting Regression Model

In the process of oil recovery,experiments are usually carried out on core samples to evaluate the recovery of oil,so the numerical data are fitted into a non-dimensional equation called scaling-law.This will be essential for determining the behavior of actual reservoirs.The global non-dimensional time-scale is a parameter for predicting a realistic behavior in the oil field from laboratory data.This non-dimensional universal time parameter depends on a set of primary parameters that inherit the properties of the reservoir fluids and rocks and the injection velocity,which dynamics of the process.One of the practical machine learning(ML)techniques for regression/classification problems is gradient boosting(GB)regression.The GB produces a prediction model as an ensemble of weak prediction models that can be done at each iteration by matching a least-squares base-learner with the current pseudoresiduals.Using a randomization process increases the execution speed and accuracy of GB.Hence in this study,we developed a stochastic regression model of gradient boosting(SGB)to forecast oil recovery.Different nondimensional time-scales have been used to generate data to be used with machine learning techniques.The SGB method has been found to be the best machine learning technique for predicting the non-dimensional time-scale,which depends on oil/rock properties.

Prediction of COVID-19 Confirmed Cases Using Gradient Boosting Regression Method

The fast spread of coronavirus disease(COVID-19)caused by SARSCoV-2 has become a pandemic and a serious threat to the world.As of May 30,2020,this disease had infected more than 6 million people globally,with hundreds of thousands of deaths.Therefore,there is an urgent need to predict confirmed cases so as to analyze the impact of COVID-19 and practice readiness in healthcare systems.This study uses gradient boosting regression(GBR)to build a trained model to predict the daily total confirmed cases of COVID-19.The GBR method can minimize the loss function of the training process and create a single strong learner from weak learners.Experiments are conducted on a dataset of daily confirmed COVID-19 cases from January 22,2020,to May 30,2020.The results are evaluated on a set of evaluation performance measures using 10-fold cross-validation to demonstrate the effectiveness of the GBR method.The results reveal that the GBR model achieves 0.00686 root mean square error,the lowest among several comparative models.

Grasshopper KUWAHARA and Gradient Boosting Tree for Optimal Features Classifications

This paper aims to design an optimizer followed by a Kawahara filter for optimal classification and prediction of employees’performance.The algorithm starts by processing data by a modified K-means technique as a hierarchical clustering method to quickly obtain the best features of employees to reach their best performance.The work of this paper consists of two parts.The first part is based on collecting data of employees to calculate and illustrate the performance of each employee.The second part is based on the classification and prediction techniques of the employee performance.This model is designed to help companies in their decisions about the employees’performance.The classification and prediction algorithms use the Gradient Boosting Tree classifier to classify and predict the features.Results of the paper give the percentage of employees which are expected to leave the company after predicting their performance for the coming years.Results also show that the Grasshopper Optimization,followed by“KF”with the Gradient Boosting Tree as classifier and predictor,is characterized by a high accuracy.The proposed algorithm is compared with other known techniques where our results are fund to be superior.

Coal Rock Condition Detection Model Using Acoustic Emission and Light Gradient Boosting Machine

Coal rock mass instability fracture may result in serious hazards to underground coal mining.Acoustic emissions(AE)stimulated by internal structure fracture should carry lots of favorable information about health condition of rock mass.AE as a sensitive non-destructive test method is gradually utilized to detect anomaly conditions of coal rock.This paper proposes an improved multi-resolution feature to extract AE waveform at different frequency resolutions using Coilflet Wavelet Transform method(CWT).It is further adopt an efficient Light Gradient Boosting Machine(LightGBM)by several cascaded sub weak classifier models to merge AE features at different views of frequency for coal rock anomaly damage recognition.The results denote that the proposed method achieves excellent recognition performance on anomaly damage levels of coal rock.It is an effective method to detect the critical stability further to predict the rock mass bursting in time.

Stochastic Gradient Boosting Model for Twitter Spam Detection

In today’s world of connectivity there is a huge amount of data than we could imagine.The number of network users are increasing day by day and there are large number of social networks which keeps the users connected all the time.These social networks give the complete independence to the user to post the data either political,commercial or entertainment value.Some data may be sensitive and have a greater impact on the society as a result.The trustworthiness of data is important when it comes to public social networking sites like facebook and twitter.Due to the large user base and its openness there is a huge possibility to spread spam messages in this network.Spam detection is a technique to identify and mark data as a false data value.There are lot of machine learning approaches proposed to detect spam in social networks.The efficiency of any spam detection algorithm is determined by its cost factor and accuracy.Aiming to improve the detection of spam in the social networks this study proposes using statistical based features that are modelled through the supervised boosting approach called Stochastic gradient boosting to evaluate the twitter data sets in the English language.The performance of the proposed model is evaluated using simulation results.

Flight Delay Prediction Using Gradient Boosting Machine Learning Classifiers

With the increasing of civil aviation business,flight delay has become a key problem in civil aviation field in recent years,which has brought a considerable economic impact to airlines and related industries.The delay prediction of specific flights is very important for airlines’plan,airport resource allocation,insurance company strategy and personal arrangement.The influence factors of flight delay have high complexity and non-linear relationship.The different situations of various regions and airports,and even the deviation of airport or airline arrangement all have certain influence on flight delay,which makes the prediction more difficult.In view of the limitations of the existing delay prediction models,this paper proposes a flight delay prediction model with more generalization ability and corresponding machine learning classification algorithm.This model fully exploits temporal and spatial characteristics of higher dimensions,such as the influence of preceding flights,the situation of departure and landing airports,and the overall situation of flights on the same route.In the process of machine learning,the model is trained with historical data and tested with the latest actual data.The test result shows that the model and this machine learning algorithm can provide an effective method for the prediction of flight delay.

Application of extreme gradient boosting in predicting the viscoelastic characteristics of graphene oxide modified asphalt at medium and high temperatures

Complex modulus(G^(*))is one of the important criteria for asphalt classification according to AASHTO M320-10,and is often used to predict the linear viscoelastic behavior of asphalt binders.In addition,phase angle(φ)characterizes the deformation resilience of asphalt and is used to assess the ratio between the viscous and elastic components.It is thus important to quickly and accurately estimate these two indicators.The purpose of this investigation is to construct an extreme gradient boosting(XGB)model to predict G^(*)andφof graphene oxide(GO)modified asphaltat medium and high temperatures.Two data sets are gathered from previously published experiments,consisting of 357 samples for G^(*)and 339 samples forφ,and the se are used to develop the XGB model using nine inputs representing theasphalt binder components.The findings show that XGB is an excellent predictor of G^(*)andφof GO-modified asphalt,evaluated by the coefficient of determination R^(2)(R^(2)=0.990 and 0.9903 for G^(*)andφ,respectively)and root mean square error(RMSE=31.499 and 1.08 for G^(*)andφ,respectively).In addition,the model’s performance is compared with experimental results and five other machine learning(ML)models to highlight its accuracy.In the final step,the Shapley additive explanations(SHAP)value analysis is conducted to assess the impact of each input and the correlation between pairs of important features on asphalt’s two physical properties.

Predicting distant metastasis in nasopharyngeal carcinoma using gradient boosting tree model based on detailed magnetic resonance imaging reports

BACKGROUND Development of distant metastasis(DM)is a major concern during treatment of nasopharyngeal carcinoma(NPC).However,studies have demonstrated im-proved distant control and survival in patients with advanced NPC with the addition of chemotherapy to concomitant chemoradiotherapy.Therefore,precise prediction of metastasis in patients with NPC is crucial.AIM To develop a predictive model for metastasis in NPC using detailed magnetic resonance imaging(MRI)reports.METHODS This retrospective study included 792 patients with non-distant metastatic NPC.A total of 469 imaging variables were obtained from detailed MRI reports.Data were stratified and randomly split into training(50%)and testing sets.Gradient boosting tree(GBT)models were built and used to select variables for predicting DM.A full model comprising all variables and a reduced model with the top-five variables were built.Model performance was assessed by area under the curve(AUC).RESULTS Among the 792 patients,94 developed DM during follow-up.The number of metastatic cervical nodes(30.9%),tumor invasion in the posterior half of the nasal cavity(9.7%),two sides of the pharyngeal recess(6.2%),tubal torus(3.3%),and single side of the parapharyngeal space(2.7%)were the top-five contributors for predicting DM,based on their relative importance in GBT models.The testing AUC of the full model was 0.75(95%confidence interval[CI]:0.69-0.82).The testing AUC of the reduced model was 0.75(95%CI:0.68-0.82).For the whole dataset,the full(AUC=0.76,95%CI:0.72-0.82)and reduced models(AUC=0.76,95%CI:0.71-0.81)outperformed the tumor node-staging system(AUC=0.67,95%CI:0.61-0.73).CONCLUSION The GBT model outperformed the tumor node-staging system in predicting metastasis in NPC.The number of metastatic cervical nodes was identified as the principal contributing variable.

Rapid Prediction Model for Urban Floods Based on a Light Gradient Boosting Machine Approach and Hydrological–Hydraulic Model

Global climate change and sea level rise have led to increased losses from flooding.Accurate prediction of floods is essential to mitigating flood losses in coastal cities.Physically based models cannot satisfy the demand for real-time prediction for urban flooding due to their computational complexity.In this study,we proposed a hybrid modeling approach for rapid prediction of urban floods,coupling the physically based model with the light gradient boosting machine(LightGBM)model.A hydrological–hydraulic model was used to provide sufficient data for the LightGBM model based on the personal computer storm water management model(PCSWMM).The variables related to rainfall,tide level,and the location of flood points were used as the input for the LightGBM model.To improve the prediction accuracy,the hyperparameters of the LightGBM model are optimized by grid search algorithm and K-fold cross-validation.Taking Haidian Island,Hainan Province,China as a case study,the optimum values of the learning rate,number of estimators,and number of leaves of the LightGBM model are 0.11,450,and 12,respectively.The Nash-Sutcliffe efficiency coefficient(NSE)of the LightGBM model on the test set is 0.9896,indicating that the LightGBM model has reliable predictions and outperforms random forest(RF),extreme gradient boosting(XGBoost),and k-nearest neighbor(KNN).From the LightGBM model,the variables related to tide level were analyzed as the dominant variables for predicting the inundation depth based on the Gini index in the study area.The proposed LightGBM model provides a scientific reference for flood control in coastal cities considering its superior performance and computational efficiency.

Compressive strength prediction and optimization design of sustainable concrete based on squirrel search algorithm-extreme gradient boosting technique

Concrete is the most commonly used construction material.However,its production leads to high carbon dioxide(CO_(2))emissions and energy consumption.Therefore,developing waste-substitutable concrete components is necessary.Improving the sustainability and greenness of concrete is the focus of this research.In this regard,899 data points were collected from existing studies where cement,slag,fly ash,superplasticizer,coarse aggregate,and fine aggregate were considered potential influential factors.The complex relationship between influential factors and concrete compressive strength makes the prediction and estimation of compressive strength difficult.Instead of the traditional compressive strength test,this study combines five novel metaheuristic algorithms with extreme gradient boosting(XGB)to predict the compressive strength of green concrete based on fly ash and blast furnace slag.The intelligent prediction models were assessed using the root mean square error(RMSE),coefficient of determination(R^(2)),mean absolute error(MAE),and variance accounted for(VAF).The results indicated that the squirrel search algorithm-extreme gradient boosting(SSA-XGB)yielded the best overall prediction performance with R^(2) values of 0.9930 and 0.9576,VAF values of 99.30 and 95.79,MAE values of 0.52 and 2.50,RMSE of 1.34 and 3.31 for the training and testing sets,respectively.The remaining five prediction methods yield promising results.Therefore,the developed hybrid XGB model can be introduced as an accurate and fast technique for the performance prediction of green concrete.Finally,the developed SSA-XGB considered the effects of all the input factors on the compressive strength.The ability of the model to predict the performance of concrete with unknown proportions can play a significant role in accelerating the development and application of sustainable concrete and furthering a sustainable economy.

Stability prediction of underground entry-type excavations based on particle swarm optimization and gradient boosting decision tree

The stability of underground entry-type excavations will directly affect the working environment and the safety of staff.Empirical critical span graphs and traditional statistics learning methods can not meet the requirements of high accuracy for stability assessment of entry-type excavations.Therefore,this study proposes a new prediction method based on machine learning to scientifically adjust the critical span graph.Accordingly,the particle swarm optimization(PSO)algorithm is used to optimize the core parameters of the gradient boosting decision tree(GBDT),abbreviated as PSO-GBDT.Moreover,the classification performance of eight other classifiers including GDBT,k-nearest neighbors(KNN),two kinds of support vector machines(SVM),Gaussian naive Bayes(GNB),logistic regression(LR)and linear discriminant analysis(LDA)are also applied to compare with the proposed model.Findings revealed that compared with the other eight models,the prediction performance of PSO-GBDT is undoubtedly the most reliable,and its classification accuracy is up to 0.93.Therefore,this model has great potential to provide a more scientific and accurate choice for the stability prediction of underground excavations.In addition,each classification model is used to predict the stability category of several grid points divided by the critical span graph,and the updated critical span graph of each model is discussed in combination with previous studies.The results show that the PSO-GBDT model has the advantages of being scientific,accurate and efficient in updating the critical span graph,and its output decision boundary has strict theoretical support,which can help mine operators make favorable economic decisions.