GENERALIZATION PERFORMANCE OF MULTI-CATEGORY KERNEL MACHINES——In Memory of Professor Sun Yongsheng

Support vector machines are originally designed for binary classification. How to effectively extend it for multi-class classification is still an on-going research issue. In this paper, we consider kernel machines which are natural extensions of multi-category support vector machines originally proposed by Crammer and Singer. Based on the algorithm stability, we obtain the generalization error bounds for the kernel machines proposed in the paper.

Power Transformer Fault Diagnosis Using Random Forest and Optimized Kernel Extreme Learning Machine

Power transformer is one of the most crucial devices in power grid.It is significant to determine incipient faults of power transformers fast and accurately.Input features play critical roles in fault diagnosis accuracy.In order to further improve the fault diagnosis performance of power trans-formers,a random forest feature selection method coupled with optimized kernel extreme learning machine is presented in this study.Firstly,the random forest feature selection approach is adopted to rank 42 related input features derived from gas concentration,gas ratio and energy-weighted dissolved gas analysis.Afterwards,a kernel extreme learning machine tuned by the Aquila optimization algorithm is implemented to adjust crucial parameters and select the optimal feature subsets.The diagnosis accuracy is used to assess the fault diagnosis capability of concerned feature subsets.Finally,the optimal feature subsets are applied to establish fault diagnosis model.According to the experimental results based on two public datasets and comparison with 5 conventional approaches,it can be seen that the average accuracy of the pro-posed method is up to 94.5%,which is superior to that of other conventional approaches.Fault diagnosis performances verify that the optimum feature subset obtained by the presented method can dramatically improve power transformers fault diagnosis accuracy.

On the use of kernel machines for Mendelian randomization

Background： Properly adjusting for unmeasured confounders is critical for health studies in order to achieve valid testing and estimation of the exposure＇s causal effect on outcomes. The instrumental variable （IV） method has long been used in econometrics to estimate causal effects while accommodating the effect of unmeasured confounders. Mendefian randomization （MR）, which uses genetic variants as the instrumental variables, is an application of the instrumental variable method to biomedical research fields, and has become popular in recent years. One often-used estimator of causal effects for instrumental variables and Mendelian randomization is the two-stage least square estimator （TSLS）. The validity of TSLS relies on the accurate prediction of exposure based on IVs in its first stage. Results： In this note, we propose to model the link between exposure and genetic IVs using the least-squares kernel machine （LSKM）. Some simulation studies are used to evaluate the feasibility of LSKM in TSLS setting. Conclusions： Our results show that LSKM based on genotype score or genotype can be used effectively in TSLS. It may provide higher power when the association between exposure and genetic IVs is nonlinear.

Anomaly Detection of UAV State Data Based on Single-Class Triangular Global Alignment Kernel Extreme Learning Machine

Unmanned Aerial Vehicles(UAVs)are widely used and meet many demands in military and civilian fields.With the continuous enrichment and extensive expansion of application scenarios,the safety of UAVs is constantly being challenged.To address this challenge,we propose algorithms to detect anomalous data collected from drones to improve drone safety.We deployed a one-class kernel extreme learning machine(OCKELM)to detect anomalies in drone data.By default,OCKELM uses the radial basis(RBF)kernel function as the kernel function of themodel.To improve the performance ofOCKELM,we choose a TriangularGlobalAlignmentKernel(TGAK)instead of anRBF Kernel and introduce the Fast Independent Component Analysis(FastICA)algorithm to reconstruct UAV data.Based on the above improvements,we create a novel anomaly detection strategy FastICA-TGAK-OCELM.The method is finally validated on the UCI dataset and detected on the Aeronautical Laboratory Failures and Anomalies(ALFA)dataset.The experimental results show that compared with other methods,the accuracy of this method is improved by more than 30%,and point anomalies are effectively detected.

Word Sense Disambiguation Based Sentiment Classification Using Linear Kernel Learning Scheme

Word Sense Disambiguation has been a trending topic of research in Natural Language Processing and Machine Learning.Mining core features and performing the text classification still exist as a challenging task.Here the features of the context such as neighboring words like adjective provide the evidence for classification using machine learning approach.This paper presented the text document classification that has wide applications in information retrieval,which uses movie review datasets.Here the document indexing based on controlled vocabulary,adjective,word sense disambiguation,generating hierarchical cate-gorization of web pages,spam detection,topic labeling,web search,document summarization,etc.Here the kernel support vector machine learning algorithm helps to classify the text and feature extract is performed by cuckoo search opti-mization.Positive review and negative review of movie dataset is presented to get the better classification accuracy.Experimental results focused with context mining,feature analysis and classification.By comparing with the previous work,proposed work designed to achieve the efficient results.Overall design is per-formed with MATLAB 2020a tool.

Correlation between Combined Urinary Metal Exposure and Grip Strength under Three Statistical Models:A Cross-sectional Study in Rural Guangxi

Objective This study aimed to investigate the potential relationship between urinary metals copper(Cu),arsenic(As),strontium(Sr),barium(Ba),iron(Fe),lead(Pb)and manganese(Mn)and grip strength.Methods We used linear regression models,quantile g-computation and Bayesian kernel machine regression(BKMR)to assess the relationship between metals and grip strength.Results In the multimetal linear regression,Cu(β=−2.119),As(β=−1.318),Sr(β=−2.480),Ba(β=0.781),Fe(β=1.130)and Mn(β=−0.404)were significantly correlated with grip strength(P<0.05).The results of the quantile g-computation showed that the risk of occurrence of grip strength reduction was−1.007(95%confidence interval:−1.362,−0.652;P<0.001)when each quartile of the mixture of the seven metals was increased.Bayesian kernel function regression model analysis showed that mixtures of the seven metals had a negative overall effect on grip strength,with Cu,As and Sr being negatively associated with grip strength levels.In the total population,potential interactions were observed between As and Mn and between Cu and Mn(P_(interactions) of 0.003 and 0.018,respectively).Conclusion In summary,this study suggests that combined exposure to metal mixtures is negatively associated with grip strength.Cu,Sr and As were negatively correlated with grip strength levels,and there were potential interactions between As and Mn and between Cu and Mn.

Research on the IL-Bagging-DHKELM Short-Term Wind Power Prediction Algorithm Based on Error AP Clustering Analysis

With the continuous advancement of China’s“peak carbon dioxide emissions and Carbon Neutrality”process,the proportion of wind power is increasing.In the current research,aiming at the problem that the forecasting model is outdated due to the continuous updating of wind power data,a short-term wind power forecasting algorithm based on Incremental Learning-Bagging Deep Hybrid Kernel Extreme Learning Machine(IL-Bagging-DHKELM)error affinity propagation cluster analysis is proposed.The algorithm effectively combines deep hybrid kernel extreme learning machine(DHKELM)with incremental learning(IL).Firstly,an initial wind power prediction model is trained using the Bagging-DHKELM model.Secondly,Euclidean morphological distance affinity propagation AP clustering algorithm is used to cluster and analyze the prediction error of wind power obtained from the initial training model.Finally,the correlation between wind power prediction errors and Numerical Weather Prediction(NWP)data is introduced as incremental updates to the initial wind power prediction model.During the incremental learning process,multiple error performance indicators are used to measure the overall model performance,thereby enabling incremental updates of wind power models.Practical examples show the method proposed in this article reduces the root mean square error of the initial model by 1.9 percentage points,indicating that this method can be better adapted to the current scenario of the continuous increase in wind power penetration rate.The accuracy and precision of wind power generation prediction are effectively improved through the method.

Prediction of flyrock induced by mine blasting using a novel kernel-based extreme learning machine

Blasting is a common method of breaking rock in surface mines.Although the fragmentation with proper size is the main purpose,other undesirable effects such as flyrock are inevitable.This study is carried out to evaluate the capability of a novel kernel-based extreme learning machine algorithm,called kernel extreme learning machine(KELM),by which the flyrock distance(FRD) is predicted.Furthermore,the other three data-driven models including local weighted linear regression(LWLR),response surface methodology(RSM) and boosted regression tree(BRT) are also developed to validate the main model.A database gathered from three quarry sites in Malaysia is employed to construct the proposed models using 73 sets of spacing,burden,stemming length and powder factor data as inputs and FRD as target.Afterwards,the validity of the models is evaluated by comparing the corresponding values of some statistical metrics and validation tools.Finally,the results verify that the proposed KELM model on account of highest correlation coefficient(R) and lowest root mean square error(RMSE) is more computationally efficient,leading to better predictive capability compared to LWLR,RSM and BRT models for all data sets.

Dynamic model for predicting nitrogen oxide concentration at outlet of selective catalytic reduction denitrification system based on kernel extreme learning machine

To solve the increasing model complexity due to several input variables and large correlations under variable load conditions,a dynamic modeling method combining a kernel extreme learning machine(KELM)and principal component analysis(PCA)was proposed and applied to the prediction of nitrogen oxide(NO_(x))concentration at the outlet of a selective catalytic reduction(SCR)denitrification system.First,PCA is applied to the feature information extraction of input data,and the current and previous sequence values of the extracted information are used as the inputs of the KELM model to reflect the dynamic characteristics of the NO_(x)concentration at the SCR outlet.Then,the model takes the historical data of the NO_(x)concentration at the SCR outlet as the model input to improve its accuracy.Finally,an optimization algorithm is used to determine the optimal parameters of the model.Compared with the Gaussian process regression,long short-term memory,and convolutional neural network models,the prediction errors are reduced by approximately 78.4%,67.6%,and 59.3%,respectively.The results indicate that the proposed dynamic model structure is reliable and can accurately predict NO_(x)concentrations at the outlet of the SCR system.

Determination of influential parameters for prediction of total sediment loads in mountain rivers using kernel-based approaches

It is important to have a reasonable estimation of sediment transport rate with respect to its significant role in the planning and management of water resources projects. The complicate nature of sediment transport in gravel-bed rivers causes inaccuracies of empirical formulas in the prediction of this phenomenon. Artificial intelligences as alternative approaches can provide solutions to such complex problems. The present study aimed at investigating the capability of kernel-based approaches in predicting total sediment loads and identification of influential parameters of total sediment transport. For this purpose, Gaussian process regression(GPR), Support vector machine(SVM) and kernel extreme learning machine(KELM) are applied to enhance the prediction level of total sediment loads in 19 mountain gravel-bed streams and rivers located in the United States. Several parameters based on two scenarios are investigated and consecutive predicted results are compared with some well-known formulas. Scenario 1 considers only hydraulic characteristics and on the other side, the second scenario was formed using hydraulic and sediment properties. The obtained results reveal that using the parameters of hydraulic conditions asinputs gives a good estimation of total sediment loads. Furthermore, it was revealed that KELM method with input parameters of Froude number(Fr), ratio of average velocity(V) to shear velocity(U*) and shields number(θ) yields a correlation coefficient(R) of 0.951, a Nash-Sutcliffe efficiency(NSE) of 0.903 and root mean squared error(RMSE) of 0.021 and indicates superior results compared with other methods. Performing sensitivity analysis showed that the ratio of average velocity to shear flow velocity and the Froude number are the most effective parameters in predicting total sediment loads of gravel-bed rivers.

Deep Learning Applied to Computational Mechanics:A Comprehensive Review,State of the Art,and the Classics

Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.

复合地层小直径隧道掘进机掘进速度区间预测

合理准确预测隧道掘进机(tunnel boring machine,TBM)的掘进速度是实现TBM智能化控制的关键问题之一,复合地层小直径TBM施工的不确定性较常规地质条件更强,而传统预测方法对施工过程的不确定性考虑不足。在此通过引入区间预测方法,提出基于4种不同Bootstrap方法结合KELM-ANN模型的TBM掘进速度区间预测模型,并以南水北调安阳输水隧洞工程为例,选取142组工程实测数据验证区间预测模型的有效性。研究结果表明:基于Rademacher分布建立的模型预测结果优于其他3种方法,不仅可以得到较好的点预测结果,还可以构造出较为清晰可靠的区间将掘进速度实测值完全包络在内;随着置信水平的提高,区间可容纳的不确定性和风险也逐渐上升,通过变化区间宽度,能较好地量化和解释TBM施工过程中的不确定性因素对掘进速度的影响。研究结果可为TBM掘进性能预测和掘进参数优化提供参考。

Application of SABO-VMD-KELM in Fault Diagnosis of Wind Turbines

In order to improve the accuracy of wind turbine fault diagnosis,a wind turbine fault diagnosis method based on Subtraction-Average-Based Optimizer(SABO)optimized Variational Mode Decomposition(VMD)and Kernel Extreme Learning Machine(KELM)is proposed.Firstly,the SABO algorithm was used to optimize the VMD parameters and decompose the original signal to obtain the best modal components,and then the nine features were calculated to obtain the feature vectors.Secondly,the SABO algorithm was used to optimize the KELM parameters,and the training set and the test set were divided according to different proportions.The results were compared with the optimized model without SABO algorithm.The experimental results show that the fault diagnosis method of wind turbine based on SABO-VMD-KELM model can achieve fault diagnosis quickly and effectively,and has higher accuracy.

Deep kernel extreme learning machine classifier based on the improved sparrow search algorithm

In the classification problem,deep kernel extreme learning machine(DKELM)has the characteristics of efficient processing and superior performance,but its parameters optimization is difficult.To improve the classification accuracy of DKELM,a DKELM algorithm optimized by the improved sparrow search algorithm(ISSA),named as ISSA-DKELM,is proposed in this paper.Aiming at the parameter selection problem of DKELM,the DKELM classifier is constructed by using the optimal parameters obtained by ISSA optimization.In order to make up for the shortcomings of the basic sparrow search algorithm(SSA),the chaotic transformation is first applied to initialize the sparrow position.Then,the position of the discoverer sparrow population is dynamically adjusted.A learning operator in the teaching-learning-based algorithm is fused to improve the position update operation of the joiners.Finally,the Gaussian mutation strategy is added in the later iteration of the algorithm to make the sparrow jump out of local optimum.The experimental results show that the proposed DKELM classifier is feasible and effective,and compared with other classification algorithms,the proposed DKELM algorithm aciheves better test accuracy.

Privacy Preserving Blockchain Technique to Achieve Secure and Reliable Sharing of IoT Data

In present digital era,an exponential increase in Internet of Things(IoT)devices poses several design issues for business concerning security and privacy.Earlier studies indicate that the blockchain technology is found to be a significant solution to resolve the challenges of data security exist in IoT.In this view,this paper presents a new privacy-preserving Secure Ant Colony optimization with Multi Kernel Support Vector Machine(ACOMKSVM)with Elliptical Curve cryptosystem(ECC)for secure and reliable IoT data sharing.This program uses blockchain to ensure protection and integrity of some data while it has the technology to create secure ACOMKSVM training algorithms in partial views of IoT data,collected from various data providers.Then,ECC is used to create effective and accurate privacy that protects ACOMKSVM secure learning process.In this study,the authors deployed blockchain technique to create a secure and reliable data exchange platform across multiple data providers,where IoT data is encrypted and recorded in a distributed ledger.The security analysis showed that the specific data ensures confidentiality of critical data from each data provider and protects the parameters of the ACOMKSVM model for data analysts.To examine the performance of the proposed method,it is tested against two benchmark dataset such as Breast Cancer Wisconsin Data Set(BCWD)and Heart Disease Data Set(HDD)from UCI AI repository.The simulation outcome indicated that the ACOMKSVM model has outperformed all the compared methods under several aspects.

Short-term photovoltaic power prediction using combined K-SVD-OMP and KELM method

For photovoltaic power prediction,a kind of sparse representation modeling method using feature extraction techniques is proposed.Firstly,all these factors affecting the photovoltaic power output are regarded as the input data of the model.Next,the dictionary learning techniques using the K-mean singular value decomposition(K-SVD)algorithm and the orthogonal matching pursuit(OMP)algorithm are used to obtain the corresponding sparse encoding based on all the input data,i.e.the initial dictionary.Then,to build the global prediction model,the sparse coding vectors are used as the input of the model of the kernel extreme learning machine(KELM).Finally,to verify the effectiveness of the combined K-SVD-OMP and KELM method,the proposed method is applied to a instance of the photovoltaic power prediction.Compared with KELM,SVM and ELM under the same conditions,experimental results show that different combined sparse representation methods achieve better prediction results,among which the combined K-SVD-OMP and KELM method shows better prediction results and modeling accuracy.

Selective ensemble modeling based on nonlinear frequency spectral feature extraction for predicting load parameter in ball mills

Strong mechanical vibration and acoustical signals of grinding process contain useful information related to load parameters in ball mills. It is a challenge to extract latent features and construct soft sensor model with high dimensional frequency spectra of these signals. This paper aims to develop a selective ensemble modeling approach based on nonlinear latent frequency spectral feature extraction for accurate measurement of material to ball volume ratio. Latent features are first extracted from different vibrations and acoustic spectral segments by kernel partial least squares. Algorithms of bootstrap and least squares support vector machines are employed to produce candidate sub-models using these latent features as inputs. Ensemble sub-models are selected based on genetic algorithm optimization toolbox. Partial least squares regression is used to combine these sub-models to eliminate collinearity among their prediction outputs. Results indicate that the proposed modeling approach has better prediction performance than previous ones.

ECG quality assessment based on a kernel support vector machine and genetic algorithm with a feature matrix

We propose a systematic ECG quality classification method based on a kernel support vector machine(KSVM) and genetic algorithm(GA) to determine whether ECGs collected via mobile phone are acceptable or not. This method includes mainly three modules, i.e., lead-fall detection, feature extraction, and intelligent classification. First, lead-fall detection is executed to make the initial classification. Then the power spectrum, baseline drifts, amplitude difference, and other time-domain features for ECGs are analyzed and quantified to form the feature matrix. Finally, the feature matrix is assessed using KSVM and GA to determine the ECG quality classification results. A Gaussian radial basis function(GRBF) is employed as the kernel function of KSVM and its performance is compared with that of the Mexican hat wavelet function(MHWF). GA is used to determine the optimal parameters of the KSVM classifier and its performance is compared with that of the grid search(GS) method. The performance of the proposed method was tested on a database from PhysioNet/Computing in Cardiology Challenge 2011, which includes 1500 12-lead ECG recordings. True positive(TP), false positive(FP), and classification accuracy were used as the assessment indices. For training database set A(1000 recordings), the optimal results were obtained using the combination of lead-fall, GA, and GRBF methods, and the corresponding results were: TP 92.89%, FP 5.68%, and classification accuracy 94.00%. For test database set B(500 recordings), the optimal results were also obtained using the combination of lead-fall, GA, and GRBF methods, and the classification accuracy was 91.80%.

Sex-specific and dose-response relationships of urinary cobalt and molybdenum levels with glucose levels and insulin resistance in U.S. adults

Growing studies have linked metal exposure to diabetes risk.However,these studies had inconsistent results.We used a multiple linear regression model to investigate the sexspecific and dose-response associations between urinary metals(cobalt(Co)and molybdenum(Mo))and diabetes-related indicators(fasting plasma glucose(FPG),hemoglobin A1c(HbA1c),homeostasis model assessment for insulin resistance(HOMA-IR),and insulin)in a cross-sectional study based on the United States National Health and Nutrition Examination Survey.The urinary metal concentrations of 1423 eligible individuals were stratified on the basis of the quartile distribution.Our results showed that the urinary Co level in males at the fourth quartile(Q4)was strongly correlated with increased FPG(β=0.61,95%CI:0.17–1.04),HbA1c(β=0.31,95%CI:0.09–0.54),insulin(β=8.18,95%CI:2.84–13.52),and HOMA–IR(β=3.42,95%CI:1.40–5.44)when compared with first quartile(Q1).High urinary Mo levels(Q4 vs.Q1)were associated with elevated FPG(β=0.46,95%CI:0.17–0.75)and HbA1c(β=0.27,95%CI:0.11–0.42)in the overall population.Positive linear dose-response associations were observed between urinary Co and insulin(Pnonlinear=0.513)and HOMA–IR(Pnonlinear=0.736)in males,as well as a positive linear dose-response relationship between urinary Mo and FPG(Pnonlinear=0.826)and HbA1c(Pnonlinear=0.376)in the overall population.Significant sex-specific and dose-response relationships were observed between urinary metals(Co and Mo)and diabetes-related indicators,and the potential mechanisms should be further investigated.

Adaptive Barebones Salp Swarm Algorithm with Quasi-oppositional Learning for Medical Diagnosis Systems: A Comprehensive Analysis

The Salp Swarm Algorithm(SSA)may have trouble in dropping into stagnation as a kind of swarm intelligence method.This paper developed an adaptive barebones salp swarm algorithm with quasi-oppositional-based learning to compensate for the above weakness called QBSSA.In the proposed QBSSA,an adaptive barebones strategy can help to reach both accurate convergence speed and high solution quality;quasi-oppositional-based learning can make the population away from traping into local optimal and expand the search space.To estimate the performance of the presented method,a series of tests are performed.Firstly,CEC 2017 benchmark test suit is used to test the ability to solve the high dimensional and multimodal problems;then,based on QBSSA,an improved Kernel Extreme Learning Machine(KELM)model,named QBSSA–KELM,is built to handle medical disease diagnosis problems.All the test results and discussions state clearly that the QBSSA is superior to and very competitive to all the compared algorithms on both convergence speed and solutions accuracy.