Multi-layer perceptron-based data-driven multiscale modelling of granular materials with a novel Frobenius norm-based internal variable

One objective of developing machine learning(ML)-based material models is to integrate them with well-established numerical methods to solve boundary value problems(BVPs).In the family of ML models,recurrent neural networks(RNNs)have been extensively applied to capture history-dependent constitutive responses of granular materials,but these multiple-step-based neural networks are neither sufficiently efficient nor aligned with the standard finite element method(FEM).Single-step-based neural networks like the multi-layer perceptron(MLP)are an alternative to bypass the above issues but have to introduce some internal variables to encode complex loading histories.In this work,one novel Frobenius norm-based internal variable,together with the Fourier layer and residual architectureenhanced MLP model,is crafted to replicate the history-dependent constitutive features of representative volume element(RVE)for granular materials.The obtained ML models are then seamlessly embedded into the FEM to solve the BVP of a biaxial compression case and a rigid strip footing case.The obtained solutions are comparable to results from the FEM-DEM multiscale modelling but achieve significantly improved efficiency.The results demonstrate the applicability of the proposed internal variable in enabling MLP to capture highly nonlinear constitutive responses of granular materials.

Prediction of flyrock distance induced by mine blasting using a novel Harris Hawks optimization-based multi-layer perceptron neural network

In mining or construction projects,for exploitation of hard rock with high strength properties,blasting is frequently applied to breaking or moving them using high explosive energy.However,use of explosives may lead to the flyrock phenomenon.Flyrock can damage structures or nearby equipment in the surrounding areas and inflict harm to humans,especially workers in the working sites.Thus,prediction of flyrock is of high importance.In this investigation,examination and estimation/forecast of flyrock distance induced by blasting through the application of five artificial intelligent algorithms were carried out.One hundred and fifty-two blasting events in three open-pit granite mines in Johor,Malaysia,were monitored to collect field data.The collected data include blasting parameters and rock mass properties.Site-specific weathering index(WI),geological strength index(GSI) and rock quality designation(RQD)are rock mass properties.Multi-layer perceptron(MLP),random forest(RF),support vector machine(SVM),and hybrid models including Harris Hawks optimization-based MLP(known as HHO-MLP) and whale optimization algorithm-based MLP(known as WOA-MLP) were developed.The performance of various models was assessed through various performance indices,including a10-index,coefficient of determination(R^(2)),root mean squared error(RMSE),mean absolute percentage error(MAPE),variance accounted for(VAF),and root squared error(RSE).The a10-index values for MLP,RF,SVM,HHO-MLP and WOA-MLP are 0.953,0.933,0.937,0.991 and 0.972,respectively.R^(2) of HHO-MLP is 0.998,which achieved the best performance among all five machine learning(ML) models.

Prediction of Logistics Demand via Least Square Method and Multi-Layer Perceptron

To implement the prediction of the logistics demand capacity of a certain region,a comprehensive index system is constructed,which is composed of freight volume and other eight relevant economic indices,such as gross domestic product(GDP),consumer price index(CPI),total import and export volume,port's cargo throughput,total retail sales of consumer goods,total fixed asset investment,highway mileage,and resident population,to form the foundation for the model calculation.Based on the least square method(LSM)to fit the parameters,the study obtains an accurate mathematical model and predicts the changes of each index in the next five years.Using artificial intelligence software,the research establishes the logistics demand model of multi-layer perceptron(MLP)neural network,makes an empirical analysis on the logistics demand of Quanzhou City,and predicts its logistics demand in the next five years,which provides some references for formulating logistics planning and development strategy.

Current Harmonic Estimation in Power Transmission Lines Using Multi-layer Perceptron Learning Strategies

This main contribution of this work is to propose a new approach based on a structure of MLPs （multi-layer perceptrons） for identifying current harmonics in low power distribution systems. In this approach, MLPs are proposed and trained with signal sets that arc generated from real harmonic waveforms. After training, each trained MLP is able to identify the two coefficients of each harmonic term of the input signal. The effectiveness of the new approach is evaluated by two experiments and is also compared to another recent MLP method. Experimental results show that the proposed MLPs approach enables to identify effectively the amplitudes of harmonic terms from the signals under noisy condition. The new approach can be applied in harmonic compensation strategies with an active power filter to ensure power quality issues in electrical power systems.

MLP training in a self-organizing state space model using unscented Kalman particle filter

Many Bayesian learning approaches to the multi-layer perceptron （MLP） parameter optimization have been proposed such as the extended Kalman filter （EKF）. This paper uses the unscented Kalman particle filter （UPF） to train the MLP in a self- organizing state space （SOSS） model. This involves forming augmented state vectors consisting of all parameters （the weights of the MLP） and outputs. The UPF is used to sequentially update the true system states and high dimensional parameters that are inherent to the SOSS moder for the MLP simultaneously. Simulation results show that the new method performs better than traditional optimization methods.

Constitutive modelling of idealised granular materials using machine learning method

Predicting the constitutive response of granular soils is a fundamental goal in geomechanics.This paper presents a machine learning(ML)framework for the prediction of the stress-strain behaviour and shearinduced contact fabric evolution of an idealised granular material subject to triaxial shearing.The MLbased framework is comprised of a set of mini-triaxial tests which provide a benchmark for the setup and validation of the discrete element method(DEM)model of the granular materials,a parametric DEM simulation programme of virtual triaxial tests which provides datasets of micro-and macro-mechanical information,as well as a multi-layer perceptron(MLP)neural network which is trained and tested using the DEM-based datasets.The ML model only requires the initial void ratio of the granular sample as the input for predicting its constitutive response.The excellent agreement between the ML model prediction and experimental test and DEM simulation results indicates that the MLebased modelling approach is capable of capturing accurately the effects of initial void ratio on the constitutive behaviour of idealised granular materials,bypassing the need to incorporate the complex micromechanics underlying the macroscopic mechanical behaviour of granular materials.Lastly,a detailed comparison between the used MLP model and long short-term memory(LSTM)model was made from the perspective of technical algorithm,prediction accuracy,and computational efficiency.

Multi-objective optimization of the cathode catalyst layer micro-composition of polymer electrolyte membrane fuel cells using a multi-scale,two-phase fuel cell model and data-driven surrogates

Polymer electrolyte membrane fuel cells(PEMFCs)are considered a promising alternative to internal combustion engines in the automotive sector.Their commercialization is mainly hindered due to the cost and effectiveness of using platinum(Pt)in them.The cathode catalyst layer(CL)is considered a core component in PEMFCs,and its composition often considerably affects the cell performance(V_(cell))also PEMFC fabrication and production(C_(stack))costs.In this study,a data-driven multi-objective optimization analysis is conducted to effectively evaluate the effects of various cathode CL compositions on Vcelland Cstack.Four essential cathode CL parameters,i.e.,platinum loading(L_(Pt)),weight ratio of ionomer to carbon(wt_(I/C)),weight ratio of Pt to carbon(wt_(Pt/c)),and porosity of cathode CL(ε_(cCL)),are considered as the design variables.The simulation results of a three-dimensional,multi-scale,two-phase comprehensive PEMFC model are used to train and test two famous surrogates:multi-layer perceptron(MLP)and response surface analysis(RSA).Their accuracies are verified using root mean square error and adjusted R^(2).MLP which outperforms RSA in terms of prediction capability is then linked to a multi-objective non-dominated sorting genetic algorithmⅡ.Compared to a typical PEMFC stack,the results of the optimal study show that the single-cell voltage,Vcellis improved by 28 m V for the same stack price and the stack cost evaluated through the U.S department of energy cost model is reduced by$5.86/k W for the same stack performance.

A hybrid constriction coefficientbased particle swarm optimization and gravitational search algorithm for training multi-layer perceptron

Purpose-In this paper,a newly proposed hybridization algorithm namely constriction coefficient-based particle swarm optimization and gravitational search algorithm(CPSOGSA)has been employed for training MLP to overcome sensitivity to initialization,premature convergence,and stagnation in local optima problems of MLP.Design/methodology/approach-In this study,the exploration of the search space is carried out by gravitational search algorithm(GSA)and optimization of candidate solutions,i.e.exploitation is performed by particle swarm optimization(PSO).For training the multi-layer perceptron(MLP),CPSOGSA uses sigmoid fitness function for finding the proper combination of connection weights and neural biases to minimize the error.Secondly,a matrix encoding strategy is utilized for providing one to one correspondence between weights and biases of MLP and agents of CPSOGSA.Findings-The experimental findings convey that CPSOGSA is a better MLP trainer as compared to other stochastic algorithms because it provides superior results in terms of resolving stagnation in local optima and convergence speed problems.Besides,it gives the best results for breast cancer,heart,sine function and sigmoid function datasets as compared to other participating algorithms.Moreover,CPSOGSA also provides very competitive results for other datasets.Originality/value-The CPSOGSA performed effectively in overcoming stagnation in local optima problem and increasing the overall convergence speed of MLP.Basically,CPSOGSA is a hybrid optimization algorithm which has powerful characteristics of global exploration capability and high local exploitation power.In the research literature,a little work is available where CPSO and GSA have been utilized for training MLP.The only related research paper was given by Mirjalili et al.,in 2012.They have used standard PSO and GSA for training simple FNNs.However,the work employed only three datasets and used the MSE performance metric for evaluating the efficiency of the algorithms.In this paper,eight different standard datasets and five performance metrics have been utilized for investigating the efficiency of CPSOGSA in training MLPs.In addition,a non-parametric pair-wise statistical test namely the Wilcoxon rank-sum test has been carried out at a 5%significance level to statistically validate the simulation results.Besides,eight state-of-the-art metaheuristic algorithms were employed for comparative analysis of the experimental results to further raise the authenticity of the experimental setup.

Prediction of diabetes and hypertension using multi-layer perceptron neural networks

Background:Diabetes and hypertension are two of the commonest diseases in the world.As they unfavorably affect people of different age groups,they have become a cause of concern and must be predicted and diagnosed well in advance.Objective:This research aims to determine the effectiveness of artificial neural networks(ANNs)in predicting diabetes and blood pressure diseases and to point out the factors which have a high impact on these diseases.Sample:This work used two online datasets which consist of data collected from 768 individuals.We applied neural network algorithms to predict if the individuals have those two diseases based on some factors.Diabetes prediction is based on five factors:age,weight,fat-ratio,glucose,and insulin,while blood pressure prediction is based on six factors:age,weight,fat-ratio,blood pressure,alcohol,and smoking.Method:A model based on the Multi-Layer Perceptron Neural Network(MLP)was implemented.The inputs of the network were the factors for each disease,while the output was the prediction of the disease’s occurrence.The model performance was compared with other classifiers such as Support Vector Machine(SVM)and K-Nearest Neighbors(KNN).We used performance metrics measures to assess the accuracy and performance of MLP.Also,a tool was implemented to help diagnose the diseases and to understand the results.Result:The model predicted the two diseases with correct classification rate(CCR)of 77.6%for diabetes and 68.7%for hypertension.The results indicate that MLP correctly predicts the probability of being diseased or not,and the performance can be significantly increased compared with both SVM and KNN.This shows MLPs effectiveness in early disease prediction.

Modeling spatio-temporal distribution of soil moisture by deep learning-based cellular automata model

Soil moisture content （SMC） is a key hydrological parameter in agriculture,meteorology and climate change,and understanding of spatio-temporal distributions of SMC in farmlands is important to address the precise irrigation scheduling.However,the hybrid interaction of static and dynamic environmental parameters makes it particularly difficult to accurately and reliably model the distribution of SMC.At present,deep learning wins numerous contests in machine learning and hence deep belief network （DBN） ,a breakthrough in deep learning is trained to extract the transition functions for the simulation of the cell state changes.In this study,we used a novel macroscopic cellular automata （MCA） model by combining DBN to predict the SMC over an irrigated corn field （an area of 22 km^2） in the Zhangye oasis,Northwest China.Static and dynamic environmental variables were prepared with regard to the complex hydrological processes.The widely used neural network,multi-layer perceptron （MLP） ,was utilized for comparison to DBN.The hybrid models （MLP-MCA and DBN-MCA） were calibrated and validated on SMC data within four months,i.e.June to September 2012,which were automatically observed by a wireless sensor network （WSN） .Compared with MLP-MCA,the DBN-MCA model led to a decrease in root mean squared error （RMSE） by 18%.Thus,the differences of prediction errors increased due to the propagating errors of variables,difficulties of knowing soil properties and recording irrigation amount in practice.The sequential Gaussian simulation （s Gs） was performed to assess the uncertainty of soil moisture estimations.Calculated with a threshold of SMC for each grid cell,the local uncertainty of simulated results in the post processing suggested that the probability of SMC less than 25% will be difference in different areas at different time periods.The current results showed that the DBN-MCA model performs better than the MLP-MCA model,and the DBN-MCA model provides a powerful tool for predicting SMC in highly non-linear forms.Moreover,because modeling soil moisture by using environmental variables is gaining increasing popularity,DBN techniques could contribute a lot to enhancing the calibration of MCA-based SMC estimations and hence provide an alternative approach for SMC monitoring in irrigation systems on the basis of canals.

Parkinson’s Disease Detection Using Biogeography-Based Optimization

In recent years,Parkinson’s Disease(PD)as a progressive syndrome of the nervous system has become highly prevalent worldwide.In this study,a novel hybrid technique established by integrating a Multi-layer Perceptron Neural Network(MLP)with the Biogeography-based Optimization(BBO)to classify PD based on a series of biomedical voice measurements.BBO is employed to determine the optimal MLP parameters and boost prediction accuracy.The inputs comprised of 22 biomedical voice measurements.The proposed approach detects two PD statuses:0-disease status and 1-good control status.The performance of proposed methods compared with PSO,GA,ACO and ES method.The outcomes affirm that the MLP-BBO model exhibits higher precision and suitability for PD detection.The proposed diagnosis system as a type of speech algorithm detects early Parkinson’s symptoms,and consequently,it served as a promising new robust tool with excellent PD diagnosis performance.

Defect feature recognition method of glass fibre-reinforced structure based on visual image analysis

Glass fibre-reinforced(GFR)structure is extensively used in radome,spoiler and some other equipment.In engineering practice,due to the influence of wear,aging,impact,chemical corrosion of surface structure and other factors,the internal structure of this kind of structure gradually evolves into a defect state and expands to form defects such as bubbles,scratches,shorts,cracks,cavitation erosion,stains and other defects.These defects have posed a serious threat to the quality and performance of GFR structure.From the propagation process of GFR structure defects,its duration is random and may be very short.Therefore,designing a scientific micro defect intelligent detection system for GFR structure to enhance the maintainability of GFR structure will not only help to reduce emergencies,but also have positive theoretical significance and application value to ensure safe production and operation.Firstly,the defect detection mechanism of GFR structure is discussed,and the defect detection principle and defect area identification method are analyzed.Secondly,the processing process of defect edge signal is discussed,a classifier based on MLP is established,and the algorithm of the classifier is designed.Finally,the effectiveness of this method is proved by real-time monitoring and defect diagnosis of a typical GFR structure.The experimental results show that this method improves the efficiency of defect detection and has high defect feature recognition accuracy,which provides a new idea for the on-line detection of GFR structure defects.

An Improved SPSA Algorithm for System Identification Using Fuzzy Rules for Training Neural Networks

Simultaneous perturbation stochastic approximation （SPSA） belongs to the class of gradient-free optimization methods that extract gradient information from successive objective function evaluation. This paper describes an improved SPSA algorithm, which entails fuzzy adaptive gain sequences, gradient smoothing, and a step rejection procedure to enhance convergence and stability. The proposed fuzzy adaptive simultaneous perturbation approximation （FASPA） algorithm is particularly well suited to problems involving a large number of parameters such as those encountered in nonlinear system identification using neural networks （NNs）. Accordingly, a multilayer perceptron （MLP） network with popular training algorithms was used to predicate the system response. We found that an MLP trained by FASPSA had the desired accuracy that was comparable to results obtained by traditional system identification algorithms. Simulation results for typical nonlinear systems demonstrate that the proposed NN architecture trained with FASPSA yields improved system identification as measured by reduced time of convergence and a smaller identification error.

Extracting invariable fault features of rotating machines with multi-ICA networks

This paper proposes novel multi-layer neural networks based on Independent Component Analysis for feature extraction of fault modes. By the use of ICA, invariable features embedded in multi-channel vibration measurements under different operating conditions (rotating speed and/or load) can be captured together.Thus, stable MLP classifiers insensitive to the variation of operation conditions are constructed. The successful results achieved by selected experiments indicate great potential of ICA in health condition monitoring of rotating machines.

Environmental Sound Classification Using Deep Learning

Perhaps hearing impairment individuals cannot identify the environmental sounds due to noise around them.However,very little research has been conducted in this domain.Hence,the aim of this study is to categorize sounds generated in the environment so that the impairment individuals can distinguish the sound categories.To that end first we define nine sound classes--air conditioner,car horn,children playing,dog bark,drilling,engine idling,jackhammer,siren,and street music--typically exist in the environment.Then we record 100 sound samples from each category and extract features of each sound category using Mel-Frequency Cepstral Coefficients(MFCC).The training dataset is developed using this set of features together with the class variable;sound category.Sound classification is a complex task and hence,we use two Deep Learning techniques;Multi Layer Perceptron(MLP)and Convolution Neural Network(CNN)to train classification models.The models are tested using a separate test set and the performances of the models are evaluated using precision,recall and F1-score.The results show that the CNN model outperforms the MLP.However,the MLP also provided a decent accuracy in classifying unknown environmental sounds.

An Efficient Ensemble Model for Various Scale Medical Data

Electronic Health Records(EHRs)are the digital form of patients’medical reports or records.EHRs facilitate advanced analytics and aid in better decision-making for clinical data.Medical data are very complicated and using one classification algorithm to reach good results is difficult.For this reason,we use a combination of classification techniques to reach an efficient and accurate classification model.This model combination is called the Ensemble model.We need to predict new medical data with a high accuracy value in a small processing time.We propose a new ensemble model MDRL which is efficient with different datasets.The MDRL gives the highest accuracy value.It saves the processing time instead of processing four different algorithms sequentially;it executes the four algorithms in parallel.We implement five different algorithms on five variant datasets which are Heart Disease,Health General,Diabetes,Heart Attack,and Covid-19 Datasets.The four algorithms are Random Forest(RF),Decision Tree(DT),Logistic Regression(LR),and Multi-layer Perceptron(MLP).In addition to MDRL(our proposed ensemble model)which includes MLP,DT,RF,and LR together.From our experiments,we conclude that our ensemble model has the best accuracy value for most datasets.We reach that the combination of the Correlation Feature Selection(CFS)algorithm and our ensemble model is the best for giving the highest accuracy value.The accuracy values for our ensemble model based on CFS are 98.86,97.96,100,99.33,and 99.37 for heart disease,health general,Covid-19,heart attack,and diabetes datasets respectively.

A Sleep Scoring System Using EEG Combined Spectral and Detrended Fluctuation Analysis Features

Most of sleep disorders are diagnosed based on the sleep scoring and assessments. The purpose of this study is to combine detrended fluctuation analysis features and spectral features of single electroencephalograph (EEG) channel for the purpose of building an automated sleep staging system based on the hybrid prediction engine model. The testing results of the model were promising as the classification accuracies were 98.85%, 92.26%, 94.4%, 95.16% and 93.68% for the wake, non-rapid eye movement S1, non-rapid eye movement S2, non-rapid eye movement S3 and rapid eye movement sleep stages, respectively. The overall classification accuracy was 85.18%. We concluded that it might be possible to employ this approach to build an industrial sleep assessment system that reduced the number of channels that affected the sleep quality and the effort excreted by sleep specialists through the process of the sleep scoring.

Prediction of sintered density of binary W(Mo)alloys using machine learning

Powder metallurgy is the optimal method for the consolidation and preparation of W(Mo)alloys,which exhibit excellent application prospects at high temperatures.The properties of W(Mo)alloys are closely related to the sintered density.However,controlling the sintered density and porosity of these alloys is still challenging.In the past,the regulation methods mainly focused on timeconsuming and costly trial-and-error experiments.In this study,the sintering data for more than a dozen W(Mo)alloys constituted a small-scale dataset,including both solid and liquid phases sintering.Furthermore,simple descriptors were used to predict the sintered density of W(Mo)alloys based on the descriptor selection strategy and machine learning method(ML),where ML algorithm included the least absolute shrinkage and selection operator(Lasso)regression,k-nearest neighbor(k-NN),random forest(RF),and multi-layer perceptron(MLP).The results showed that the interpretable descriptors extracted by our proposed selection strategy and the MLP neural network achieved a high prediction accuracy(R>0.950).By further predicting the sintered density of W(Mo)alloys using different sintering processes,the error between the predicted and experimental values was less than 0.063,confirming the application potential of the model.

Parallel compact integration in handwritten Chinese character recognition

In this paper, a new parallel compact integration scheme based on multi-layer perceptron (MLP) networks is proposed to solve handwritten Chinese character recognition (HCCR) problems. The idea of metasynthesis is applied to HCCR, and compact MLP network classifier is defined. Human intelligence and computer capabilities are combined together effectively through a procedure of two-step supervised learning. Compared with previous integration schemes, this scheme is characterized with parallel compact structure and better performance. It provides a promising way for applying MLP to large vocabulary classification.

Exploiting articulatory features for pitch accent detection

Articulatory features describe how articulators are involved in making sounds.Speakers often use a more exaggerated way to pronounce accented phonemes,so articulatory features can be helpful in pitch accent detection.Instead of using the actual articulatory features obtained by direct measurement of articulators,we use the posterior probabilities produced by multi-layer perceptrons(MLPs) as articulatory features.The inputs of MLPs are frame-level acoustic features pre-processed using the split temporal context-2(STC-2) approach.The outputs are the posterior probabilities of a set of articulatory attributes.These posterior probabilities are averaged piecewise within the range of syllables and eventually act as syllable-level articulatory features.This work is the first to introduce articulatory features into pitch accent detection.Using the articulatory features extracted in this way,together with other traditional acoustic features,can improve the accuracy of pitch accent detection by about 2%.