Fully Distributed Learning for Deep Random Vector Functional-Link Networks

In the contemporary era, the proliferation of information technology has led to an unprecedented surge in data generation, with this data being dispersed across a multitude of mobile devices. Facing these situations and the training of deep learning model that needs great computing power support, the distributed algorithm that can carry out multi-party joint modeling has attracted everyone’s attention. The distributed training mode relieves the huge pressure of centralized model on computer computing power and communication. However, most distributed algorithms currently work in a master-slave mode, often including a central server for coordination, which to some extent will cause communication pressure, data leakage, privacy violations and other issues. To solve these problems, a decentralized fully distributed algorithm based on deep random weight neural network is proposed. The algorithm decomposes the original objective function into several sub-problems under consistency constraints, combines the decentralized average consensus (DAC) and alternating direction method of multipliers (ADMM), and achieves the goal of joint modeling and training through local calculation and communication of each node. Finally, we compare the proposed decentralized algorithm with several centralized deep neural networks with random weights, and experimental results demonstrate the effectiveness of the proposed algorithm.

Underwater Image Classification Based on EfficientnetB0 and Two-Hidden-Layer Random Vector Functional Link

The ocean plays an important role in maintaining the equilibrium of Earth’s ecology and providing humans access to a wealth of resources.To obtain a high-precision underwater image classification model,we propose a classification model that combines an EfficientnetB0 neural network and a two-hidden-layer random vector functional link network(EfficientnetB0-TRVFL).The features of underwater images were extracted using the EfficientnetB0 neural network pretrained via ImageNet,and a new fully connected layer was trained on the underwater image dataset using the transfer learning method.Transfer learning ensures the initial performance of the network and helps in the development of a high-precision classification model.Subsequently,a TRVFL was proposed to improve the classification property of the model.Net construction of the two hidden layers exhibited a high accuracy when the same hidden layer nodes were used.The parameters of the second hidden layer were obtained using a novel calculation method,which reduced the outcome error to improve the performance instability caused by the random generation of parameters of RVFL.Finally,the TRVFL classifier was used to classify features and obtain classification results.The proposed EfficientnetB0-TRVFL classification model achieved 87.28%,74.06%,and 99.59%accuracy on the MLC2008,MLC2009,and Fish-gres datasets,respectively.The best convolutional neural networks and existing methods were stacked up through box plots and Kolmogorov-Smirnov tests,respectively.The increases imply improved systematization properties in underwater image classification tasks.The image classification model offers important performance advantages and better stability compared with existing methods.

Optimized functional linked neural network for predicting diaphragm wall deflection induced by braced excavations in clays

Deep excavation during the construction of underground systems can cause movement on the ground,especially in soft clay layers.At high levels,excessive ground movements can lead to severe damage to adjacent structures.In this study,finite element analyses(FEM)and the hardening small strain(HSS)model were performed to investigate the deflection of the diaphragm wall in the soft clay layer induced by braced excavations.Different geometric and mechanical properties of the wall were investigated to study the deflection behavior of the wall in soft clays.Accordingly,1090 hypothetical cases were surveyed and simulated based on the HSS model and FEM to evaluate the wall deflection behavior.The results were then used to develop an intelligent model for predicting wall deflection using the functional linked neural network(FLNN)with different functional expansions and activation functions.Although the FLNN is a novel approach to predict wall deflection;however,in order to improve the accuracy of the FLNN model in predicting wall deflection,three swarm-based optimization algorithms,such as artificial bee colony(ABC),Harris’s hawk’s optimization(HHO),and hunger games search(HGS),were hybridized to the FLNN model to generate three novel intelligent models,namely ABC-FLNN,HHO-FLNN,HGS-FLNN.The results of the hybrid models were then compared with the basic FLNN and MLP models.They revealed that FLNN is a good solution for predicting wall deflection,and the application of different functional expansions and activation functions has a significant effect on the outcome predictions of the wall deflection.It is remarkably interesting that the performance of the FLNN model was better than the MLP model with a mean absolute error(MAE)of 19.971,root-mean-squared error(RMSE)of 24.574,and determination coefficient(R^(2))of 0.878.Meanwhile,the performance of the MLP model only obtained an MAE of 20.321,RMSE of 27.091,and R^(2)of 0.851.Furthermore,the results also indicated that the proposed hybrid models,i.e.,ABC-FLNN,HHO-FLNN,HGS-FLNN,yielded more superior performances than those of the FLNN and MLP models in terms of the prediction of deflection behavior of diaphragm walls with an MAE in the range of 11.877 to 12.239,RMSE in the range of 15.821 to 16.045,and R^(2)in the range of 0.949 to 0.951.They can be used as an alternative tool to simulate diaphragm wall deflections under different conditions with a high degree of accuracy.

Functional Link Neural Network for Predicting Crystallization Temperature of Ammonium Chloride in Air Cooler System

The air cooler is an important equipment in the petroleum refining industry.Ammonium chloride(NH4 Cl)deposition-induced corrosion is one of its main failure forms.In this study,the ammonium salt crystallization temperature is chosen as the key decision variable of NH4 Cl deposition-induced corrosion through in-depth mechanism research and experimental analysis.The functional link neural network(FLNN)is adopted as the basic algorithm for modeling because of its advantages in dealing with non-linear problems and its fast-computational ability.A hybrid FLNN attached to a small norm is built to improve the generalization performance of the model.Then,the trained model is used to predict the NH4 Cl salt crystallization temperature in the air cooler of a sour water stripper plant.Experimental results show the proposed improved FLNN algorithm can achieve better generalization performance than the PLS,the back propagation neural network,and the conventional FLNN models.

Application of functional-link neural network in evaluation of sublayer suspension based on FWD test

Several methods for evaluating the sublayer suspension beneath old pavement with falling weight deflectormeter(FWD), were summarized and the respective advantages and disadvantages were analyzed. Based on these methods, the evaluation principles were improved and a new type of the neural network, functional-link neural network was proposed to evaluate the sublayer suspension with FWD test results. The concept of function link, learning method of functional-link neural network and the establishment process of neural network model were studied in detail. Based on the old pavement over-repairing engineering of Kaiping section, Guangdong Province in G325 National Highway, the application of functional-link neural network in evaluation of sublayer suspension beneath old pavement based on FWD test data on the spot was investigated. When learning rate is 0.1 and training cycles are 405, the functional-link network error is less than 0.000 1, while the optimum chosen 4-8-1 BP needs over 10 000 training cycles to reach the same accuracy with less precise evaluation results. Therefore, in contrast to common BP neural network,the functional-link neural network adopts single layer structure to learn and calculate, which simplifies the network, accelerates the convergence speed and improves the accuracy. Moreover the trained functional-link neural network can be (adopted) to directly evaluate the sublayer suspension based on FWD test data on the site. Engineering practice indicates that the functional-link neural model gains very excellent results and effectively guides the pavement over-repairing construction.

Recovery of saturated signal waveform acquired from high-energy particles with artificial neural networks

Artificial neural networks(ANNs)are a core component of artificial intelligence and are frequently used in machine learning.In this report,we investigate the use of ANNs to recover the saturated signals acquired in highenergy particle and nuclear physics experiments.The inherent properties of the detector and hardware imply that particles with relatively high energies probably often generate saturated signals.Usually,these saturated signals are discarded during data processing,and therefore,some useful information is lost.Thus,it is worth restoring the saturated signals to their normal form.The mapping from a saturated signal waveform to a normal signal waveform constitutes a regression problem.Given that the scintillator and collection usually do not form a linear system,typical regression methods such as multi-parameter fitting are not immediately applicable.One important advantage of ANNs is their capability to process nonlinear regression problems.To recover the saturated signal,three typical ANNs were tested including backpropagation(BP),simple recurrent(Elman),and generalized radial basis function(GRBF)neural networks(NNs).They represent a basic network structure,a network structure with feedback,and a network structure with a kernel function,respectively.The saturated waveforms were produced mainly by the environmental gamma in a liquid scintillation detector for the China Dark Matter Detection Experiment(CDEX).The training and test data sets consisted of 6000 and 3000 recordings of background radiation,respectively,in which saturation was simulated by truncating each waveform at 40%of the maximum signal.The results show that the GBRF-NN performed best as measured using a Chi-squared test to compare the original and reconstructed signals in the region in which saturation was simulated.A comparison of the original and reconstructed signals in this region shows that the GBRF neural network produced the best performance.This ANN demonstrates a powerful efficacy in terms of solving the saturation recovery problem.The proposed method outlines new ideas and possibilities for the recovery of saturated signals in high-energy particle and nuclear physics experiments.This study also illustrates an innovative application of machine learning in the analysis of experimental data in particle physics.

A comparative study on the application of various artificial neural networks to simultaneous prediction of rock fragmentation and backbreak

In blasting operation,the aim is to achieve proper fragmentation and to avoid undesirable events such as backbreak.Therefore,predicting rock fragmentation and backbreak is very important to arrive at a technically and economically successful outcome.Since many parameters affect the blasting results in a complicated mechanism,employment of robust methods such as artificial neural network may be very useful.In this regard,this paper attends to simultaneous prediction of rock fragmentation and backbreak in the blasting operation of Tehran Cement Company limestone mines in Iran.Back propagation neural network（BPNN） and radial basis function neural network（RBFNN） are adopted for the simulation.Also,regression analysis is performed between independent and dependent variables.For the BPNN modeling,a network with architecture 6-10-2 is found to be optimum whereas for the RBFNN,architecture 636-2 with spread factor of 0.79 provides maximum prediction aptitude.Performance comparison of the developed models is fulfilled using value account for（VAF）,root mean square error（RMSE）,determination coefficient（R2） and maximum relative error（MRE）.As such,it is observed that the BPNN model is the most preferable model providing maximum accuracy and minimum error.Also,sensitivity analysis shows that inputs burden and stemming are the most effective parameters on the outputs fragmentation and backbreak,respectively.On the other hand,for both of the outputs,specific charge is the least effective parameter.

Calculation method of ship collision force on bridge using artificial neural network

Ship collision on bridge is a dynamic process featured by high nonlinearity and instantaneity. Calculating ship-bridge collision force typically involves either the use of design-specification-stipulated equivalent static load, or the use of finite element method (FEM) which is more time-consuming and requires supercomputing resources. In this paper, we proposed an alternative approach that combines FEM with artificial neural network (ANN). The radial basis function neural network (RBFNN) employed for calculating the impact force in consideration of ship-bridge collision mechanics. With ship velocity and mass as the input vectors and ship collision force as the output vector, the neural networks for different network parameters are trained by the learning samples obtained from finite element simulation results. The error analyses of the learning and testing samples show that the proposed RBFNN is accurate enough to calculate ship-bridge collision force. The input-output relationship obtained by the RBFNN is essentially consistent with the typical empirical formulae. Finally, a special toolbox is developed for calculation efficiency in application using MATLAB software.

Evaluation of Intensive Urban Land Use Based on an Artificial Neural Network Model:A Case Study of Nanjing City,China

In this paper, the artificial neural network(ANN) model was used to evaluate the degree of intensive urban land use in Nanjing City, China. The construction and application of the ANN model took into account the comprehensive, spatial and complex nature of urban land use. Through a preliminary calculation of the degree of intensive land use of the sample area, representative sample area selection and using the back propagation neural network model to train, the intensive land use level of each evaluation unit is finally determined in the study area. Results show that the method can effectively correct the errors caused by the limitations of the model itself and the determination of the ideal value and weights when the multifactor comprehensive evaluation is used alone. The ANN model can make the evaluation results more objective and practical. The evaluation results show a tendency of decreasing land use intensity from the core urban area to the periphery and the industrial functional area has relatively low land use intensity compared with other functional areas. Based on the evaluation results, some suggestions are put forward, such as transforming the mode of urban spatial expansion, strengthening the integration and potential exploitation of the land in the urban built-up area, and strengthening the control of the construction intensity of protected areas.

Application of FCM Algorithm Combined with Artificial Neural Network in TBM Operation Data

Fuzzy clustering theory is widely used in data mining of full-face tunnel boring machine.However,the traditional fuzzy clustering algorithm based on objective function is difficult to effectively cluster functional data.We propose a new Fuzzy clustering algorithm,namely FCM-ANN algorithm.The algorithm replaces the clustering prototype of the FCM algorithm with the predicted value of the artificial neural network.This makes the algorithm not only satisfy the clustering based on the traditional similarity criterion,but also can effectively cluster the functional data.In this paper,we first use the t-test as an evaluation index and apply the FCM-ANN algorithm to the synthetic datasets for validity testing.Then the algorithm is applied to TBM operation data and combined with the crossvalidation method to predict the tunneling speed.The predicted results are evaluated by RMSE and R^(2).According to the experimental results on the synthetic datasets,we obtain the relationship among the membership threshold,the number of samples,the number of attributes and the noise.Accordingly,the datasets can be effectively adjusted.Applying the FCM-ANN algorithm to the TBM operation data can accurately predict the tunneling speed.The FCM-ANN algorithm has improved the traditional fuzzy clustering algorithm,which can be used not only for the prediction of tunneling speed of TBM but also for clustering or prediction of other functional data.

Signal prediction based on empirical mode decomposition and artificial neural networks

In view of the usefulness of Empirical Mode Decomposition （EMD）, Artificial Neural Networks （ ANN）, and Most Relevant Matching Extension （MRME） methods in dealing with nonlinear signals, we pro- pose a new way of combining these methods to deal with signal prediction. We found the results of combining EMD with either ANN or MRME to have higher prediction precision for a time series than the result of using EMD alone.

Prediction of Salinity Variations in a Tidal Estuary Using Artificial Neural Network and Three-Dimensional Hydrodynamic Models

The simulation of salinity at different locations of a tidal river using physically-based hydrodynamic models is quite cumbersome because it requires many types of data, such as hydrological and hydraulic time series at boundaries, river geometry, and adjusted coefficients. Therefore, an artificial neural network (ANN) technique using a back-propagation neural network (BPNN) and a radial basis function neural network (RBFNN) is adopted as an effective alternative in salinity simulation studies. The present study focuses on comparing the performance of BPNN, RBFNN, and three-dimensional hydrodynamic models as applied to a tidal estuarine system. The observed salinity data sets collected from 18 to 22 May, 16 to 22 October, and 26 to 30 October 2002 (totaling 4320 data points) were used for BPNN and RBFNN model training and for hydrodynamic model calibration. The data sets collected from 30 May to 2 June and 11 to 15 November 2002 (totaling 2592 data points) were adopted for BPNN and RBFNN model verification and for hydrodynamic model verification. The results revealed that the ANN (BPNN and RBFNN) models were capable of predicting the nonlinear time series behavior of salinity to the multiple forcing signals of water stages at different stations and freshwater input at upstream boundaries. The salinity predicted by the ANN models was better than that predicted by the physically based hydrodynamic model. This study suggests that BPNN and RBFNN models are easy-to-use modeling tools for simulating the salinity variation in a tidal estuarine system.

Inventory Management and Demand Forecasting Improvement of a Forecasting Model Based on Artificial Neural Networks

Forecasting is predicting or estimating a future event or trend.Supply chains have been constantly growing in most countries ever since the industrial revolution of the 18th century.As the competitiveness between supply chains intensifies day by day,companies are shifting their focus to predictive analytics techniques to minimize costs and boost productivity and profits.Excessive inventory(overstock)and stock outs are very significant issues for suppliers.Excessive inventory levels can lead to loss of revenue because the company's capital is tied up in excess inventory.Excess inventory can also lead to increased storage,insurance costs and labor as well as lower and degraded quality based on the nature of the product.Shortages or out of stock can lead to lost sales and a decline in customer contentment and loyalty to the store.If clients are unable to find the right products on the shelves,they may switch to another vendor or purchase alternative items.Demand forecasting is valuable for planning,scheduling and improving the coordination of all supply chain activities.This paper discusses the use of neural networks for seasonal time series forecasting.Our objective is to evaluate the contribution of the correct choice of the transfer function by proposing a new form of the transfer function to improve the quality of the forecast.

Which return regime induces overconfidence behavior?Artificial intelligence and a nonlinear approach

Overconfidence behavior,one form of positive illusion,has drawn considerable attention throughout history because it is viewed as the main reason for many crises.Investors’overconfidence,which can be observed as overtrading following positive returns,may lead to inefficiencies in stock markets.To the best of our knowledge,this is the first study to examine the presence of investor overconfidence by employing an artificial intelligence technique and a nonlinear approach to impulse responses to analyze the impact of different return regimes on the overconfidence attitude.We examine whether investors in an emerging stock market(Borsa Istanbul)exhibit overconfidence behavior using a feed-forward,neural network,nonlinear Granger causality test and nonlinear impulseresponse functions based on local projections.These are the first applications in the relevant literature due to the novelty of these models in forecasting high-dimensional,multivariate time series.The results obtained from distinguishing between the different market regimes to analyze the responses of trading volume to return shocks contradict those in the literature,which is the key contribution of the study.The empirical findings imply that overconfidence behavior exhibits asymmetries in different return regimes and is persistent during the 20-day forecasting horizon.Overconfidence is more persistent in the low-than in the high-return regime.In the negative interest-rate period,a high-return regime induces overconfidence behavior,whereas in the positive interest-rate period,a low-return regime induces overconfidence behavior.Based on the empirical findings,investors should be aware that portfolio gains may result in losses depending on aggressive and excessive trading strategies,particularly in low-return regimes.

Estimation of vegetation biophysical parameters by remote sensing using radial basis function neural network

Hyperspectral reflectance (350~2500 nm) data were recorded at two different sites of rice in two experiment fields including two cultivars, and three levels of nitrogen (N) application. Twenty-five Vegetation Indices (VIs) were used to predict the rice agronomic parameters including Leaf Area Index (LAI, m2 green leaf/m2 soil) and Green Leaf Chlorophyll Density (GLCD, mg chlorophyll/m2 soil) by the traditional regression models and Radial Basis Function Neural Network (RBF). RBF emerged as a variant of Artificial Neural Networks (ANNs) in the late 1980’s. A large variety of training algorithms has been tested for training RBF networks. In this study, Original RBF (ORBF), Gradient Descent RBF (GDRBF), and Generalized Regression Neural Network (GRNN) were employed. Results showed that green waveband Normalized Difference Vegetation Index (NDVIgreen) and TCARI/OSAVI have the best prediction power for LAI by exponent model and ORBF respectively, and that TCARI/OSAVI has the best prediction power for GLCD by exponent model and GDRBF. The best performances of RBF are compared with the traditional models, showing that the relationship between VIs and agronomic variables are further improved when RBF is used. Compared with the best traditional models, ORBF using TCARI/OSAVI improves the prediction power for LAI by lowering the Root Mean Square Error (RMSE) for 0.1119, and GDRBF using TCARI/OSAVI improves the prediction power for GLCD by lowering the RMSE for 26.7853. It is concluded that RBF provides a useful exploratory and predictive tool when applied to the sensitive VIs.

Development of Trees Management System Using Radial Basis Function Neural Network for Rain Forecast

Agriculture and farming are mainly dependent on weather especially in Malaysia as it received heavy rainfall throughout the years. An efficient crop or tree management system with a weather forecast needed for suitable planning of farming operation. Radial Basis Function Neural Network (RBFNN) algorithm was used in this study to predict rainfall and the main focus of this study is to analyze the factor that affects the performance of neural model. This study found that the model works better the more hidden nodes and the optimum learning rate is 0.01 with the RMSE 49% and the percentage accuracy is 57%. Besides that, it is found that the meteorology data also affect the model performance. Future research can be conducted to improve the rainfall forecast of this study and improve the tree management system.

Pan evaporation modeling in different agroclimatic zones using functional link artificial neural network

Pan evaporation is an important climatic variable for developing efficient water resource management strategies.In the past,many machine learning models are reported in the literature for pan evaporation modeling using the different combinationof available climatic variables.In order to develop a novel model with improved accuracy and reduced computational complexity,the functional link artificial neural network(FLANN)is chosen as an architecture to estimate daily pan evaporation in three agro-climatic zones(ACZs)of Chhattisgarh state in east-central India.Single neuron and single layer in its structure make it less complex as compared to other multilayer neural networks and neuro-fuzzy based hybrid models.Estimation results obtained with the FLANN model are compared with those obtained by multi-layer artificial neural networks(MLANN)and two empirical methods using the same raw data and corresponding features.Statistical indices like root mean square error(RMSE),mean absolute error(MAE)and efficiency factor(EF)is also computed to evaluate the model performance.It is demonstrated that pan evaporation estimates obtained with the proposed FLANN models provide an improved estimation of pan evaporation(RMSE=0.85 to 1.27 mm d^(-1),MAE=0.63 to 0.95 mm d^(-1) and EF=0.70 to 0.89)as compared to MLANN(RMSE=0.94 to 1.58 mm d^(-1),MAE=0.73 to 1.14 mm d^(-1) and EF=0.62 to 0.88)and empirical(RMSE=1.19 to 2.19 mm d^(-1),MAE=0.91 to 1.62 mm d^(-1) and EF=0.49 to 0.88)models in different ACZs.

Multilayer perceptron and Chebyshev polynomials-based functional link artificial neural network for solving differential equations

This paper discusses the issues of computational efforts and the accuracy of solutions of differential equations using multilayer perceptron and Chebyshev polynomials-based functional link artificial neural networks.Some ordinary and partial differential equations have been solved by both these techniques and pros and cons of both these type of feedforward networks have been discussed in detail.Apart from that,various factors that affect the accuracy of the solution have also been analyzed.

Numeral eddy current sensor modelling based on genetic neural network

This paper presents a method used to the numeral eddy current sensor modelling based on the genetic neural network to settle its nonlinear problem. The principle and algorithms of genetic neural network are introduced. In this method, the nonlinear model parameters of the numeral eddy current sensor are optimized by genetic neural network （GNN） according to measurement data. So the method remains both the global searching ability of genetic algorithm and the good local searching ability of neural network. The nonlinear model has the advantages of strong robustness, on-line modelling and high precision. The maximum nonlinearity error can be reduced to 0.037% by using GNN. However, the maximum nonlinearity error is 0.075% using the least square method.

Simulation and prediction of monthly accumulated runoff,based on several neural network models under poor data availability

Most previous research on areas with abundant rainfall shows that simulations using rainfall-runoff modes have a very high prediction accuracy and applicability when using a back-propagation(BP), feed-forward, multilayer perceptron artificial neural network(ANN). However, in runoff areas with relatively low rainfall or a dry climate, more studies are needed. In these areas—of which oasis-plain areas are a particularly good example—the existence and development of runoff depends largely on that which is generated from alpine regions. Quantitative analysis of the uncertainty of runoff simulation under climate change is the key to improving the utilization and management of water resources in arid areas. Therefore, in this context, three kinds of BP feed-forward, three-layer ANNs with similar structure were chosen as models in this paper.Taking the oasis–plain region traverse by the Qira River Basin in Xinjiang, China, as the research area, the monthly accumulated runoff of the Qira River in the next month was simulated and predicted. The results showed that the training precision of a compact wavelet neural network is low; but from the forecasting results, it could be concluded that the training algorithm can better reflect the whole law of samples. The traditional artificial neural network(TANN) model and radial basis-function neural network(RBFNN) model showed higher accuracy in the training and prediction stage. However, the TANN model, more sensitive to the selection of input variables, requires a large number of numerical simulations to determine the appropriate input variables and the number of hidden-layer neurons. Hence, The RBFNN model is more suitable for the study of such problems. And it can be extended to other similar research arid-oasis areas on the southern edge of the Kunlun Mountains and provides a reference for sustainable water-resource management of arid-oasis areas.