Generalization Capabilities of Feedforward Neural Networks for Pattern Recognition

This paper studies the generalization capability of feedforward neural networks (FNN).The mechanism of FNNs for classification is investigated from the geometric and probabilistic viewpoints. It is pointed out that the outputs of the output layer in the FNNs for classification correspond to the estimates of posteriori probability of the input pattern samples with desired outputs 1 or 0. The theorem for the generalized kernel function in the radial basis function networks (RBFN) is given. For an 2-layer perceptron network (2-LPN). an idea of using extended samples to improve generalization capability is proposed. Finally. the experimental results of radar target classification are given to verify the generaliztion capability of the RBFNs.

A Modified Algorithm for Feedforward Neural Networks

As a most popular learning algorithm for the feedforward neural networks, the classic BP algorithm has its many shortages. To overcome some of the shortages, a modified learning algorithm is proposed in the article. And the simulation result illustrate the modified algorithm is more effective and practicable.

IDENTIFICATION OF NONLINEAR TIME VARYING SYSTEM USING FEEDFORWARD NEURAL NETWORKS

As it is well known,it is difficult to identify a nonlinear time varying system using traditional identification approaches,especially under unknown nonlinear function.Neural networks have recently emerged as a successful tool in the area of identification and control of time invariant nonlinear systems.However,it is still difficult to apply them to complicated time varying system identification.In this paper we present a learning algorithm for identification of the nonlinear time varying system using feedforward neural networks.The main idea of this approach is that we regard the weights of the network as a state of a time varying system,then use a Kalman filter to estimate the state.Thus the network implements nonlinear and time varying mapping.We derived both the global and local learning algorithms.Simulation results demonstrate the effectiveness of this approach.

Fully Connected Feedforward Neural Networks Based CSI Feedback Algorithm

In modern wireless communication systems,the accurate acquisition of channel state information(CSI)is critical to the performance of beamforming,non-orthogonal multiple access(NOMA),etc.However,with the application of massive MIMO in 5G,the number of antennas increases by hundreds or even thousands times,which leads to excessive feedback overhead and poses a huge challenge to the conventional channel state information feedback scheme.In this paper,by using deep learning technology,we develop a system framework for CSI feedback based on fully connected feedforward neural networks(FCFNN),named CF-FCFNN.Through learning the training set composed of CSI,CF-FCFNN is able to recover the original CSI from the compressed CSI more accurately compared with the existing method based on deep learning without increasing the algorithm complexity.

Improving the spaceborne GNSS-R altimetric precision based on the novel multilayer feedforward neural network weighted joint prediction model

Global navigation satellite system-reflection(GNSS-R)sea surface altimetry based on satellite constellation platforms has become a new research direction and inevitable trend,which can meet the altimetric precision at the global scale required for underwater navigation.At present,there are still research gaps for GNSS-R altimetry under this mode,and its altimetric capability cannot be specifically assessed.Therefore,GNSS-R satellite constellations that meet the global altimetry needs to be designed.Meanwhile,the matching precision prediction model needs to be established to quantitatively predict the GNSS-R constellation altimetric capability.Firstly,the GNSS-R constellations altimetric precision under different configuration parameters is calculated,and the mechanism of the influence of orbital altitude,orbital inclination,number of satellites and simulation period on the precision is analyzed,and a new multilayer feedforward neural network weighted joint prediction model is established.Secondly,the fit of the prediction model is verified and the performance capability of the model is tested by calculating the R2 value of the model as 0.9972 and the root mean square error(RMSE)as 0.0022,which indicates that the prediction capability of the model is excellent.Finally,using the novel multilayer feedforward neural network weighted joint prediction model,and considering the research results and realistic costs,it is proposed that when the constellation is set to an orbital altitude of 500 km,orbital inclination of 75and the number of satellites is 6,the altimetry precision can reach 0.0732 m within one year simulation period,which can meet the requirements of underwater navigation precision,and thus can provide a reference basis for subsequent research on spaceborne GNSS-R sea surface altimetry.

Convergence of On-Line Gradient Methods for Two-Layer Feedforward Neural Networks

A discussion is given on the convergence of the on-line gradient methods for two-layer feedforward neural networks in general cases. The theories are applied to some usual activation functions and energy functions.

L^2(R^d) Approximation Capability of Incremental Constructive Feedforward Neural Networks with Random Hidden Units

This paper studies approximation capability to L^2（Rd） functions of incremental constructive feedforward neural networks （FNN） with random hidden units. Two kinds of therelayered feedforward neural networks are considered： radial basis function （RBF） neural networks and translation and dilation invariant （TDI） neural networks. In comparison with conventional methods that existence approach is mainly used in approximation theories for neural networks, we follow a constructive approach to prove that one may simply randomly choose parameters of hidden units and then adjust the weights between the hidden units and the output unit to make the neural network approximate any function in L2 （Rd） to any accuracy. Our result shows given any non-zero activation function g ： R＋ → R and g（｜｜x｜｜R^d） ∈ L^2（Rd） for RBF hidden units, or any non-zero activation function g（x） ∈ L^2（R^d） for TDI hidden units, the incremental network function fn with randomly generated hidden units converges to any target function in L2 （R^d） with probability one as the number of hidden units n → ∞, if one only properly adjusts the weights between the hidden units and output unit.

Influence of the Gaussian colored noise and electromagnetic radiation on the propagation of subthreshold signals in feedforward neural networks

Iterative methods are used to simulate the in vitro feedforward neural networks in physiological experiments.Emissivity can be propagated to a minimum of ten groups.However,the discharge activity of each group will be more synchronized.The feedforward neural networks have a wide range of applications in machine learning,and the weight of synapses considerably influences the propagation of weak signals.Herein,we investigated the effect of Gaussian colored noise and electromagnetic radiation on the propagation of the subthreshold excitatory postsynaptic current signals in the input layer of the multilayer Izhikevich neural feedforward networks.In the absence of electromagnetic radiation,the excitatory postsynaptic current signal is stably propagated and amplified in multilayer feedforward neural networks under the optimal Gaussian colored noise strength or correlation time in the output layer of the network.Compared with the case in which there is no electromagnetic radiation,the presence of electromagnetic radiation slightly reduces the propagation of weak signals.Further,the time required to propagate the excitatory postsynaptic current signal to the output layer increases with the increasing feedback gain.The feedforward neural network considered in this study is a considerably simple model.More complex structures,such as backward connection and delayed feedback,can be observed in real biological systems.Hence,the next step will be to study more complex neural models with neuron models based on the physiological experimental data and compare them with real biological systems.Furthermore,the study of neural networks can be combined with an experimental study about the auditory nervous system of bats to understand the biological mechanism associated with the auditory system function of bats from two perspectives.

A Novel Evolutionary Feedforward Neural Network with Artificial Immunology

A hybrid algorithm to design the multi layer feedforward neural network was proposed. Evolutionary programming is used to design the network that makes the training process tending to global optima. Artificial immunology combined with simulated annealing algorithm is used to specify the initial weight vectors, therefore improves the probabiligy of training algorithm to converge to global optima. The applications of the neural network in the modulation style recognition of analog modulated rader signals demonstrate the good performance of the network.

Development of a Novel Feedforward Neural Network Model Based on Controllable Parameters for Predicting Effluent Total Nitrogen

The problem of effluent total nitrogen(TN)at most of the wastewater treatment plants(WWTPs)in China is important for meeting the related water quality standards,even under the condition of high energy consumption.To achieve better prediction and control of effluent TN concentration,an efficient prediction model,based on controllable operation parameters,was constructed in a sequencing batch reactor process.Compared with previous models,this model has two main characteristics:①Superficial gas velocity and anoxic time are controllable operation parameters and are selected as the main input parameters instead of dissolved oxygen to improve the model controllability,and②the model prediction accuracy is improved on the basis of a feedforward neural network(FFNN)with algorithm optimization.The results demonstrated that the FFNN model was efficiently optimized by scaled conjugate gradient,and the performance was excellent compared with other models in terms of the correlation coefficient(R).The optimized FFNN model could provide an accurate prediction of effluent TN based on influent water parameters and key control parameters.This study revealed the possible application of the optimized FFNN model for the efficient removal of pollutants and lower energy consumption at most of the WWTPs.

Online Gradient Methods with a Punishing Term for Neural Networks

Online gradient methods are widely used for training the weight of neural networks and for other engineering computations. In certain cases, the resulting weight may become very large, causing difficulties in the implementation of the network by electronic circuits. In this paper we introduce a punishing term into the error function of the training procedure to prevent this situation. The corresponding convergence of the iterative training procedure and the boundedness of the weight sequence are proved. A supporting numerical example is also provided.

Relations Between Wavelet Network and Feedforward Neural Network

A comparison of construction forms and base functions is made between feedforward neural network and wavelet network. The relations between them are studied from the constructions of wavelet functions or dilation functions in wavelet network by different activation functions in feedforward neural network. It is concluded that some wavelet function is equal to the linear combination of several neurons in feedforward neural network.

Feedforward Neural Network for joint inversion of geophysical data to identify geothermal sweet spots in Gandhar,Gujarat,India

Artificial Neural Networks(ANNs)are used in numerous engineering and scientific disciplines as an automated approach to resolve a number of problems.However,to build an artificial neural network that is prudent enough to rely on,vast quantities of relevant data have to be fed.In this study,we analysed the scope of artificial neural networks in geothermal reservoir architecture.In particular,we attempted to solve joint inversion problem through Feedforward Neural Network(FNN)technique.In order to identify geothermal sweet spots in the subsurface,an extensive geophysical studies were conducted in Gandhar area of Gujarat,India.The data were acquired along six profile lines for gravity,magnetics and magnetotellurics.Initially low velocity zone was identified using refraction seismic technique in order to set a common datum level for other potential data.The depth of low velocity zone in Gandhar was identified at 11 m.The FNN backpropagation method was applied to gain the global minima of the data space and model space as desired.The input dataset fed to the inversion algorithm in the form of gravity,magnetic susceptibility and resistivity helped to predict the suitable model after network training in multiple steps.The joint inversion of data is conducive to understanding the subsurface geological and lithological features along with probable geothermal sweet spots.The results of this study show the geothermal sweet spots at depth ranging from 200 m to 300 m.The results from our study can be used for targeted zones for geothermal water exploitation.

Multi-component quantitative and feed-forward neural network for pattern classification of raw and wine-processed Corni Fructus

Background:To promote the quality evaluation,clarify the processing mechanism and distinguish origins of Corni Fructus(cornus)from different regions.Methods:This study developed a high performance liquid chromatography method for simultaneous determination of 5-hydroxymethylfurfural,2 phenolic acids and 4 iridoid glycosides and the reference fingerprint of cornus from different regions.In addition,the feedforward neural network model provided a pattern classification of sample regions.Results:The content of morroniside and loganin were the highest in all raw cornus samples ranging from 9.45μg/mg to 16.3μg/mg and 6.64μg/mg to 13.7μg/mg,respectively.The level of sweroside in raw cornus from Henan(0.83μg/mg^(-1).39μg/mg)and Zhejiang(0.64μg/mg^(-1).17μg/mg)were greater than other origins.After wine-processing,the glucose or fructose were dehydrated to increase the levels of 5-hydroxymethylfurfural.The C-4 position of-COOCH3 of hot-sensitive iridoid glycosides was hydrolyzed to generate-COOH as stable components.Polyphenol derivatives may be degraded to increase the content of phenolic acid.Subsequently,an excellent feedforward neural network model for identification of raw cornus and wine-prepared cornus was established which could distinguish the sample origins.Conclusion:This work provided a trustworthy method to evaluate the quality and distinguish the sources of cornus.Meanwhile,the clear processing mechanism provided a scientific foundation for controlling the cornus quality during wine-processing.

Neural Network inverse Adaptive Controller Based on Davidon Least Square

General neural network inverse adaptive controller has two flaws: the first is the slow convergence speed; the second is the invalidation to the non-minimum phase system. These defects limit the scope in which the neural network inverse adaptive controller is used. We employ Davidon least squares in training the multi-layer feedforward neural network used in approximating the inverse model of plant to expedite the convergence, and then through constructing the pseudo-plant, a neural network inverse adaptive controller is put forward which is still effective to the nonlinear non-minimum phase system. The simulation results show the validity of this scheme.

Prediction of nuclear charge density distribution with feedback neural network

Nuclear charge density distribution plays an important role in both nuclear and atomic physics,for which the two-parameter Fermi(2pF)model has been widely applied as one of the most frequently used models.Currently,the feedforward neural network has been employed to study the available 2pF model parameters for 86 nuclei,and the accuracy and precision of the parameter-learning effect are improved by introducing A^(1∕3)into the input parameter of the neural network.Furthermore,the average result of multiple predictions is more reliable than the best result of a single prediction and there is no significant difference between the average result of the density and parameter values for the average charge density distribution.In addition,the 2pF parameters of 284(near)stable nuclei are predicted in this study,which provides a reference for the experiment.

Aeromagnetic Compensation Algorithm Based on Levenberg-Marquard Neural Network

The magnetic compensation of aeromagnetic survey is an important calibration work,which has a great impact on the accuracy of measurement.In an aeromagnetic survey flight,measurement data consists of diurnal variation,aircraft maneuver interference field,and geomagnetic field.In this paper,appropriate physical features and the modular feedforward neural network(MFNN)with Levenberg-Marquard(LM)back propagation algorithm are adopted to supervised learn fluctuation of measuring signals and separate the interference magnetic field from the measurement data.LM algorithm is a kind of least square estimation algorithm of nonlinear parameters.It iteratively calculates the jacobian matrix of error performance and the adjustment value of gradient with the regularization method.LM algorithm’s computing efficiency is high and fitting error is very low.The fitting performance and the compensation accuracy of LM-MFNN algorithm are proved to be much better than those of TOLLES-LAWSON(T-L)model with the linear least square(LS)solution by fitting experiments with five different aeromagnetic surveys’data.

Neural Network for Estimating Daily Global Solar Radiation Using Temperature, Humidity and Pressure as Unique Climatic Input Variables

Solar radiation is one of the most important parameters for applications, development and research related to renewable energy. However, solar radiation measurements are not a simple task for several reasons. In the cases where data are not available, it is very common the use of computational models to estimate the missing data, which are based mainly on the search for relationships between weather variables, such as temperature, humidity, precipitation, cloudiness, sunshine hours, etc. But, many of these are subjective and difficult to measure, and thus they are not always available. In this paper, we propose a method for estimating daily global solar radiation, combining empirical models and artificial neural networks. The model uses temperature, relative humidity and atmospheric pressure as the only climatic input variables. Also, this method is compared with linear regression to verify that the data have nonlinear components. The models are adjusted and validated using data from five meteorological stations in the province of Tucumán, Argentina. Results show that neural networks have better accuracy than empirical models and linear regression, obtaining on average, an error of 2.83 [MJ/m2] in the validation dataset.

Which return regime induces overconfidence behavior?Artificial intelligence and a nonlinear approach

Overconfidence behavior,one form of positive illusion,has drawn considerable attention throughout history because it is viewed as the main reason for many crises.Investors’overconfidence,which can be observed as overtrading following positive returns,may lead to inefficiencies in stock markets.To the best of our knowledge,this is the first study to examine the presence of investor overconfidence by employing an artificial intelligence technique and a nonlinear approach to impulse responses to analyze the impact of different return regimes on the overconfidence attitude.We examine whether investors in an emerging stock market(Borsa Istanbul)exhibit overconfidence behavior using a feed-forward,neural network,nonlinear Granger causality test and nonlinear impulseresponse functions based on local projections.These are the first applications in the relevant literature due to the novelty of these models in forecasting high-dimensional,multivariate time series.The results obtained from distinguishing between the different market regimes to analyze the responses of trading volume to return shocks contradict those in the literature,which is the key contribution of the study.The empirical findings imply that overconfidence behavior exhibits asymmetries in different return regimes and is persistent during the 20-day forecasting horizon.Overconfidence is more persistent in the low-than in the high-return regime.In the negative interest-rate period,a high-return regime induces overconfidence behavior,whereas in the positive interest-rate period,a low-return regime induces overconfidence behavior.Based on the empirical findings,investors should be aware that portfolio gains may result in losses depending on aggressive and excessive trading strategies,particularly in low-return regimes.

High-accuracy target tracking for multistatic passive radar based on a deep feedforward neural network

In radar systems,target tracking errors are mainly from motion models and nonlinear measurements.When we evaluate a tracking algorithm,its tracking accuracy is the main criterion.To improve the tracking accuracy,in this paper we formulate the tracking problem into a regression model from measurements to target states.A tracking algorithm based on a modified deep feedforward neural network(MDFNN)is then proposed.In MDFNN,a filter layer is introduced to describe the temporal sequence relationship of the input measurement sequence,and the optimal measurement sequence size is analyzed.Simulations and field experimental data of the passive radar show that the accuracy of the proposed algorithm is better than those of extended Kalman filter(EKF),unscented Kalman filter(UKF),and recurrent neural network(RNN)based tracking methods under the considered scenarios.