A Kind of Second-Order Learning Algorithm Based on Generalized Cost Criteria in Multi-Layer Feed-Forward Neural Networks

A kind of second order algorithm--recursive approximate Newton algorithm was given by Karayiannis. The algorithm was simplified when it was formulated. Especially, the simplification to matrix Hessian was very reluctant, which led to the loss of valuable information and affected performance of the algorithm to certain extent. For multi layer feed forward neural networks, the second order back propagation recursive algorithm based generalized cost criteria was proposed. It is proved that it is equivalent to Newton recursive algorithm and has a second order convergent rate. The performance and application prospect are analyzed. Lots of simulation experiments indicate that the calculation of the new algorithm is almost equivalent to the recursive least square multiple algorithm. The algorithm and selection of networks parameters are significant and the performance is more excellent than BP algorithm and the second order learning algorithm that was given by Karayiannis.

Near-infrared Spectral Detection of the Content of Soybean Fat Acids Based on Genetic Multilayer Feed forward Neural Network

In the paper, a method of building mathematic model employing genetic multilayer feed forward neural network is presented, and the quantitative relationship of chemical measured values and near-infrared spectral data is established. In the paper, quantitative mathematic model related chemical assayed values and near-infrared spectral data is established by means of genetic multilayer feed forward neural network, acquired near-infrared spectral data are taken as input of network with the content of five kinds of fat acids tested from chemical method as output, weight values of multilayer feed forward neural network are trained by genetic algorithms and detection model of neural network of soybean is built. A kind of multilayer feed forward neural network trained by genetic algorithms is designed in the paper. Through experiments, all the related coefficients of five fat acids can approach 0.9 which satisfies the preliminary test of soybean breeding.

Feed-Forward Neural Network Based Petroleum Wells Equipment Failure Prediction

In the oil industry, the productivity of oil wells depends on the performance of the sub-surface equipment system. These systems often have problems stemming from sand, corrosion, internal pressure variation, or other factors. In order to ensure high equipment performance and avoid high-cost losses, it is essential to identify the source of possible failures in the early stage. However, this requires additional maintenance fees and human power. Moreover, the losses caused by these problems may lead to interruptions in the whole production process. In order to minimize maintenance costs, in this paper, we introduce a model for predicting equipment failure based on processing the historical data collected from multiple sensors. The state of the system is predicted by a Feed-Forward Neural Network (FFNN) with an SGD and Backpropagation algorithm is applied in the training process. Our model’s primary goal is to identify potential malfunctions at an early stage to ensure the production process’ continued high performance. We also evaluated the effectiveness of our model against other solutions currently available in the industry. The results of our study show that the FFNN can attain an accuracy score of 97% on the given dataset, which exceeds the performance of the models provided.

Combined Signal Processing Based Techniques and Feed Forward Neural Networks for Pathological Voice Detection and Classification

This paper presents the pathological voice detection and classification techniques using signal processing based methodologies and Feed Forward Neural Networks(FFNN).The important pathological voices such as Autism Spectrum Disorder(ASD)and Down Syndrome(DS)are considered for analysis.These pathological voices are known to manifest in different ways in the speech of children and adults.Therefore,it is possible to discriminate ASD and DS children from normal ones using the acoustic features extracted from the speech of these subjects.The important attributes hidden in the pathological voices are extracted by applying different signal processing techniques.In this work,three group of feature vectors such as perturbation measures,noise parameters and spectral-cepstral modeling are derived from the signals.The detection and classification is done by means of Feed For-ward Neural Network(FFNN)classifier trained with Scaled Conjugate Gradient(SCG)algorithm.The performance of the network is evaluated by finding various performance metrics and the the experimental results clearly demonstrate that the proposed method gives better performance compared with other methods discussed in the literature.

Hausdorff Dimension of Multi-Layer Neural Networks

This elucidation investigates the Hausdorff dimension of the output space of multi-layer neural networks. When the factor map from the covering space of the output space to the output space has a synchronizing word, the Hausdorff dimension of the output space relates to its topological entropy. This clarifies the geometrical structure of the output space in more details.

Grid Side Distributed Energy Storage Cloud Group End Region Hierarchical Time-Sharing Configuration Algorithm Based onMulti-Scale and Multi Feature Convolution Neural Network

There is instability in the distributed energy storage cloud group end region on the power grid side.In order to avoid large-scale fluctuating charging and discharging in the power grid environment and make the capacitor components showa continuous and stable charging and discharging state,a hierarchical time-sharing configuration algorithm of distributed energy storage cloud group end region on the power grid side based on multi-scale and multi feature convolution neural network is proposed.Firstly,a voltage stability analysis model based onmulti-scale and multi feature convolution neural network is constructed,and the multi-scale and multi feature convolution neural network is optimized based on Self-OrganizingMaps(SOM)algorithm to analyze the voltage stability of the cloud group end region of distributed energy storage on the grid side under the framework of credibility.According to the optimal scheduling objectives and network size,the distributed robust optimal configuration control model is solved under the framework of coordinated optimal scheduling at multiple time scales;Finally,the time series characteristics of regional power grid load and distributed generation are analyzed.According to the regional hierarchical time-sharing configuration model of“cloud”,“group”and“end”layer,the grid side distributed energy storage cloud group end regional hierarchical time-sharing configuration algorithm is realized.The experimental results show that after applying this algorithm,the best grid side distributed energy storage configuration scheme can be determined,and the stability of grid side distributed energy storage cloud group end region layered timesharing configuration can be improved.

Feed-Forward Artificial Neural Network Model for Air Pollutant Index Prediction in the Southern Region of Peninsular Malaysia

This paper describes the application of principal component analysis (PCA) and artificial neural network (ANN) to predict the air pollutant index (API) within the seven selected Malaysian air monitoring stations in the southern region of Peninsular Malaysia based on seven years database (2005-2011). Feed-forward ANN was used as a prediction method. The feed-forward ANN analysis demonstrated that the rotated principal component scores (RPCs) were the best input parameters to predict API. From the 4 RPCs, only 10 (CO, O3, PM10, NO2, CH4, NmHC, THC, wind direction, humidity and ambient temp) out of 12 prediction variables were the most significant parameters to predict API. The results proved that the ANN method can be applied successfully as tools for decision making and problem solving for better atmospheric management.

Performance Comparison of Neural Networks for HRTFs Approximation

0 ＩｎｔｒｏｄｕｃｔｉｏｎＨｅａｄｒｅｌａｔｅｄｔｒａｎｓｆｅｒｆｕｎｃｔｉｏｎｓ(ＨＲＴＦｓ)ｒｅｆｅｒｔｏｔｈｅｓｐｅｃｔｒａｌｆｉｌｔｅｒｉｎｇｆｒｏｍｓｏｕｎｄｓｏｕｒｃｅｓｔｏｌｉｓｔｅｎｅｒｓ’ｅａｒｄｒｕｍｓ.ＳｉｎｃｅＨＲＴＦｓ(ｎｏｎ?..

Using Neural Networks to Predict Secondary Structure for Protein Folding

Protein Secondary Structure Prediction (PSSP) is considered as one of the major challenging tasks in bioinformatics, so many solutions have been proposed to solve that problem via trying to achieve more accurate prediction results. The goal of this paper is to develop and implement an intelligent based system to predict secondary structure of a protein from its primary amino acid sequence by using five models of Neural Network (NN). These models are Feed Forward Neural Network (FNN), Learning Vector Quantization (LVQ), Probabilistic Neural Network (PNN), Convolutional Neural Network (CNN), and CNN Fine Tuning for PSSP. To evaluate our approaches two datasets have been used. The first one contains 114 protein samples, and the second one contains 1845 protein samples.

Identification and Prediction of Internet Traffic Using Artificial Neural Networks

This paper presents the development of an artificial neural network (ANN) model based on the multi-layer perceptron (MLP) for analyzing internet traffic data over IP networks. We applied the ANN to analyze a time series of measured data for network response evaluation. For this reason, we used the input and output data of an internet traffic over IP networks to identify the ANN model, and we studied the performance of some training algorithms used to estimate the weights of the neuron. The comparison between some training algorithms demonstrates the efficiency and the accu-racy of the Levenberg-Marquardt (LM) and the Resilient back propagation (Rp) algorithms in term of statistical crite-ria. Consequently, the obtained results show that the developed models, using the LM and the Rp algorithms, can successfully be used for analyzing internet traffic over IP networks, and can be applied as an excellent and fundamental tool for the management of the internet traffic at different times.

An Improved SPSA Algorithm for System Identification Using Fuzzy Rules for Training Neural Networks

Simultaneous perturbation stochastic approximation (SPSA) belongs to the class of gradient-free optimization methods that extract gradient information from successive objective function evaluation. This paper describes an improved SPSA algorithm, which entails fuzzy adaptive gain sequences, gradient smoothing, and a step rejection procedure to enhance convergence and stability. The proposed fuzzy adaptive simultaneous perturbation approximation (FASPA) algorithm is particularly well suited to problems involving a large number of parameters such as those encountered in nonlinear system identification using neural networks (NNs). Accordingly, a multilayer perceptron (MLP) network with popular training algorithms was used to predicate the system response. We found that an MLP trained by FASPSA had the desired accuracy that was comparable to results obtained by traditional system identification algorithms. Simulation results for typical nonlinear systems demonstrate that the proposed NN architecture trained with FASPSA yields improved system identification as measured by reduced time of convergence and a smaller identification error.

Using Feed Forward BPNN for Forecasting All Share Price Index

Use of artificial neural networks has become a significant and an emerging research method due to its capability of capturing nonlinear behavior instead of conventional time series methods. Among them, feed forward back propagation neural network (BPNN) is the widely used network topology for forecasting stock prices indices. In this study, we attempted to find the best network topology for one step ahead forecasting of All Share Price Index (ASPI), Colombo Stock Exchange (CSE) by employing feed forward BPNN. The daily data including ASPI, All Share Total Return Index (ASTRI), Market Price Earnings Ratio (PER), and Market Price to Book Value (PBV) were collected from CSE over the period from January 2nd 2012 to March 20th 2014. The experiment is implemented by prioritizing the number of inputs, learning rate, number of hidden layer neurons, and the number of training sessions. Eight models were selected on basis of input data and the number of training sessions. Then the best model was used for forecasting next trading day ASPI value. Empirical result reveals that the proposed model can be used as an approximation method to obtain next day value. In addition, it showed that the number of inputs, number of hidden layer neurons and the training times are significant factors that can be affected to the accuracy of forecast value.

A Condition States Assessment System for Concrete Bridges Using Neural Networks

Due to continuing aging and heavy utilization of many bridges and the limited available funds, the importance of proper bridge condition state assessment has risen recently, which is the crucial point for rational decision-making on MR&R activities. This paper presents a prototype of the concrete bridge condition state assessment system (CBCSAS) with the following sub-modules: inspection, parameter recognition, structural assessment, main cause identification and priority-to-action. And multi-layer neural networks, which may combine with fuzzy set theory or not, are performed to realize the structural assessment with embedding expert knowledge into the assessment system.

基于多层全连接神经网络的6C地震波极化向量识别研究

利用机器学习原理,提出一种基于多层全连接(multi-layer fully connected, MFC)神经网络的六分量(six-component, 6C)地震波极化向量识别方法。首先利用6C地震波各波型极化向量数学模型和一系列仿真参数得到5种波型和噪声波型各5 000个极化向量数据集,然后随机选取其中5 000个作为测试集,其余划分为训练集,进行MFC神经网络与支持向量机(support vector machine, SVM)的综合辨识性能对比实验。结果表明,MFC神经网络模型识别5种极化向量类型(SH波和Love波视为一类)和6种极化向量类型的效果均显著优于SVM模型,平均识别率分别达到99.786%和87.940%。

SSA-MLP模型在岩质边坡稳定性预测中的应用

岩质边坡的力学参数量化及稳定性分析对岩质边坡灾害的防治具有重要意义。Hoek-Brown(H B)准则是一种用于确定岩体力学参数的经典方法,能反映出边坡岩体变形和位移的非线性破坏特征。在此基础上,首先,提出一种麻雀搜索算法(Sparrow Search Algorithm,SSA)改进多层感知器(Multi-Layer Perceptron,MLP)的神经网络模型,并用于边坡稳定性预测、指标敏感性分析及参数反演。其次,将收集的1085组岩质边坡的几何参数和H B准则参数等作为输入变量,极限平衡理论Bishop法求解的安全系数作为输出变量,对SSA MLP模型进行训练学习和性能评估。最后,将该模型运用于25个边坡实例,验证模型的有效性。结果显示,该模型收敛速度快、精度高,为边坡稳定性分析和参数量化提供了一种新思路。

基于概率建模的分层产液劈分方法

传统产液劈分方法无法考虑层间干扰及注水井和邻井的影响,难以准确判断井下实际状况。同时,海上油田产液剖面测试成本高,常规的机器学习方法面临样本数量少的问题。基于此,提出一种基于贝叶斯神经网络和极限梯度提升算法的多层合采产液劈分混合学习模型。概率方法可以识别预测中的不确定性,通过将神经网络与概率建模结合,进行分层产液数据分布特征挖掘,结合主控因素分析,混合学习算法可以实现小层产液量的准确预测,可以依据较少的数据获得更为稳健的模型。为验证所提方法的有效性,将其应用于实际油田某区块进行产液剖面预测。结果表明:相比KH劈分方法在计算中劈分系数固定,不会随着生产过程波动,所提出的方法可从历史数据中学习,预测精度达到87.9%,预测结果更加逼近真实单层产液量。

集成全尺度融合和循环注意力的医学图像分割网络

深度学习中的编解码网络在图像特征提取和分层特征融合方面具有卓越的性能,常被用于医学图像分割。但是,目前主流的编解码网络分割方法仍面临编码和解码阶段单一网络挖掘的图像特征信息不足,以及仅使用简单的跳跃连接而无法充分利用全尺度特征包含的粗粒度信息和细粒度信息等问题。为了解决上述问题,提出了一种集成全尺度融合和循环注意力的医学图像分割网络。首先,在U-Net编码器中加入了结合多层感知机(MLP)的卷积MLP模块来提取图像的全局特征信息,用于扩大编码器的特征感受野。其次,通过全尺度特征融合模块使得各尺度跳跃连接特征进行粗粒度信息和细粒度信息的有效融合,减小各尺度跳跃连接特征间的语义差异,突出图像的关键特征信息。最后,解码器通过提出的结合循环神经网络(RNN)和注意力机制的循环注意力解码模块(RADU)来逐级精细化图像特征信息,加强特征提取的同时避免信息冗余,并得到高精度分割结果。在4个数据集上将所提方法与主流较优的方法进行比较,所提方法在像素精度和骰子相似系数两个指标上的图像分割精度均有提高。因此,所提出的用于医学图像分割的编解码网络利用全尺度特征融合模块和循环注意力解码模块,能够获得较优异的高精度分割结果,并且模型具有良好的噪声鲁棒性和抗干扰能力。

集改进图卷积和多层池化的点云分类模型

针对基于图卷积的点云分类模型在提取点云不同语义区域的特征信息以及高效利用聚合的高维特征方面存在的问题,本文提出了一种新的点云分类模型,该模型采用了动态自适应图卷积和多层池化相结合的方法。具体而言,本文采用了残差结构来构建更深层的卷积,以学习不同语义区域点对特征中不同层次的特征信息,从而生成动态自适应调整卷积核,针对不同的点对动态更新边的特征关系,从而提取更为精确的局部特征。同时,本文将聚合的高维特征输入到多层最大池化模块中,回收利用第一次最大池化后丢弃的特征信息进行多层最大池化,从而获取更为丰富的高维特征,提高分类模型的精度。实验结果表明,在ModelNet40数据集上,本文提出的分类模型的总体精度达到93.3%,平均精度为90.7%,明显优于目前主流的点云分类模型,并具有较强的鲁棒性。

声波移动障碍反散射问题研究

研究一种具有Dirichlet边界条件的时域声波移动障碍反散射问题。首先对时域波动方程进行求解,在有限观测时间内对移动障碍物采集动态声波近场数据;其次建立一个时域声波移动障碍反散射问题的神经网络模型,该模型将时域近场数据作为输入序列,已知形状的移动障碍物位置参数作为输出序列,求解该反散射问题;最后对于不同移动速度的情况,有效地反演出移动障碍物的位置和轨迹。数值实验表明了该方法的有效性。

多输入傅里叶神经网络及其麻雀搜索优化

鉴于反向传播(BP)神经网络存在灵敏度高但收敛速度慢,以及已有傅里叶神经网络不具备多输入数据特征提取能力,借助多个傅里叶神经网络构建能接收多维数据的堆叠神经网络,进而将其与多层感知器融合,获得基于梯度下降的多输入傅里叶神经网络。结合此神经网络获取全局最优参数值难的因素,通过在麻雀搜索算法中引入Cat混沌映射、动态种群规模调节机制及参数自适应调节方案,提出改进型麻雀搜索算法,并将其应用于多输入傅里叶神经网络的参数优化及高维函数优化问题的求解。理论分析可得,所提算法的计算复杂度主要由种群规模和优化问题的维度决定。比较性的数值实验表明,所获神经网络提取多源数据特征的能力和泛化能力强,同时所提算法处理高维优化问题具有明显优势且收敛速度快。