An Improved SPSA Algorithm for System Identification Using Fuzzy Rules for Training Neural Networks

Simultaneous perturbation stochastic approximation （SPSA） belongs to the class of gradient-free optimization methods that extract gradient information from successive objective function evaluation. This paper describes an improved SPSA algorithm, which entails fuzzy adaptive gain sequences, gradient smoothing, and a step rejection procedure to enhance convergence and stability. The proposed fuzzy adaptive simultaneous perturbation approximation （FASPA） algorithm is particularly well suited to problems involving a large number of parameters such as those encountered in nonlinear system identification using neural networks （NNs）. Accordingly, a multilayer perceptron （MLP） network with popular training algorithms was used to predicate the system response. We found that an MLP trained by FASPSA had the desired accuracy that was comparable to results obtained by traditional system identification algorithms. Simulation results for typical nonlinear systems demonstrate that the proposed NN architecture trained with FASPSA yields improved system identification as measured by reduced time of convergence and a smaller identification error.

Prediction of diabetes and hypertension using multi-layer perceptron neural networks

Background:Diabetes and hypertension are two of the commonest diseases in the world.As they unfavorably affect people of different age groups,they have become a cause of concern and must be predicted and diagnosed well in advance.Objective:This research aims to determine the effectiveness of artificial neural networks(ANNs)in predicting diabetes and blood pressure diseases and to point out the factors which have a high impact on these diseases.Sample:This work used two online datasets which consist of data collected from 768 individuals.We applied neural network algorithms to predict if the individuals have those two diseases based on some factors.Diabetes prediction is based on five factors:age,weight,fat-ratio,glucose,and insulin,while blood pressure prediction is based on six factors:age,weight,fat-ratio,blood pressure,alcohol,and smoking.Method:A model based on the Multi-Layer Perceptron Neural Network(MLP)was implemented.The inputs of the network were the factors for each disease,while the output was the prediction of the disease’s occurrence.The model performance was compared with other classifiers such as Support Vector Machine(SVM)and K-Nearest Neighbors(KNN).We used performance metrics measures to assess the accuracy and performance of MLP.Also,a tool was implemented to help diagnose the diseases and to understand the results.Result:The model predicted the two diseases with correct classification rate(CCR)of 77.6%for diabetes and 68.7%for hypertension.The results indicate that MLP correctly predicts the probability of being diseased or not,and the performance can be significantly increased compared with both SVM and KNN.This shows MLPs effectiveness in early disease prediction.

A hybrid constriction coefficientbased particle swarm optimization and gravitational search algorithm for training multi-layer perceptron

Purpose-In this paper,a newly proposed hybridization algorithm namely constriction coefficient-based particle swarm optimization and gravitational search algorithm(CPSOGSA)has been employed for training MLP to overcome sensitivity to initialization,premature convergence,and stagnation in local optima problems of MLP.Design/methodology/approach-In this study,the exploration of the search space is carried out by gravitational search algorithm(GSA)and optimization of candidate solutions,i.e.exploitation is performed by particle swarm optimization(PSO).For training the multi-layer perceptron(MLP),CPSOGSA uses sigmoid fitness function for finding the proper combination of connection weights and neural biases to minimize the error.Secondly,a matrix encoding strategy is utilized for providing one to one correspondence between weights and biases of MLP and agents of CPSOGSA.Findings-The experimental findings convey that CPSOGSA is a better MLP trainer as compared to other stochastic algorithms because it provides superior results in terms of resolving stagnation in local optima and convergence speed problems.Besides,it gives the best results for breast cancer,heart,sine function and sigmoid function datasets as compared to other participating algorithms.Moreover,CPSOGSA also provides very competitive results for other datasets.Originality/value-The CPSOGSA performed effectively in overcoming stagnation in local optima problem and increasing the overall convergence speed of MLP.Basically,CPSOGSA is a hybrid optimization algorithm which has powerful characteristics of global exploration capability and high local exploitation power.In the research literature,a little work is available where CPSO and GSA have been utilized for training MLP.The only related research paper was given by Mirjalili et al.,in 2012.They have used standard PSO and GSA for training simple FNNs.However,the work employed only three datasets and used the MSE performance metric for evaluating the efficiency of the algorithms.In this paper,eight different standard datasets and five performance metrics have been utilized for investigating the efficiency of CPSOGSA in training MLPs.In addition,a non-parametric pair-wise statistical test namely the Wilcoxon rank-sum test has been carried out at a 5%significance level to statistically validate the simulation results.Besides,eight state-of-the-art metaheuristic algorithms were employed for comparative analysis of the experimental results to further raise the authenticity of the experimental setup.

基于前馈多层感知器的网络入侵检测的多数据包分析

提出了一种新型网络入侵检测模型,在该模型中,首先将截获的数据包结合历史数据包数据库进行协议分析,找出可能存在的入侵行为的相关数据包,然后采用前馈多层感知器神经网络对这些相关的数据包进行回归分析,最终获得检测结果。该模型与传统采用单数据包检测方式的网络入侵检测系统(NIDS)模型相比,具有更低的漏检率。

可变神经网络结构下的遥感影像光谱分解方法

多层感知神经网络(MLP)是主流的非线性分解方法,但是目前缺乏有效方法处理MLP分解结果中的丰度负值问题。为此,提出一种可变神经网络结构的方法,逐步去除负值丰度对应的端元,并调整相应的网络结构使之针对剩余的端元进行分解。通过武汉地区模拟TM遥感影像实验可以发现,该方法与传统MLP方法以及线性光谱分解方法的平均误差分别为0.077 7、0.081 9、0.094 3,说明该方法的分解精度高于其他2种分解方法,能克服丰度负值问题。

面向复杂场景图像的文本定位新方法

针对复杂场景文本,提出了通过投影产生候选文本块的新算法和针对该算法的候选文本块分析方法。首先根据MLP网络的输出确定图像每个像素点是文本像素点还是非文本像素点,得到候选二值图像。然后根据候选二值图像使用投影法生成候选文本块,针对该投影法,本文提出了频率分析法剔除非文本块,有效的提高了定位准确率。实验表明,本文的方法实现简单,而且可以得到较好的文本定位效果。

发音错误检测中基于多数据流的Tandem特征方法

针对发音错误检测中标注的发音数据资源有限的情况,提出在Tandem系统框架下利用其他数据来提高特征的区分性。以中国人的英语发音为研究对象,选取了相对容易获取的无校正发音数据、母语普通话和母语英语作为辅助数据,实验结果表明,这几种数据都能够有效地提高系统性能,其中无校正数据表现出最好的性能。同时,比较了不同的扩展帧长,以多层神经感知(MLP)和深度神经网络(DNN)作为典型的浅层和深层神经网络,以及Tandem特征的不同结构对系统性能的影响。最后,多数据流融合的策略用于进一步提高系统性能,基于DNN的无校正发音数据流和母语英语数据流合并的Tandem特征取得了最好的性能,与基线系统相比,识别正确率提高了7.96%,错误类型诊断正确率提高了14.71%。

基于多过程信号的轧辊磨削表面粗糙度智能预测

由于轧辊磨削表面粗糙度预测困难,且其预测精度不足,为此,笔者提出了一种基于多过程信号的轧辊磨削表面粗糙度智能预测方法。首先,以砂轮转速、磨削深度、拖板速度和头架转速为变量,对轧辊进行了全因素磨削实验,采集了磨削过程中的多过程信号,即声发射信号、振动信号和主轴电流信号,测量了磨后轧辊的表面粗糙度;对信号进行了分段处理,强化了信号与粗糙度的关联,并对粗糙度进行了离散化处理,将回归问题转化为分类问题;然后,提取了各类信号在时域和频域上的众多特征值,并利用主成分分析法(PCA)对其进行了特征降维融合,构建了多种类型的特征输入;最后,利用网格搜索法优化了多层感知机(MLP)网络,得到了粗糙度的预测模型,实现了对轧辊磨削表面粗糙度的智能预测。研究结果表明:相较于单信号方案,多信号方案能够提供更全面、准确的信息;基于PCA的降维融合特征能进一步提高MLP网络的预测效果,其准确率为78.16%,F1值为0.7776,平均偏离距离为0.29。

Modeling spatio-temporal distribution of soil moisture by deep learning-based cellular automata model

Soil moisture content （SMC） is a key hydrological parameter in agriculture,meteorology and climate change,and understanding of spatio-temporal distributions of SMC in farmlands is important to address the precise irrigation scheduling.However,the hybrid interaction of static and dynamic environmental parameters makes it particularly difficult to accurately and reliably model the distribution of SMC.At present,deep learning wins numerous contests in machine learning and hence deep belief network （DBN） ,a breakthrough in deep learning is trained to extract the transition functions for the simulation of the cell state changes.In this study,we used a novel macroscopic cellular automata （MCA） model by combining DBN to predict the SMC over an irrigated corn field （an area of 22 km^2） in the Zhangye oasis,Northwest China.Static and dynamic environmental variables were prepared with regard to the complex hydrological processes.The widely used neural network,multi-layer perceptron （MLP） ,was utilized for comparison to DBN.The hybrid models （MLP-MCA and DBN-MCA） were calibrated and validated on SMC data within four months,i.e.June to September 2012,which were automatically observed by a wireless sensor network （WSN） .Compared with MLP-MCA,the DBN-MCA model led to a decrease in root mean squared error （RMSE） by 18%.Thus,the differences of prediction errors increased due to the propagating errors of variables,difficulties of knowing soil properties and recording irrigation amount in practice.The sequential Gaussian simulation （s Gs） was performed to assess the uncertainty of soil moisture estimations.Calculated with a threshold of SMC for each grid cell,the local uncertainty of simulated results in the post processing suggested that the probability of SMC less than 25% will be difference in different areas at different time periods.The current results showed that the DBN-MCA model performs better than the MLP-MCA model,and the DBN-MCA model provides a powerful tool for predicting SMC in highly non-linear forms.Moreover,because modeling soil moisture by using environmental variables is gaining increasing popularity,DBN techniques could contribute a lot to enhancing the calibration of MCA-based SMC estimations and hence provide an alternative approach for SMC monitoring in irrigation systems on the basis of canals.

Parkinson’s Disease Detection Using Biogeography-Based Optimization

In recent years,Parkinson’s Disease(PD)as a progressive syndrome of the nervous system has become highly prevalent worldwide.In this study,a novel hybrid technique established by integrating a Multi-layer Perceptron Neural Network(MLP)with the Biogeography-based Optimization(BBO)to classify PD based on a series of biomedical voice measurements.BBO is employed to determine the optimal MLP parameters and boost prediction accuracy.The inputs comprised of 22 biomedical voice measurements.The proposed approach detects two PD statuses:0-disease status and 1-good control status.The performance of proposed methods compared with PSO,GA,ACO and ES method.The outcomes affirm that the MLP-BBO model exhibits higher precision and suitability for PD detection.The proposed diagnosis system as a type of speech algorithm detects early Parkinson’s symptoms,and consequently,it served as a promising new robust tool with excellent PD diagnosis performance.

Environmental Sound Classification Using Deep Learning

Perhaps hearing impairment individuals cannot identify the environmental sounds due to noise around them.However,very little research has been conducted in this domain.Hence,the aim of this study is to categorize sounds generated in the environment so that the impairment individuals can distinguish the sound categories.To that end first we define nine sound classes--air conditioner,car horn,children playing,dog bark,drilling,engine idling,jackhammer,siren,and street music--typically exist in the environment.Then we record 100 sound samples from each category and extract features of each sound category using Mel-Frequency Cepstral Coefficients(MFCC).The training dataset is developed using this set of features together with the class variable;sound category.Sound classification is a complex task and hence,we use two Deep Learning techniques;Multi Layer Perceptron(MLP)and Convolution Neural Network(CNN)to train classification models.The models are tested using a separate test set and the performances of the models are evaluated using precision,recall and F1-score.The results show that the CNN model outperforms the MLP.However,the MLP also provided a decent accuracy in classifying unknown environmental sounds.

CNN and MLP neural network ensembles for packet classification and adversary defense

Machine learning techniques such as artificial neural networks are seeing increased use in the examination of communication network research questions.Central to many of these research questions is the need to classify packets and improve visibility.Multi-Layer Perceptron(MLP)neural networks and Convolutional Neural Networks(CNNs)have been used to successfully identify individual packets.However,some datasets create instability in neural network models.Machine learning can also be subject to data injection and misclassification problems.In addition,when attempting to address complex communication network challenges,extremely high classification accuracy is required.Neural network ensembles can work towards minimizing or even eliminating some of these problems by comparing results from multiple models.After ensembles tuning,training time can be reduced,and a viable and effective architecture can be obtained.Because of their effectiveness,ensembles can be utilized to defend against data poisoning attacks attempting to create classification errors.In this work,ensemble tuning and several voting strategies are explored that consistently result in classification accuracy above 99%.In addition,ensembles are shown to be effective against these types of attack by maintaining accuracy above 98%.

Design and analysis of control system using neural network for regulated DC power supply

Conventional control systems used for regulated power supplies,including the proportional integral and derivation(PID)controller,have some serious disadvantages.The PID controller has a delayed feedback associated with the control action and requires a lot of mathematical derivations.This paper presents a novel controlling system based on the artificial neural network(ANN),which can be used to regulate the output voltage of the DC power supply.Using MATLABTM,the designed control system was tested and analyzed with two types of back-propagation algorithms.This paper presents the results of the simulation that includes sum-squared error(SSE)and mean-squared error(MSE),and gives a detailed comparison of these values for the two algorithms.Hardware verification of the new system,using RS232 interface and Microsoft Visual Basic 6.0,was implemented,showing very good consistency with the simulation results.The proposed control system,compared to PID and other conventional controllers,requires less mathematical derivation in design and it is easier to implement.

Parallel compact integration in handwritten Chinese character recognition

In this paper, a new parallel compact integration scheme based on multi-layer perceptron (MLP) networks is proposed to solve handwritten Chinese character recognition (HCCR) problems. The idea of metasynthesis is applied to HCCR, and compact MLP network classifier is defined. Human intelligence and computer capabilities are combined together effectively through a procedure of two-step supervised learning. Compared with previous integration schemes, this scheme is characterized with parallel compact structure and better performance. It provides a promising way for applying MLP to large vocabulary classification.