A Multilayer Recurrent Fuzzy Neural Network for Accurate Dynamic System Modeling

A multilayer recurrent fuzzy neural network(MRFNN)is proposed for accurate dynamic system modeling.The proposed MRFNN has six layers combined with T-S fuzzy model.The recurrent structures are formed by local feedback connections in the membership layer and the rule layer.With these feedbacks,the fuzzy sets are time-varying and the temporal problem of dynamic system can be solved well.The parameters of MRFNN are learned by chaotic search(CS)and least square estimation(LSE)simultaneously,where CS is for tuning the premise parameters and LSE is for updating the consequent coefficients accordingly.Results of simulations show the proposed approach is effective for dynamic system modeling with high accuracy.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

RECURRENT NEURAL NETWORK MODEL BASED ON PROJECTIVE OPERATOR AND ITS APPLICATION TO OPTIMIZATION PROBLEMS

The recurrent neural network （RNN） model based on projective operator was studied. Different from the former study, the value region of projective operator in the neural network in this paper is a general closed convex subset of n-dimensional Euclidean space and it is not a compact convex set in general, that is, the value region of projective operator is probably unbounded. It was proved that the network has a global solution and its solution trajectory converges to some equilibrium set whenever objective function satisfies some conditions. After that, the model was applied to continuously differentiable optimization and nonlinear or implicit complementarity problems. In addition, simulation experiments confirm the efficiency of the RNN.

Nonlinear model predictive control based on hyper chaotic diagonal recurrent neural network

Nonlinear model predictive controllers(NMPC)can predict the future behavior of the under-controlled system using a nonlinear predictive model.Here,an array of hyper chaotic diagonal recurrent neural network(HCDRNN)was proposed for modeling and predicting the behavior of the under-controller nonlinear system in a moving forward window.In order to improve the convergence of the parameters of the HCDRNN to improve system’s modeling,the extent of chaos is adjusted using a logistic map in the hidden layer.A novel NMPC based on the HCDRNN array(HCDRNN-NMPC)was proposed that the control signal with the help of an improved gradient descent method was obtained.The controller was used to control a continuous stirred tank reactor(CSTR)with hard-nonlinearities and input constraints,in the presence of uncertainties including external disturbance.The results of the simulations show the superior performance of the proposed method in trajectory tracking and disturbance rejection.Parameter convergence and neglectable prediction error of the neural network(NN),guaranteed stability and high tracking performance are the most significant advantages of the proposed scheme.

Multi-GPU Based Recurrent Neural Network Language Model Training

Recurrent neural network language models (RNNLMs) have been applied in a wide range of research fields, including nature language processing and speech recognition. One challenge in training RNNLMs is the heavy computational cost of the crucial back-propagation (BP) algorithm. This paper presents an effective approach to train recurrent neural network on multiple GPUs, where parallelized stochastic gradient descent (SGD) is applied. Results on text-based experiments show that the proposed approach achieves 3.4× speedup on 4 GPUs than the single one, without any performance loss in language model perplexity.

Robust stability analysis of Takagi-Sugeno uncertain stochastic fuzzy recurrent neural networks with mixed time-varying delays

In this paper, the global stability of Takagi-Sugeno （TS） uncertain stochastic fuzzy recurrent neural networks with discrete and distributed time-varying delays （TSUSFRNNs） is considered. A novel LMI-based stability criterion is obtained by using Lyapunov functional theory to guarantee the asymptotic stability of TSUSFRNNs. The proposed stability conditions are demonstrated through numerical examples. Furthermore, the supplementary requirement that the time derivative of time-varying delays must be smaller than one is removed. Comparison results are demonstrated to show that the proposed method is more able to guarantee the widest stability region than the other methods available in the existing literature.

Robust exponential stability analysis of a larger class of discrete-time recurrent neural networks

The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced to provide a general framework for stability analysis of RNNs. Most of the existing RNNs can be transformed into SNNMs to be analyzed in a unified way. Applying Lyapunov stability theory method and S-Procedure technique, two useful criteria of robust exponential stability for the discrete-time SNNMs are derived. The conditions presented are formulated as linear matrix inequalities (LMIs) to be easily solved using existing efficient convex optimization techniques. An example is presented to demonstrate the transformation procedure and the effectiveness of the results.

Vulnerability Detection of Ethereum Smart Contract Based on SolBERT-BiGRU-Attention Hybrid Neural Model

In recent years,with the great success of pre-trained language models,the pre-trained BERT model has been gradually applied to the field of source code understanding.However,the time cost of training a language model from zero is very high,and how to transfer the pre-trained language model to the field of smart contract vulnerability detection is a hot research direction at present.In this paper,we propose a hybrid model to detect common vulnerabilities in smart contracts based on a lightweight pre-trained languagemodel BERT and connected to a bidirectional gate recurrent unitmodel.The downstream neural network adopts the bidirectional gate recurrent unit neural network model with a hierarchical attention mechanism to mine more semantic features contained in the source code of smart contracts by using their characteristics.Our experiments show that our proposed hybrid neural network model SolBERT-BiGRU-Attention is fitted by a large number of data samples with smart contract vulnerabilities,and it is found that compared with the existing methods,the accuracy of our model can reach 93.85%,and the Micro-F1 Score is 94.02%.

A REALISTIC MODEL OF NEURAL NETWORKS

A realistic model of neural networks was proposed in this paper.The dynamicprocess of neural impulse discharging was considered.The equations of the model correspondto postsynaptic potentials,receptor potentials,initial segment graded potentials and the impulsetrain along the axon respectively.To solve the equations numerically,a recurrent algorithm and itscorresponding flow chart was also developed.The simulation results can imitate adaptation,post-excitation inhibition,and phase locking of sensory receptors;they can also imitate the transientresponses of lateral inhibitory network and Mach band phenomenon when they trended to besteady.The simulation results also showed that the lateral inhibitory network was sensitive tomoving objects.

Adaptive Internal Model Control of a DC Motor Drive System Using Dynamic Neural Network

This work concerns the study of problems relating to the adaptive internal model control of DC motor in both cases conventional and neural. The most important aspects of design building blocks of adaptive internal model control are the choice of architectures, learning algorithms, and examples of learning. The choice of parametric adaptation algorithm for updating elements of the conventional adaptive internal model control shows limitations. To overcome these limitations, we chose the architectures of neural networks deduced from the conventional models and the Levenberg-marquardt during the adjustment of system parameters of the adaptive neural internal model control. The results of this latest control showed compensation for disturbance, good trajectory tracking performance and system stability.

Deep learning neural networks for spatially explicit prediction of flash flood probability

Flood probability maps are essential for a range of applications,including land use planning and developing mitigation strategies and early warning systems.This study describes the potential application of two architectures of deep learning neural networks,namely convolutional neural networks(CNN)and recurrent neural networks(RNN),for spatially explicit prediction and mapping of flash flood probability.To develop and validate the predictive models,a geospatial database that contained records for the historical flood events and geo-environmental characteristics of the Golestan Province in northern Iran was constructed.The step-wise weight assessment ratio analysis(SWARA)was employed to investigate the spatial interplay between floods and different influencing factors.The CNN and RNN models were trained using the SWARA weights and validated using the receiver operating characteristics technique.The results showed that the CNN model(AUC=0.832,RMSE=0.144)performed slightly better than the RNN model(AUC=0.814,RMSE=0.181)in predicting future floods.Further,these models demonstrated an improved prediction of floods compared to previous studies that used different models in the same study area.This study showed that the spatially explicit deep learning neural network models are successful in capturing the heterogeneity of spatial patterns of flood probability in the Golestan Province,and the resulting probability maps can be used for the development of mitigation plans in response to the future floods.The general policy implication of our study suggests that design,implementation,and verification of flood early warning systems should be directed to approximately 40%of the land area characterized by high and very susceptibility to flooding.

基于CNN-BiGRU-Attention的短期电力负荷预测

针对目前电力负荷数据随机性强,影响因素复杂,传统单一预测模型精度低的问题,结合卷积神经网络(Convolutional neural network,CNN)、双向门控循环单元(Bi-directional gated recurrent unit,BiGRU)以及注意力机制(Attention)在短期电力负荷预测上的不同优点,提出一种基于CNN-BiGRU-Attention的混合预测模型。该方法首先通过CNN对历史负荷和气象数据进行初步特征提取,然后利用BiGRU进一步挖掘特征数据间时序关联,再引入注意力机制,对BiGRU输出状态给与不同权重,强化关键特征,最后完成负荷预测。试验结果表明,该模型的平均绝对百分比误差(Mean absolute percentage error,MAPE)、均方根误差(Root mean square error,RMSE)、判定系数(R-square,R~2)分别为0.167%、0.057%、0.993,三项指标明显优于其他模型,具有更高的预测精度和稳定性,验证了模型在短期负荷预测中的优势。

基于CNN-GRU-ISSA-XGBoost的短期光伏功率预测

针对光伏功率随机性及波动性大,单一预测模型往往难以准确分析历史数据波动规律,从而导致预测精度不高的问题,提出一种基于卷积神经网络-门控循环单元(CNN-GRU)和改进麻雀搜索算法(ISSA)优化的极限梯度提升(XGBoost)模型的短期光伏功率预测组合模型.首先去除历史数据中的异常值并对其进行归一化处理,利用主成分分析法(PCA)进行特征选取,以便更好地识别影响光伏功率的关键因素.然后采用CNN网络提取数据的空间特征,再经过GRU网络提取时间特征,针对XGBoost模型手动配置参数困难、随机性大的问题,利用ISSA对模型超参数寻优.最后对两种方法预测的结果用误差倒数法减小误差的同时对权重进行更新,得到新的预测值,从而完成对光伏功率的预测.实验结果表明,所提出的CNN-GRU-ISSA-XGBoost组合模型具有更强的适应性和更高的精度.

基于循环神经网络的2-DOF软体机械臂运动建模与控制

因现有软体机械臂材料刚度小、模量不稳定,导致建模与控制难度大.提出一种基于循环神经网络(recurrentneuralnetwork,RNN)的方法,用于二自由度(two-degree-of-freedom,2-DOF)软体机械臂的运动建模与控制.使用动作捕捉仪采集不同气压、负载下的位置坐标,并将其导入门控循环单元(gated recurrentunit,GRU)神经网络模型进行训练.当调节超参数至网络结构最优时,测试集准确度可达98.87%.在此基础上,构建气压与负载到末端位置的映射函数.实验结果表明,本方法可将机械臂的控制精度提升至6»8 mm,显著降低了软体机器人的控制与建模难度.

基于“分解-重组-预测-集成”模式的Heston期权定价模型

精准合理地期权定价对于改善市场流动性、优化投资者结构、稳定金融市场拥有重要意义。本文提出了一种结合“分解-重组-预测-集成”思想的Heston期权定价模型,该模型利用Heston模型进行初始定价,通过自适应噪声完全集合经验模态分解(CEEMDAN)对定价误差进行分解与重构,获得高频项、低频项及趋势项,然后使用门控循环单元(GRU)估计高频项及低频项,使用差分整合移动平均自回归(ARIMA)估计趋势项,所有估计值集成汇总得到定价误差估计值,最后使用定价误差估计值对Heston模型的初始定价结果进行修正后获得最终定价结果。使用华夏上证50ETF、华泰柏瑞沪深300ETF和嘉实沪深300ETF期权数据验证模型,实证结果显示,在模型结构更加简单的基础上,本文提出模型的精度普遍优于基准模型。

基于用户性格和语义-结构特征的文本评论情感分类方法

由于传统文本评论情感分类方法通常忽略用户性格对于情感分类结果的影响,提出一种基于用户性格和语义-结构特征的文本评论情感分类方法(User Personality and Semantic-structural Features based Sentiment Classification Method for Text Comments,BF_Bi GAC).依据大五人格模型能够有效表达用户性格的优势,通过计算不同维度性格得分,从评论文本中获取用户性格特征.利用双向门控循环单元(Bidirectional Gated Recurrent Unit,Bi GRU)和卷积神经网络(Convolutional Neural Network,CNN)可以有效提取文本上下文语义特征和局部结构特征的优势,提出一种基于Bi GRU、CNN和双层注意力机制的文本语义-结构特征获取方法.为区分不同类型特征的影响,引入混合注意力层实现对用户性格特征和文本语义-结构特征的有效融合,以此获得最终的文本向量表达.在IMDB、Yelp-2、Yelp-5及Ekman四个评论数据集上的对比实验结果表明,BF_Bi GAC在分类准确率(Accuracy)和加权macro F_(1)值(F_(w))上均获得较好表现,相对于拼接Bi GRU、CNN的情感分类方法(Sentiment Classification Method Concatenating Bi GRU and CNN,Bi G-RU_CNN)在Accuracy值上分别提升0.020、0.012、0.017及0.011,相对于拼接CNN、Bi GRU的情感分类方法(Sentiment Classification Method Concatenating CNN and Bi GRU,Conv Bi LSTM)F_(w)值上分别提升0.022、0.013、0.028及0.023;相对于预训练模型BERT和Ro BERTa,BF_Bi GAC在保证分类精度的情况下获得了较高的运行效率.

基于自回归小波神经网络的机械臂自适应滑模控制

针对机械臂存在模型不确定和未知扰动的问题,提出一种动力学模型参数分块逼近的神经网络非奇异终端滑模(nonsingular terminal sliding mode, NTSM)控制方法.为加快系统跟踪误差的收敛速度,避免传统终端滑模存在的奇异性问题,采用一种非奇异终端滑模面.利用多组自回归小波神经网络(self-recurrent wavelet neural network, SRWNN)分块逼近系统未知的动力学模型参数,并采用自适应更新律调整权重.通过积分控制项补偿SRWNN的逼近误差,并使用Lyapunov稳定性理论证明了系统稳定性.使用MATLAB进行仿真分析,分块SRWNN滑模控制与滑模控制、整体SRWNN滑模控制相比,关节角度跟踪误差的平均稳态误差分别降低了31.9%、76.5%,表明此方法是一种可靠、有效的轨迹跟踪控制方法.

基于人工神经网络的自然语言处理技术研究

文章探讨了基于人工神经网络的自然语言处理技术,首先,阐述了人工神经网络的定义、结构、工作原理,以及与深度学习的关系。其次,详细研究了基于人工神经网络的自然语言处理技术,包括神经网络模型、词嵌入技术、循环神经网络、长短期记忆网络、转换器模型与自注意力机制等,并分析了这些技术面临的挑战。最后,通过实验设计与结果分析验证了所提出方法的有效性。文章研究内容对于推动自然语言处理技术的发展和应用具有重要意义。

Idea plagiarism detection with recurrent neural networks and vector space model

Purpose-Natural languages have a fundamental quality of suppleness that makes it possible to present a single idea in plenty of different ways.This feature is often exploited in the academic world,leading to the theft of work referred to as plagiarism.Many approaches have been put forward to detect such cases based on various text features and grammatical structures of languages.However,there is a huge scope of improvement for detecting intelligent plagiarism.Design/methodology/approach-To realize this,the paper introduces a hybrid model to detect intelligent plagiarism by breaking the entire process into three stages:(1)clustering,(2)vector formulation in each cluster based on semantic roles,normalization and similarity index calculation and(3)Summary generation using encoder-decoder.An effective weighing scheme has been introduced to select terms used to build vectors based on K-means,which is calculated on the synonym set for the said term.If the value calculated in the last stage lies above a predefined threshold,only then the next semantic argument is analyzed.When the similarity score for two documents is beyond the threshold,a short summary for plagiarized documents is created.Findings-Experimental results show that this method is able to detect connotation and concealment used in idea plagiarism besides detecting literal plagiarism.Originality/value-The proposed model can help academics stay updated by providing summaries of relevant articles.It would eliminate the practice of plagiarism infesting the academic community at an unprecedented pace.The model will also accelerate the process of reviewing academic documents,aiding in the speedy publishing of research articles.

基于RCMAC干扰观测器的高超声速飞行控制

利用自回归小脑模型神经网络(recurrent cerebella model neural network,RCMAC)良好的非线性逼近能力和自学习能力,结合反馈线性化和反演控制方法,提出了一种自适应非线性控制策略,用于高速再入飞行器控制系统的设计。该方案将RCMAC干扰观测器(recurrent cerebella disturbance observer,RCDO)用于估计系统模型的不确定项,同时采用反演控制方式设计伪线性控制项,并利用符号函数逼近误差的上界,根据Lyapunov稳定性理论设计了权值更新规则,保证闭环系统信号有界。高速再入飞行器的六自由度仿真结果验证了方法的有效性和鲁棒性。