A multilayer recurrent fuzzy neural network(MRFNN)is proposed for accurate dynamic system modeling.The proposed MRFNN has six layers combined with T-S fuzzy model.The recurrent structures are formed by local feedback ...A multilayer recurrent fuzzy neural network(MRFNN)is proposed for accurate dynamic system modeling.The proposed MRFNN has six layers combined with T-S fuzzy model.The recurrent structures are formed by local feedback connections in the membership layer and the rule layer.With these feedbacks,the fuzzy sets are time-varying and the temporal problem of dynamic system can be solved well.The parameters of MRFNN are learned by chaotic search(CS)and least square estimation(LSE)simultaneously,where CS is for tuning the premise parameters and LSE is for updating the consequent coefficients accordingly.Results of simulations show the proposed approach is effective for dynamic system modeling with high accuracy.展开更多
A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a force...A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.展开更多
The recurrent neural network (RNN) model based on projective operator was studied. Different from the former study, the value region of projective operator in the neural network in this paper is a general closed con...The recurrent neural network (RNN) model based on projective operator was studied. Different from the former study, the value region of projective operator in the neural network in this paper is a general closed convex subset of n-dimensional Euclidean space and it is not a compact convex set in general, that is, the value region of projective operator is probably unbounded. It was proved that the network has a global solution and its solution trajectory converges to some equilibrium set whenever objective function satisfies some conditions. After that, the model was applied to continuously differentiable optimization and nonlinear or implicit complementarity problems. In addition, simulation experiments confirm the efficiency of the RNN.展开更多
Nonlinear model predictive controllers(NMPC)can predict the future behavior of the under-controlled system using a nonlinear predictive model.Here,an array of hyper chaotic diagonal recurrent neural network(HCDRNN)was...Nonlinear model predictive controllers(NMPC)can predict the future behavior of the under-controlled system using a nonlinear predictive model.Here,an array of hyper chaotic diagonal recurrent neural network(HCDRNN)was proposed for modeling and predicting the behavior of the under-controller nonlinear system in a moving forward window.In order to improve the convergence of the parameters of the HCDRNN to improve system’s modeling,the extent of chaos is adjusted using a logistic map in the hidden layer.A novel NMPC based on the HCDRNN array(HCDRNN-NMPC)was proposed that the control signal with the help of an improved gradient descent method was obtained.The controller was used to control a continuous stirred tank reactor(CSTR)with hard-nonlinearities and input constraints,in the presence of uncertainties including external disturbance.The results of the simulations show the superior performance of the proposed method in trajectory tracking and disturbance rejection.Parameter convergence and neglectable prediction error of the neural network(NN),guaranteed stability and high tracking performance are the most significant advantages of the proposed scheme.展开更多
Recurrent neural network language models (RNNLMs) have been applied in a wide range of research fields, including nature language processing and speech recognition. One challenge in training RNNLMs is the heavy comput...Recurrent neural network language models (RNNLMs) have been applied in a wide range of research fields, including nature language processing and speech recognition. One challenge in training RNNLMs is the heavy computational cost of the crucial back-propagation (BP) algorithm. This paper presents an effective approach to train recurrent neural network on multiple GPUs, where parallelized stochastic gradient descent (SGD) is applied. Results on text-based experiments show that the proposed approach achieves 3.4× speedup on 4 GPUs than the single one, without any performance loss in language model perplexity.展开更多
In this paper, the global stability of Takagi-Sugeno (TS) uncertain stochastic fuzzy recurrent neural networks with discrete and distributed time-varying delays (TSUSFRNNs) is considered. A novel LMI-based stabili...In this paper, the global stability of Takagi-Sugeno (TS) uncertain stochastic fuzzy recurrent neural networks with discrete and distributed time-varying delays (TSUSFRNNs) is considered. A novel LMI-based stability criterion is obtained by using Lyapunov functional theory to guarantee the asymptotic stability of TSUSFRNNs. The proposed stability conditions are demonstrated through numerical examples. Furthermore, the supplementary requirement that the time derivative of time-varying delays must be smaller than one is removed. Comparison results are demonstrated to show that the proposed method is more able to guarantee the widest stability region than the other methods available in the existing literature.展开更多
The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced t...The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced to provide a general framework for stability analysis of RNNs. Most of the existing RNNs can be transformed into SNNMs to be analyzed in a unified way. Applying Lyapunov stability theory method and S-Procedure technique, two useful criteria of robust exponential stability for the discrete-time SNNMs are derived. The conditions presented are formulated as linear matrix inequalities (LMIs) to be easily solved using existing efficient convex optimization techniques. An example is presented to demonstrate the transformation procedure and the effectiveness of the results.展开更多
In recent years,with the great success of pre-trained language models,the pre-trained BERT model has been gradually applied to the field of source code understanding.However,the time cost of training a language model ...In recent years,with the great success of pre-trained language models,the pre-trained BERT model has been gradually applied to the field of source code understanding.However,the time cost of training a language model from zero is very high,and how to transfer the pre-trained language model to the field of smart contract vulnerability detection is a hot research direction at present.In this paper,we propose a hybrid model to detect common vulnerabilities in smart contracts based on a lightweight pre-trained languagemodel BERT and connected to a bidirectional gate recurrent unitmodel.The downstream neural network adopts the bidirectional gate recurrent unit neural network model with a hierarchical attention mechanism to mine more semantic features contained in the source code of smart contracts by using their characteristics.Our experiments show that our proposed hybrid neural network model SolBERT-BiGRU-Attention is fitted by a large number of data samples with smart contract vulnerabilities,and it is found that compared with the existing methods,the accuracy of our model can reach 93.85%,and the Micro-F1 Score is 94.02%.展开更多
A realistic model of neural networks was proposed in this paper.The dynamicprocess of neural impulse discharging was considered.The equations of the model correspondto postsynaptic potentials,receptor potentials,initi...A realistic model of neural networks was proposed in this paper.The dynamicprocess of neural impulse discharging was considered.The equations of the model correspondto postsynaptic potentials,receptor potentials,initial segment graded potentials and the impulsetrain along the axon respectively.To solve the equations numerically,a recurrent algorithm and itscorresponding flow chart was also developed.The simulation results can imitate adaptation,post-excitation inhibition,and phase locking of sensory receptors;they can also imitate the transientresponses of lateral inhibitory network and Mach band phenomenon when they trended to besteady.The simulation results also showed that the lateral inhibitory network was sensitive tomoving objects.展开更多
This work concerns the study of problems relating to the adaptive internal model control of DC motor in both cases conventional and neural. The most important aspects of design building blocks of adaptive internal mod...This work concerns the study of problems relating to the adaptive internal model control of DC motor in both cases conventional and neural. The most important aspects of design building blocks of adaptive internal model control are the choice of architectures, learning algorithms, and examples of learning. The choice of parametric adaptation algorithm for updating elements of the conventional adaptive internal model control shows limitations. To overcome these limitations, we chose the architectures of neural networks deduced from the conventional models and the Levenberg-marquardt during the adjustment of system parameters of the adaptive neural internal model control. The results of this latest control showed compensation for disturbance, good trajectory tracking performance and system stability.展开更多
Flood probability maps are essential for a range of applications,including land use planning and developing mitigation strategies and early warning systems.This study describes the potential application of two archite...Flood probability maps are essential for a range of applications,including land use planning and developing mitigation strategies and early warning systems.This study describes the potential application of two architectures of deep learning neural networks,namely convolutional neural networks(CNN)and recurrent neural networks(RNN),for spatially explicit prediction and mapping of flash flood probability.To develop and validate the predictive models,a geospatial database that contained records for the historical flood events and geo-environmental characteristics of the Golestan Province in northern Iran was constructed.The step-wise weight assessment ratio analysis(SWARA)was employed to investigate the spatial interplay between floods and different influencing factors.The CNN and RNN models were trained using the SWARA weights and validated using the receiver operating characteristics technique.The results showed that the CNN model(AUC=0.832,RMSE=0.144)performed slightly better than the RNN model(AUC=0.814,RMSE=0.181)in predicting future floods.Further,these models demonstrated an improved prediction of floods compared to previous studies that used different models in the same study area.This study showed that the spatially explicit deep learning neural network models are successful in capturing the heterogeneity of spatial patterns of flood probability in the Golestan Province,and the resulting probability maps can be used for the development of mitigation plans in response to the future floods.The general policy implication of our study suggests that design,implementation,and verification of flood early warning systems should be directed to approximately 40%of the land area characterized by high and very susceptibility to flooding.展开更多
由于传统文本评论情感分类方法通常忽略用户性格对于情感分类结果的影响,提出一种基于用户性格和语义-结构特征的文本评论情感分类方法(User Personality and Semantic-structural Features based Sentiment Classification Method for ...由于传统文本评论情感分类方法通常忽略用户性格对于情感分类结果的影响,提出一种基于用户性格和语义-结构特征的文本评论情感分类方法(User Personality and Semantic-structural Features based Sentiment Classification Method for Text Comments,BF_Bi GAC).依据大五人格模型能够有效表达用户性格的优势,通过计算不同维度性格得分,从评论文本中获取用户性格特征.利用双向门控循环单元(Bidirectional Gated Recurrent Unit,Bi GRU)和卷积神经网络(Convolutional Neural Network,CNN)可以有效提取文本上下文语义特征和局部结构特征的优势,提出一种基于Bi GRU、CNN和双层注意力机制的文本语义-结构特征获取方法.为区分不同类型特征的影响,引入混合注意力层实现对用户性格特征和文本语义-结构特征的有效融合,以此获得最终的文本向量表达.在IMDB、Yelp-2、Yelp-5及Ekman四个评论数据集上的对比实验结果表明,BF_Bi GAC在分类准确率(Accuracy)和加权macro F_(1)值(F_(w))上均获得较好表现,相对于拼接Bi GRU、CNN的情感分类方法(Sentiment Classification Method Concatenating Bi GRU and CNN,Bi G-RU_CNN)在Accuracy值上分别提升0.020、0.012、0.017及0.011,相对于拼接CNN、Bi GRU的情感分类方法(Sentiment Classification Method Concatenating CNN and Bi GRU,Conv Bi LSTM)F_(w)值上分别提升0.022、0.013、0.028及0.023;相对于预训练模型BERT和Ro BERTa,BF_Bi GAC在保证分类精度的情况下获得了较高的运行效率.展开更多
Purpose-Natural languages have a fundamental quality of suppleness that makes it possible to present a single idea in plenty of different ways.This feature is often exploited in the academic world,leading to the theft...Purpose-Natural languages have a fundamental quality of suppleness that makes it possible to present a single idea in plenty of different ways.This feature is often exploited in the academic world,leading to the theft of work referred to as plagiarism.Many approaches have been put forward to detect such cases based on various text features and grammatical structures of languages.However,there is a huge scope of improvement for detecting intelligent plagiarism.Design/methodology/approach-To realize this,the paper introduces a hybrid model to detect intelligent plagiarism by breaking the entire process into three stages:(1)clustering,(2)vector formulation in each cluster based on semantic roles,normalization and similarity index calculation and(3)Summary generation using encoder-decoder.An effective weighing scheme has been introduced to select terms used to build vectors based on K-means,which is calculated on the synonym set for the said term.If the value calculated in the last stage lies above a predefined threshold,only then the next semantic argument is analyzed.When the similarity score for two documents is beyond the threshold,a short summary for plagiarized documents is created.Findings-Experimental results show that this method is able to detect connotation and concealment used in idea plagiarism besides detecting literal plagiarism.Originality/value-The proposed model can help academics stay updated by providing summaries of relevant articles.It would eliminate the practice of plagiarism infesting the academic community at an unprecedented pace.The model will also accelerate the process of reviewing academic documents,aiding in the speedy publishing of research articles.展开更多
文摘A multilayer recurrent fuzzy neural network(MRFNN)is proposed for accurate dynamic system modeling.The proposed MRFNN has six layers combined with T-S fuzzy model.The recurrent structures are formed by local feedback connections in the membership layer and the rule layer.With these feedbacks,the fuzzy sets are time-varying and the temporal problem of dynamic system can be solved well.The parameters of MRFNN are learned by chaotic search(CS)and least square estimation(LSE)simultaneously,where CS is for tuning the premise parameters and LSE is for updating the consequent coefficients accordingly.Results of simulations show the proposed approach is effective for dynamic system modeling with high accuracy.
基金supported by the Ministry of Trade,Industry & Energy(MOTIE,Korea) under Industrial Technology Innovation Program (No.10063424,'development of distant speech recognition and multi-task dialog processing technologies for in-door conversational robots')
文摘A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.
文摘The recurrent neural network (RNN) model based on projective operator was studied. Different from the former study, the value region of projective operator in the neural network in this paper is a general closed convex subset of n-dimensional Euclidean space and it is not a compact convex set in general, that is, the value region of projective operator is probably unbounded. It was proved that the network has a global solution and its solution trajectory converges to some equilibrium set whenever objective function satisfies some conditions. After that, the model was applied to continuously differentiable optimization and nonlinear or implicit complementarity problems. In addition, simulation experiments confirm the efficiency of the RNN.
文摘Nonlinear model predictive controllers(NMPC)can predict the future behavior of the under-controlled system using a nonlinear predictive model.Here,an array of hyper chaotic diagonal recurrent neural network(HCDRNN)was proposed for modeling and predicting the behavior of the under-controller nonlinear system in a moving forward window.In order to improve the convergence of the parameters of the HCDRNN to improve system’s modeling,the extent of chaos is adjusted using a logistic map in the hidden layer.A novel NMPC based on the HCDRNN array(HCDRNN-NMPC)was proposed that the control signal with the help of an improved gradient descent method was obtained.The controller was used to control a continuous stirred tank reactor(CSTR)with hard-nonlinearities and input constraints,in the presence of uncertainties including external disturbance.The results of the simulations show the superior performance of the proposed method in trajectory tracking and disturbance rejection.Parameter convergence and neglectable prediction error of the neural network(NN),guaranteed stability and high tracking performance are the most significant advantages of the proposed scheme.
文摘Recurrent neural network language models (RNNLMs) have been applied in a wide range of research fields, including nature language processing and speech recognition. One challenge in training RNNLMs is the heavy computational cost of the crucial back-propagation (BP) algorithm. This paper presents an effective approach to train recurrent neural network on multiple GPUs, where parallelized stochastic gradient descent (SGD) is applied. Results on text-based experiments show that the proposed approach achieves 3.4× speedup on 4 GPUs than the single one, without any performance loss in language model perplexity.
文摘In this paper, the global stability of Takagi-Sugeno (TS) uncertain stochastic fuzzy recurrent neural networks with discrete and distributed time-varying delays (TSUSFRNNs) is considered. A novel LMI-based stability criterion is obtained by using Lyapunov functional theory to guarantee the asymptotic stability of TSUSFRNNs. The proposed stability conditions are demonstrated through numerical examples. Furthermore, the supplementary requirement that the time derivative of time-varying delays must be smaller than one is removed. Comparison results are demonstrated to show that the proposed method is more able to guarantee the widest stability region than the other methods available in the existing literature.
基金the National Natural Science Foundation of China (No. 60504024)the Research Project of Zhejiang Provin-cial Education Department (No. 20050905), China
文摘The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced to provide a general framework for stability analysis of RNNs. Most of the existing RNNs can be transformed into SNNMs to be analyzed in a unified way. Applying Lyapunov stability theory method and S-Procedure technique, two useful criteria of robust exponential stability for the discrete-time SNNMs are derived. The conditions presented are formulated as linear matrix inequalities (LMIs) to be easily solved using existing efficient convex optimization techniques. An example is presented to demonstrate the transformation procedure and the effectiveness of the results.
基金supported by the National Natural Science Foundation of China(Grant Nos.62272120,62106030,U20B2046,62272119,61972105)the Technology Innovation and Application Development Projects of Chongqing(Grant Nos.cstc2021jscx-gksbX0032,cstc2021jscxgksbX0029).
文摘In recent years,with the great success of pre-trained language models,the pre-trained BERT model has been gradually applied to the field of source code understanding.However,the time cost of training a language model from zero is very high,and how to transfer the pre-trained language model to the field of smart contract vulnerability detection is a hot research direction at present.In this paper,we propose a hybrid model to detect common vulnerabilities in smart contracts based on a lightweight pre-trained languagemodel BERT and connected to a bidirectional gate recurrent unitmodel.The downstream neural network adopts the bidirectional gate recurrent unit neural network model with a hierarchical attention mechanism to mine more semantic features contained in the source code of smart contracts by using their characteristics.Our experiments show that our proposed hybrid neural network model SolBERT-BiGRU-Attention is fitted by a large number of data samples with smart contract vulnerabilities,and it is found that compared with the existing methods,the accuracy of our model can reach 93.85%,and the Micro-F1 Score is 94.02%.
基金Supported by National Natural Science Foundation of China
文摘A realistic model of neural networks was proposed in this paper.The dynamicprocess of neural impulse discharging was considered.The equations of the model correspondto postsynaptic potentials,receptor potentials,initial segment graded potentials and the impulsetrain along the axon respectively.To solve the equations numerically,a recurrent algorithm and itscorresponding flow chart was also developed.The simulation results can imitate adaptation,post-excitation inhibition,and phase locking of sensory receptors;they can also imitate the transientresponses of lateral inhibitory network and Mach band phenomenon when they trended to besteady.The simulation results also showed that the lateral inhibitory network was sensitive tomoving objects.
文摘This work concerns the study of problems relating to the adaptive internal model control of DC motor in both cases conventional and neural. The most important aspects of design building blocks of adaptive internal model control are the choice of architectures, learning algorithms, and examples of learning. The choice of parametric adaptation algorithm for updating elements of the conventional adaptive internal model control shows limitations. To overcome these limitations, we chose the architectures of neural networks deduced from the conventional models and the Levenberg-marquardt during the adjustment of system parameters of the adaptive neural internal model control. The results of this latest control showed compensation for disturbance, good trajectory tracking performance and system stability.
基金conducted by the Basic Research Project of the Korea Institute of Geoscience and Mineral Resources(KIGAM)funded by the Ministry of Science and ICT。
文摘Flood probability maps are essential for a range of applications,including land use planning and developing mitigation strategies and early warning systems.This study describes the potential application of two architectures of deep learning neural networks,namely convolutional neural networks(CNN)and recurrent neural networks(RNN),for spatially explicit prediction and mapping of flash flood probability.To develop and validate the predictive models,a geospatial database that contained records for the historical flood events and geo-environmental characteristics of the Golestan Province in northern Iran was constructed.The step-wise weight assessment ratio analysis(SWARA)was employed to investigate the spatial interplay between floods and different influencing factors.The CNN and RNN models were trained using the SWARA weights and validated using the receiver operating characteristics technique.The results showed that the CNN model(AUC=0.832,RMSE=0.144)performed slightly better than the RNN model(AUC=0.814,RMSE=0.181)in predicting future floods.Further,these models demonstrated an improved prediction of floods compared to previous studies that used different models in the same study area.This study showed that the spatially explicit deep learning neural network models are successful in capturing the heterogeneity of spatial patterns of flood probability in the Golestan Province,and the resulting probability maps can be used for the development of mitigation plans in response to the future floods.The general policy implication of our study suggests that design,implementation,and verification of flood early warning systems should be directed to approximately 40%of the land area characterized by high and very susceptibility to flooding.
文摘由于传统文本评论情感分类方法通常忽略用户性格对于情感分类结果的影响,提出一种基于用户性格和语义-结构特征的文本评论情感分类方法(User Personality and Semantic-structural Features based Sentiment Classification Method for Text Comments,BF_Bi GAC).依据大五人格模型能够有效表达用户性格的优势,通过计算不同维度性格得分,从评论文本中获取用户性格特征.利用双向门控循环单元(Bidirectional Gated Recurrent Unit,Bi GRU)和卷积神经网络(Convolutional Neural Network,CNN)可以有效提取文本上下文语义特征和局部结构特征的优势,提出一种基于Bi GRU、CNN和双层注意力机制的文本语义-结构特征获取方法.为区分不同类型特征的影响,引入混合注意力层实现对用户性格特征和文本语义-结构特征的有效融合,以此获得最终的文本向量表达.在IMDB、Yelp-2、Yelp-5及Ekman四个评论数据集上的对比实验结果表明,BF_Bi GAC在分类准确率(Accuracy)和加权macro F_(1)值(F_(w))上均获得较好表现,相对于拼接Bi GRU、CNN的情感分类方法(Sentiment Classification Method Concatenating Bi GRU and CNN,Bi G-RU_CNN)在Accuracy值上分别提升0.020、0.012、0.017及0.011,相对于拼接CNN、Bi GRU的情感分类方法(Sentiment Classification Method Concatenating CNN and Bi GRU,Conv Bi LSTM)F_(w)值上分别提升0.022、0.013、0.028及0.023;相对于预训练模型BERT和Ro BERTa,BF_Bi GAC在保证分类精度的情况下获得了较高的运行效率.
基金This work is supported by Technical Education Quality Program-TEQIP III.The project is implemented by NPIU,which is a unit of MHRD,Govt of India for implementation of world bank assisted projects in Technical Education.
文摘Purpose-Natural languages have a fundamental quality of suppleness that makes it possible to present a single idea in plenty of different ways.This feature is often exploited in the academic world,leading to the theft of work referred to as plagiarism.Many approaches have been put forward to detect such cases based on various text features and grammatical structures of languages.However,there is a huge scope of improvement for detecting intelligent plagiarism.Design/methodology/approach-To realize this,the paper introduces a hybrid model to detect intelligent plagiarism by breaking the entire process into three stages:(1)clustering,(2)vector formulation in each cluster based on semantic roles,normalization and similarity index calculation and(3)Summary generation using encoder-decoder.An effective weighing scheme has been introduced to select terms used to build vectors based on K-means,which is calculated on the synonym set for the said term.If the value calculated in the last stage lies above a predefined threshold,only then the next semantic argument is analyzed.When the similarity score for two documents is beyond the threshold,a short summary for plagiarized documents is created.Findings-Experimental results show that this method is able to detect connotation and concealment used in idea plagiarism besides detecting literal plagiarism.Originality/value-The proposed model can help academics stay updated by providing summaries of relevant articles.It would eliminate the practice of plagiarism infesting the academic community at an unprecedented pace.The model will also accelerate the process of reviewing academic documents,aiding in the speedy publishing of research articles.