Effects of data smoothing and recurrent neural network(RNN)algorithms for real-time forecasting of tunnel boring machine(TBM)performance

Tunnel boring machines(TBMs)have been widely utilised in tunnel construction due to their high efficiency and reliability.Accurately predicting TBM performance can improve project time management,cost control,and risk management.This study aims to use deep learning to develop real-time models for predicting the penetration rate(PR).The models are built using data from the Changsha metro project,and their performances are evaluated using unseen data from the Zhengzhou Metro project.In one-step forecast,the predicted penetration rate follows the trend of the measured penetration rate in both training and testing.The autoregressive integrated moving average(ARIMA)model is compared with the recurrent neural network(RNN)model.The results show that univariate models,which only consider historical penetration rate itself,perform better than multivariate models that take into account multiple geological and operational parameters(GEO and OP).Next,an RNN variant combining time series of penetration rate with the last-step geological and operational parameters is developed,and it performs better than other models.A sensitivity analysis shows that the penetration rate is the most important parameter,while other parameters have a smaller impact on time series forecasting.It is also found that smoothed data are easier to predict with high accuracy.Nevertheless,over-simplified data can lose real characteristics in time series.In conclusion,the RNN variant can accurately predict the next-step penetration rate,and data smoothing is crucial in time series forecasting.This study provides practical guidance for TBM performance forecasting in practical engineering.

Optimized Phishing Detection with Recurrent Neural Network and Whale Optimizer Algorithm

Phishing attacks present a persistent and evolving threat in the cybersecurity land-scape,necessitating the development of more sophisticated detection methods.Traditional machine learning approaches to phishing detection have relied heavily on feature engineering and have often fallen short in adapting to the dynamically changing patterns of phishingUniformResource Locator(URLs).Addressing these challenge,we introduce a framework that integrates the sequential data processing strengths of a Recurrent Neural Network(RNN)with the hyperparameter optimization prowess of theWhale Optimization Algorithm(WOA).Ourmodel capitalizes on an extensive Kaggle dataset,featuring over 11,000 URLs,each delineated by 30 attributes.The WOA’s hyperparameter optimization enhances the RNN’s performance,evidenced by a meticulous validation process.The results,encapsulated in precision,recall,and F1-score metrics,surpass baseline models,achieving an overall accuracy of 92%.This study not only demonstrates the RNN’s proficiency in learning complex patterns but also underscores the WOA’s effectiveness in refining machine learning models for the critical task of phishing detection.

An INS/GNSS integrated navigation in GNSS denied environment using recurrent neural network

In view of the failure of GNSS signals,this paper proposes an INS/GNSS integrated navigation method based on the recurrent neural network(RNN).This proposed method utilizes the calculation principle of INS and the memory function of the RNN to estimate the errors of the INS,thereby obtaining a continuous,reliable and high-precision navigation solution.The performance of the proposed method is firstly demonstrated using an INS/GNSS simulation environment.Subsequently,an experimental test on boat is also conducted to validate the performance of the method.The results show a promising application prospect for RNN in the field of positioning for INS/GNSS integrated navigation in the absence of GNSS signal,as it outperforms extreme learning machine(ELM)and EKF by approximately 30%and 60%,respectively.

Remaining Useful Life Prediction for a Roller in a Hot Strip Mill Based on Deep Recurrent Neural Networks

Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productivity of the hot rolling process.In addition,the RUL prediction for rollers is helpful in transitioning from the current regular maintenance strategy to conditional-based maintenance.Therefore,a new method that can extract coarse-grained and fine-grained features from batch data to predict the RUL of the rollers is proposed in this paper.Firstly,a new deep learning network architecture based on recurrent neural networks that can make full use of the extracted coarsegrained fine-grained features to estimate the heath indicator(HI)is developed,where the HI is able to indicate the health state of the roller.Following that,a state-space model is constructed to describe the HI,and the probabilistic distribution of RUL can be estimated by extrapolating the HI degradation model to a predefined failure threshold.Finally,application to a hot strip mill is given to verify the effectiveness of the proposed methods using data collected from an industrial site,and the relatively low RMSE and MAE values demonstrate its advantages compared with some other popular deep learning methods.

Practical Options for Adopting Recurrent Neural Network and Its Variants on Remaining Useful Life Prediction

The remaining useful life(RUL)of a system is generally predicted by utilising the data collected from the sensors that continuously monitor different indicators.Recently,different deep learning(DL)techniques have been used for RUL prediction and achieved great success.Because the data is often time-sequential,recurrent neural network(RNN)has attracted significant interests due to its efficiency in dealing with such data.This paper systematically reviews RNN and its variants for RUL prediction,with a specific focus on understanding how different components(e.g.,types of optimisers and activation functions)or parameters(e.g.,sequence length,neuron quantities)affect their performance.After that,a case study using the well-studied NASA’s C-MAPSS dataset is presented to quantitatively evaluate the influence of various state-of-the-art RNN structures on the RUL prediction performance.The result suggests that the variant methods usually perform better than the original RNN,and among which,Bi-directional Long Short-Term Memory generally has the best performance in terms of stability,precision and accuracy.Certain model structures may fail to produce valid RUL prediction result due to the gradient vanishing or gradient exploring problem if the parameters are not chosen appropriately.It is concluded that parameter tuning is a crucial step to achieve optimal prediction performance.

New Stability Criteria for Recurrent Neural Networks with a Time-varying Delay

This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore, the relationship among the timevarying delay, its upper bound and their difierence, is taken into account, and novel bounding techniques for 1- τ（t） are employed. As a result, without ignoring any useful term in the derivative of the Lyapunov-Krasovskii functional, the resulting delay-dependent criteria show less conservative than the existing ones. Finally, a numerical example is given to demonstrate the effectiveness of the proposed methods.

Robust exponential stability analysis of a larger class of discrete-time recurrent neural networks

The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced to provide a general framework for stability analysis of RNNs. Most of the existing RNNs can be transformed into SNNMs to be analyzed in a unified way. Applying Lyapunov stability theory method and S-Procedure technique, two useful criteria of robust exponential stability for the discrete-time SNNMs are derived. The conditions presented are formulated as linear matrix inequalities (LMIs) to be easily solved using existing efficient convex optimization techniques. An example is presented to demonstrate the transformation procedure and the effectiveness of the results.

Global stability of interval recurrent neural networks

The robust global exponential stability of a class of interval recurrent neural networks（RNNs） is studied,and a new robust stability criterion is obtained in the form of linear matrix inequality.The problem of robust stability of interval RNNs is transformed into a problem of solving a class of linear matrix inequalities.Thus,the robust stability of interval RNNs can be analyzed by directly using the linear matrix inequalities（LMI） toolbox of MATLAB.Numerical example is given to show the effectiveness of the obtained results.

ENDPOINT DETECTOR OF NOISY SPEECH SIGNAL USING A RECURRENT NEURAL NETWORK

ＩｎｔｒｏｄｕｃｔｉｏｎＥｎｄｐｏｉｎｔｄｅｔｅｃｔｉｏｎｏｆｓｐｅｅｃｈｓｉｇｎａｌｉｓｉｍｐｏｒｔａｎｔｉｎｍａｎｙａｒｅａｓｏｆｓｐｅｅｃｈｐｒｏｃｅｓｓｉｎｇｔｅｃｈｎｏｌｏｇｙ，ｓｕｃｈａｓｓｐｅｅｃｈｅｎｈａｎｃｅｍｅｎｔ，ｓｐｅｅｃｈｒ...

A Prediction Method of Trend-Type Capacity Index Based on Recurrent Neural Network

Due to the increase in the types of business and equipment in telecommunications companies,the performance index data collected in the operation and maintenance process varies greatly.The diversity of index data makes it very difficult to perform high-precision capacity prediction.In order to improve the forecasting efficiency of related indexes,this paper designs a classification method of capacity index data,which divides the capacity index data into trend type,periodic type and irregular type.Then for the prediction of trend data,it proposes a capacity index prediction model based on Recurrent Neural Network(RNN),denoted as RNN-LSTM-LSTM.This model includes a basic RNN,two Long Short-Term Memory(LSTM)networks and two Fully Connected layers.The experimental results show that,compared with the traditional Holt-Winters,Autoregressive Integrated Moving Average(ARIMA)and Back Propagation(BP)neural network prediction model,the mean square error(MSE)of the proposed RNN-LSTM-LSTM model are reduced by 11.82%and 20.34%on the order storage and data migration,which has greatly improved the efficiency of trend-type capacity index prediction.

基于RF-RNN模型的DNS隐蔽信道检测方法

为提高检测隐蔽信道的灵敏度,提出一种基于随机森林(Random Forest,RF)和循环神经网络(Recurrent Neural Network,RNN)的域名系统(Domain Name System,DNS)隐蔽信道检测方法。该方法采用域名检测作为主要手段,使用RF模型对域名进行分类,通过深度学习方法挖掘更高阶的特征表示。实验结果表明,与单一模型相比,该方法在检测准确性和健壮性方面均取得了显著提升。

Hyperparameter Tuning for Deep Neural Networks Based Optimization Algorithm

For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.

Short-TermWind Power Prediction Based on Combinatorial Neural Networks

Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.

基于时空记忆解耦RNN的雷暴预测方法

使用循环神经网络进行雷暴的外推预测,利用气象雷达历史反射率因子资料给出未来一小时的雷暴预测结果。网络的核心是时空长短时记忆(spatiotemporal long short-term memory,ST-LSTM)单元,加入了记忆解耦结构以分离时间记忆和空间记忆状态。在中国香港天文台(Hong Kong Observatorg,HKO)的HKO-7数据集的基础上筛选雷暴数据,构建训练及测试数据集。将有记忆解耦结构、无记忆解耦结构的ST-LSTM网络和MIM(memory in memory)网络以及传统的单体质心法进行比较。预报评分因子数值比较和个例分析检验结果表明,预测神经网络在探测成功概率、临界成功指数上均高于单体质心法,虚警率低于单体质心法。加入记忆解耦结构的网络预报因子评分高于ST-LSTM网络和MIM网络,雷暴回波外推的预测效果更好,尤其是强回波的预测效果更好。

Multimodal emotion recognition based on deep neural network

In order to increase the accuracy rate of emotion recognition in voiceand video,the mixed convolutional neural network(CNN)and recurrent neural network(RNN)ae used to encode and integrate the two information sources.For the audio signals,several frequency bands as well as some energy functions are extacted as low-level features by using a sophisticated audio technique,and then they are encoded w it a one-dimensional(I D)convolutional neural network to abstact high-level features.Finally,tiese are fed into a recurrent neural network for te sake of capturing dynamic tone changes in a temporal dimensionality.As a contrast,a two-dimensional(2D)convolutional neural network and a similar RNN are used to capture dynamic facial appearance changes of temporal sequences.The method was used in te Chinese Natral Audio-'Visual Emotion Database in te Chinese Conference on Pattern Recognition(CCPR)in2016.Experimental results demonstrate that te classification average precision of the proposed metiod is41.15%,which is increased by16.62%compaed with te baseline algorithm offered by the CCPR in2016.It is proved ta t te proposed method has higher accuracy in te identification of emotional information.

基于注意力机制RNN模型的癫痫患者脑电信号识别方法

针对癫痫患者脑电信号(electroencephalogram,EEG)数据识别提出了一种基于注意力机制的RNN(recurrent neural networks)模型.传统EEG特征分析耗时巨大且过度依赖专家经验,极大限制了脑活动识别方法的应用推广.因此,提出一种新的EEG识别方法以解决上述问题.首先对癫痫患者EEG的基本特征进行分析,进而采用基于注意力机制RNN模型消除各种干扰信号,利用XGBoost分类器识别EEG数据的类别,达到自动细化识别原始EEG的目的,最后在公共EEG数据集上进行大量实验,验证所提方法对EEG识别的准确性.实验结果表明,与一些成熟的EEG识别方法相比,本文所提方法在识别精度上有了进一步提升.

BI-DIRECTIONAL COHEN-GROSSBERG NEURAL NETWORK WITH DISCRETE TIME

Discrete-time version of the bi-directional Cohen-Grossberg neural network is stud-ied in this paper. Some sufficient conditions are obtained to ensure the global exponen-tial stability of such networks with discrete time based on Lyapunov method. These results do not require the symmetry of the connection matrix and the monotonicity, boundedness and differentiability of the activation function.

Detection of Abnormal Network Traffic Using Bidirectional Long Short-Term Memory

Nowadays,web systems and servers are constantly at great risk from cyberattacks.This paper proposes a novel approach to detecting abnormal network traffic using a bidirectional long short-term memory(LSTM)network in combination with the ensemble learning technique.First,the binary classification module was used to detect the current abnormal flow.Then,the abnormal flows were fed into the multilayer classification module to identify the specific type of flow.In this research,a deep learning bidirectional LSTM model,in combination with the convolutional neural network and attention technique,was deployed to identify a specific attack.To solve the real-time intrusion-detecting problem,a stacking ensemble-learning model was deployed to detect abnormal intrusion before being transferred to the attack classification module.The class-weight technique was applied to overcome the data imbalance between the attack layers.The results showed that our approach gained good performance and the F1 accuracy on the CICIDS2017 data set reached 99.97%,which is higher than the results obtained in other research.

Deep-fake video detection approaches using convolutional–recurrent neural networks

Deep-Fake is an emerging technology used in synthetic media which manipulates individuals in existing images and videos with someone else’s likeness.This paper presents the comparative study of different deep neural networks employed for Deep-Fake video detection.In the model,the features from the training data are extracted with the intended Convolution Neural Network model to form feature vectors which are further analysed using a dense layer,a Long Short-Term Memoryand Gated Recurrent by adopting transfer learning with fine tuning for training the models.The model is evaluated to detect Artificial Intelligence based Deep fakes images and videos using benchmark datasets.Comparative analysis shows that the detections are majorly biased towards domain of the dataset but there is a noteworthy improvement in the model performance parameters by using Transfer Learning whereas Convolutional-Recurrent Neural Network has benefits in sequence detection.

Delay-Dependent Exponential Stability Criterion for BAM Neural Networks with Time-Varying Delays

By employing the Lyapunov stability theory and linear matrix inequality（LMI）technique,delay-dependent stability criterion is derived to ensure the exponential stability of bi-directional associative memory（BAM）neural networks with time-varying delays.The proposed condition can be checked easily by LMI control toolbox in Matlab.A numerical example is given to demonstrate the effectiveness of our results.