A gated recurrent unit model to predict Poisson’s ratio using deep learning

Static Poisson’s ratio(vs)is crucial for determining geomechanical properties in petroleum applications,namely sand production.Some models have been used to predict vs;however,the published models were limited to specific data ranges with an average absolute percentage relative error(AAPRE)of more than 10%.The published gated recurrent unit(GRU)models do not consider trend analysis to show physical behaviors.In this study,we aim to develop a GRU model using trend analysis and three inputs for predicting n s based on a broad range of data,n s(value of 0.1627-0.4492),bulk formation density(RHOB)(0.315-2.994 g/mL),compressional time(DTc)(44.43-186.9 μs/ft),and shear time(DTs)(72.9-341.2μ s/ft).The GRU model was evaluated using different approaches,including statistical error an-alyses.The GRU model showed the proper trends,and the model data ranges were wider than previous ones.The GRU model has the largest correlation coefficient(R)of 0.967 and the lowest AAPRE,average percent relative error(APRE),root mean square error(RMSE),and standard deviation(SD)of 3.228%,1.054%,4.389,and 0.013,respectively,compared to other models.The GRU model has a high accuracy for the different datasets:training,validation,testing,and the whole datasets with R and AAPRE values were 0.981 and 2.601%,0.966 and 3.274%,0.967 and 3.228%,and 0.977 and 2.861%,respectively.The group error analyses of all inputs show that the GRU model has less than 5% AAPRE for all input ranges,which is superior to other models that have different AAPRE values of more than 10% at various ranges of inputs.

Aerial target threat assessment based on gated recurrent unit and self-attention mechanism

Aerial threat assessment is a crucial link in modern air combat, whose result counts a great deal for commanders to make decisions. With the consideration that the existing threat assessment methods have difficulties in dealing with high dimensional time series target data, a threat assessment method based on self-attention mechanism and gated recurrent unit(SAGRU) is proposed. Firstly, a threat feature system including air combat situations and capability features is established. Moreover, a data augmentation process based on fractional Fourier transform(FRFT) is applied to extract more valuable information from time series situation features. Furthermore, aiming to capture key characteristics of battlefield evolution, a bidirectional GRU and SA mechanisms are designed for enhanced features.Subsequently, after the concatenation of the processed air combat situation and capability features, the target threat level will be predicted by fully connected neural layers and the softmax classifier. Finally, in order to validate this model, an air combat dataset generated by a combat simulation system is introduced for model training and testing. The comparison experiments show the proposed model has structural rationality and can perform threat assessment faster and more accurately than the other existing models based on deep learning.

A HybridManufacturing ProcessMonitoringMethod Using Stacked Gated Recurrent Unit and Random Forest

This study proposed a new real-time manufacturing process monitoring method to monitor and detect process shifts in manufacturing operations.Since real-time production process monitoring is critical in today’s smart manufacturing.The more robust the monitoring model,the more reliable a process is to be under control.In the past,many researchers have developed real-time monitoring methods to detect process shifts early.However,thesemethods have limitations in detecting process shifts as quickly as possible and handling various data volumes and varieties.In this paper,a robust monitoring model combining Gated Recurrent Unit(GRU)and Random Forest(RF)with Real-Time Contrast(RTC)called GRU-RF-RTC was proposed to detect process shifts rapidly.The effectiveness of the proposed GRU-RF-RTC model is first evaluated using multivariate normal and nonnormal distribution datasets.Then,to prove the applicability of the proposed model in a realmanufacturing setting,the model was evaluated using real-world normal and non-normal problems.The results demonstrate that the proposed GRU-RF-RTC outperforms other methods in detecting process shifts quickly with the lowest average out-of-control run length(ARL1)in all synthesis and real-world problems under normal and non-normal cases.The experiment results on real-world problems highlight the significance of the proposed GRU-RF-RTC model in modern manufacturing process monitoring applications.The result reveals that the proposed method improves the shift detection capability by 42.14%in normal and 43.64%in gamma distribution problems.

Radar Quantitative Precipitation Estimation Based on the Gated Recurrent Unit Neural Network and Echo-Top Data

The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). In this study, we propose a new method, GRU_Z-ET, by introducing Z and ET as two independent variables into the GRU neural network to conduct the quantitative single-polarization radar precipitation estimation. The performance of GRU_Z-ET is compared with that of the other three methods in three heavy rainfall cases in China during 2018, namely, the traditional Z-R relationship(Z=300R1.4), the optimal Z-R relationship(Z=79R1.68) and the GRU neural network with only Z as the independent input variable(GRU_Z). The results indicate that the GRU_Z-ET performs the best, while the traditional Z-R relationship performs the worst. The performances of the rest two methods are similar.To further evaluate the performance of the GRU_Z-ET, 200 rainfall events with 21882 total samples during May–July of 2018 are used for statistical analysis. Results demonstrate that the spatial correlation coefficients, threat scores and probability of detection between the observed and estimated precipitation are the largest for the GRU_Z-ET and the smallest for the traditional Z-R relationship, and the root mean square error is just the opposite. In addition, these statistics of GRU_Z are similar to those of optimal Z-R relationship. Thus, it can be concluded that the performance of the GRU_ZET is the best in the four methods for the quantitative precipitation estimation.

Predicting Wavelet-Transformed Stock Prices Using a Vanishing Gradient Resilient Optimized Gated Recurrent Unit with a Time Lag

The development of accurate prediction models continues to be highly beneficial in myriad disciplines. Deep learning models have performed well in stock price prediction and give high accuracy. However, these models are largely affected by the vanishing gradient problem escalated by some activation functions. This study proposes the use of the Vanishing Gradient Resilient Optimized Gated Recurrent Unit (OGRU) model with a scaled mean Approximation Coefficient (AC) time lag which should counter slow convergence, vanishing gradient and large error metrics. This study employed the Rectified Linear Unit (ReLU), Hyperbolic Tangent (Tanh), Sigmoid and Exponential Linear Unit (ELU) activation functions. Real-life datasets including the daily Apple and 5-minute Netflix closing stock prices were used, and they were decomposed using the Stationary Wavelet Transform (SWT). The decomposed series formed a decomposed data model which was compared to an undecomposed data model with similar hyperparameters and different default lags. The Apple daily dataset performed well with a Default_1 lag, using an undecomposed data model and the ReLU, attaining 0.01312, 0.00854 and 3.67 minutes for RMSE, MAE and runtime. The Netflix data performed best with the MeanAC_42 lag, using decomposed data model and the ELU achieving 0.00620, 0.00487 and 3.01 minutes for the same metrics.

Real-time analysis and prediction of shield cutterhead torque using optimized gated recurrent unit neural network

An accurate prediction of earth pressure balance(EPB)shield moving performance is important to ensure the safety tunnel excavation.A hybrid model is developed based on the particle swarm optimization(PSO)and gated recurrent unit(GRU)neural network.PSO is utilized to assign the optimal hyperparameters of GRU neural network.There are mainly four steps:data collection and processing,hybrid model establishment,model performance evaluation and correlation analysis.The developed model provides an alternative to tackle with time-series data of tunnel project.Apart from that,a novel framework about model application is performed to provide guidelines in practice.A tunnel project is utilized to evaluate the performance of proposed hybrid model.Results indicate that geological and construction variables are significant to the model performance.Correlation analysis shows that construction variables(main thrust and foam liquid volume)display the highest correlation with the cutterhead torque(CHT).This work provides a feasible and applicable alternative way to estimate the performance of shield tunneling.

Multi-Scale Convolutional Gated Recurrent Unit Networks for Tool Wear Prediction in Smart Manufacturing

As an integrated application of modern information technologies and artificial intelligence,Prognostic and Health Management(PHM)is important for machine health monitoring.Prediction of tool wear is one of the symbolic applications of PHM technology in modern manufacturing systems and industry.In this paper,a multi-scale Convolutional Gated Recurrent Unit network(MCGRU)is proposed to address raw sensory data for tool wear prediction.At the bottom of MCGRU,six parallel and independent branches with different kernel sizes are designed to form a multi-scale convolutional neural network,which augments the adaptability to features of different time scales.These features of different scales extracted from raw data are then fed into a Deep Gated Recurrent Unit network to capture long-term dependencies and learn significant representations.At the top of the MCGRU,a fully connected layer and a regression layer are built for cutting tool wear prediction.Two case studies are performed to verify the capability and effectiveness of the proposed MCGRU network and results show that MCGRU outperforms several state-of-the-art baseline models.

Micro-seismic Event Detection of Hot Dry Rock based on the Gated Recurrent Unit Model and a Support Vector Machine

Micro-seismic monitoring is one of the most critical technologies that guide hydraulic fracturing in hot dry rock resource development. Micro-seismic monitoring requires high precision detection of micro-seismic events with a low signal-to-noise ratio. Because of this requirement, we propose a recurrent neural network model named gated recurrent unit and support vector machine(GRU;VM). The proposed model ensures high accuracy while reducing the parameter number and hardware requirement in the training process. Since micro-seismic events in hot dry rock produce large wave amplitudes and strong vibrations, it is difficult to reverse the onset of each individual event. In this study, we utilize a support vector machine(SVM) as a classifier to improve the micro-seismic event detection accuracy. To validate the methodology, we compare the simulation results of the short-term-average to the long-term-average(STA/LTA) method with GRU;VM method by using hot dry rock micro-seismic event data in Qinghai Province, China. Our proposed method has an accuracy of about 95% for identifying micro-seismic events with low signal-to-noise ratios. By ignoring smaller micro-seismic events, the detection procedure can be processed more efficiently, which is able to provide a real-time observation on the types of hydraulic fracturing in the reservoirs.

Multi-Scale Fusion Model Based on Gated Recurrent Unit for Enhancing Prediction Accuracy of State-of-Charge in Battery Energy Storage Systems

Accurate prediction of the state-of-charge(SOC)of battery energy storage system(BESS)is critical for its safety and lifespan in electric vehicles.To overcome the imbalance of existing methods between multi-scale feature fusion and global feature extraction,this paper introduces a novel multi-scale fusion(MSF)model based on gated recurrent unit(GRU),which is specifically designed for complex multi-step SOC prediction in practical BESSs.Pearson correlation analysis is first employed to identify SOC-related parameters.These parameters are then input into a multi-layer GRU for point-wise feature extraction.Concurrently,the parameters undergo patching before entering a dual-stage multi-layer GRU,thus enabling the model to capture nuanced information across varying time intervals.Ultimately,by means of adaptive weight fusion and a fully connected network,multi-step SOC predictions are rendered.Following extensive validation over multiple days,it is illustrated that the proposed model achieves an absolute error of less than 1.5%in real-time SOC prediction.

Short-term load forecasting model based on gated recurrent unit and multi-head attention

Short-term load forecasting(STLF)plays a crucial role in the smart grid.However,it is challenging to capture the long-time dependence and the nonlinear relationship due to the comprehensive fluctuations of the electrical load.In this paper,an STLF model based on gated recurrent unit and multi-head attention(GRU-MA)is proposed to address the aforementioned problems.The proposed model accommodates the time series and nonlinear relationship of load data through gated recurrent unit(GRU)and exploits multi-head attention(MA)to learn the decisive features and long-term dependencies.Additionally,the proposed model is compared with the support vector regression(SVR)model,the recurrent neural network and multi-head attention(RNN-MA)model,the long short-term memory and multi-head attention(LSTM-MA)model,the GRU model,and the temporal convolutional network(TCN)model using the public dataset of the Global Energy Forecasting Competition 2014(GEFCOM2014).The results demonstrate that the GRU-MA model has the best prediction accuracy.

A new method for the prediction of network security situations based on recurrent neural network with gated recurrent unit

Purpose-The purpose of this paper is to solve the shortage of the existing methods for the prediction of network security situations(NSS).Because the conventional methods for the prediction of NSS,such as support vector machine,particle swarm optimization,etc.,lack accuracy,robustness and efficiency,in this study,the authors propose a new method for the prediction of NSS based on recurrent neural network(RNN)with gated recurrent unit.Design/methodology/approach-This method extracts internal and external information features from the original time-series network data for the first time.Then,the extracted features are applied to the deep RNN model for training and validation.After iteration and optimization,the accuracy of predictions of NSS will be obtained by the well-trained model,and the model is robust for the unstable network data.Findings-Experiments on bench marked data set show that the proposed method obtains more accurate and robust prediction results than conventional models.Although the deep RNN models need more time consumption for training,they guarantee the accuracy and robustness of prediction in return for validation.Originality/value-In the prediction of NSS time-series data,the proposed internal and external information features are well described the original data,and the employment of deep RNN model will outperform the state-of-the-arts models.

Hybrid Deep Learning Model for Short-Term Wind Speed Forecasting Based on Time Series Decomposition and Gated Recurrent Unit

Accurate wind speed prediction has been becoming an indispensable technology in system security,wind energy utilization,and power grid dispatching in recent years.However,it is an arduous task to predict wind speed due to its variable and random characteristics.For the objective to enhance the performance of forecasting short-term wind speed,this work puts forward a hybrid deep learning model mixing time series decomposition algorithm and gated recurrent unit(GRU).The time series decomposition algorithm combines the following two parts:(1)the complete ensemble empirical mode decomposition with adaptive noise(CEEMDAN),and(2)wavelet packet decomposition(WPD).Firstly,the normalized wind speed time series(WSTS)are handled by CEEMDAN to gain pure fixed-frequency components and a residual signal.The WPD algorithm conducts the second-order decomposition to the first component that contains complex and high frequency signal of raw WSTS.Finally,GRU networks are established for all the relevant components of the signals,and the predicted wind speeds are obtained by superimposing the prediction of each component.Results from two case studies,adopting wind data from laboratory and wind farm,respectively,suggest that the related trend of the WSTS can be separated effectively by the proposed time series decomposition algorithm,and the accuracy of short-time wind speed prediction can be heightened significantly mixing the time series decomposition algorithm and GRU networks.

Speech Separation Algorithm Using Gated Recurrent Network Based on Microphone Array

Speech separation is an active research topic that plays an important role in numerous applications,such as speaker recognition,hearing pros-thesis,and autonomous robots.Many algorithms have been put forward to improve separation performance.However,speech separation in reverberant noisy environment is still a challenging task.To address this,a novel speech separation algorithm using gate recurrent unit(GRU)network based on microphone array has been proposed in this paper.The main aim of the proposed algorithm is to improve the separation performance and reduce the computational cost.The proposed algorithm extracts the sub-band steered response power-phase transform(SRP-PHAT)weighted by gammatone filter as the speech separation feature due to its discriminative and robust spatial position in formation.Since the GRU net work has the advantage of processing time series data with faster training speed and fewer training parameters,the GRU model is adopted to process the separation featuresof several sequential frames in the same sub-band to estimate the ideal Ratio Masking(IRM).The proposed algorithm decomposes the mixture signals into time-frequency(TF)units using gammatone filter bank in the frequency domain,and the target speech is reconstructed in the frequency domain by masking the mixture signal according to the estimated IRM.The operations of decomposing the mixture signal and reconstructing the target signal are completed in the frequency domain which can reduce the total computational cost.Experimental results demonstrate that the proposed algorithm realizes omnidirectional speech sep-aration in noisy and reverberant environments,provides good performance in terms of speech quality and intelligibility,and has the generalization capacity to reverberate.

Soil NOx Emission Prediction via Recurrent Neural Networks

This paper presents designing sequence-to-sequence recurrent neural network(RNN)architectures for a novel study to predict soil NOx emissions,driven by the imperative of understanding and mitigating environmental impact.The study utilizes data collected by the Environmental Protection Agency(EPA)to develop two distinct RNN predictive models:one built upon the long-short term memory(LSTM)and the other utilizing the gated recurrent unit(GRU).These models are fed with a combination of historical and anticipated air temperature,air moisture,and NOx emissions as inputs to forecast future NOx emissions.Both LSTM and GRU models can capture the intricate pulse patterns inherent in soil NOx emissions.Notably,the GRU model emerges as the superior performer,surpassing the LSTM model in predictive accuracy while demonstrating efficiency by necessitating less training time.Intriguingly,the investigation into varying input features reveals that relying solely on past NOx emissions as input yields satisfactory performance,highlighting the dominant influence of this factor.The study also delves into the impact of altering input series lengths and training data sizes,yielding insights into optimal configurations for enhanced model performance.Importantly,the findings promise to advance our grasp of soil NOx emission dynamics,with implications for environmental management strategies.Looking ahead,the anticipated availability of additional measurements is poised to bolster machine-learning model efficacy.Furthermore,the future study will explore physical-based RNNs,a promising avenue for deeper insights into soil NOx emission prediction.

An Improved Time Feedforward Connections Recurrent Neural Networks

Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to the strict time serial dependency,making it difficult to realize a long-term memory function.On the other hand,RNNs cells are highly complex,which will signifi-cantly increase computational complexity and cause waste of computational resources during model training.In this paper,an improved Time Feedforward Connections Recurrent Neural Networks(TFC-RNNs)model was first proposed to address the gradient issue.A parallel branch was introduced for the hidden state at time t−2 to be directly transferred to time t without the nonlinear transforma-tion at time t−1.This is effective in improving the long-term dependence of RNNs.Then,a novel cell structure named Single Gate Recurrent Unit(SGRU)was presented.This cell structure can reduce the number of parameters for RNNs cell,consequently reducing the computational complexity.Next,applying SGRU to TFC-RNNs as a new TFC-SGRU model solves the above two difficulties.Finally,the performance of our proposed TFC-SGRU was verified through sev-eral experiments in terms of long-term memory and anti-interference capabilities.Experimental results demonstrated that our proposed TFC-SGRU model can cap-ture helpful information with time step 1500 and effectively filter out the noise.The TFC-SGRU model accuracy is better than the LSTM and GRU models regarding language processing ability.

基于GRU-DRSN的双通道人体活动识别

人体活动识别(human activity recognizition, HAR)在医疗、军工、智能家居等领域有很大的应用空间。传统机器学习方法特征提取难度较大且精度不高。针对上述问题并结合传感器时序特性,提出了一种融合CBAM(convolutional block attention module)注意力机制的GRU-DRSN双通道并行模型,有效避免了传统串行模型因网络深度加深引起梯度爆炸和消失问题。同时并行结构使得两条支路具有相同的优先级,使用深度残差收缩网络(deep residual shrinkage network, DRSN)提取数据的深层空间特征,同时使用门控循环结构(gated recurrent unit, GRU)学习活动样本在时间序列上的特征,同时进行提取样本不同维度的特征,并通过CBAM模块进行特征的权重分配,最后通过Softmax层进行识别,实现了端对端的人体活动识别。使用公开数据集(wireless sensor data mining, WISDM)进行验证,模型平均精度达到了97.6%,与传统机器学习模型和前人所提神经网络模型相比,有更好的识别效果。

Prediction and scheduling of multi-energy microgrid based on BiGRU self-attention mechanism and LQPSO

To predict renewable energy sources such as solar power in microgrids more accurately,a hybrid power prediction method is presented in this paper.First,the self-attention mechanism is introduced based on a bidirectional gated recurrent neural network(BiGRU)to explore the time-series characteristics of solar power output and consider the influence of different time nodes on the prediction results.Subsequently,an improved quantum particle swarm optimization(QPSO)algorithm is proposed to optimize the hyperparameters of the combined prediction model.The final proposed LQPSO-BiGRU-self-attention hybrid model can predict solar power more effectively.In addition,considering the coordinated utilization of various energy sources such as electricity,hydrogen,and renewable energy,a multi-objective optimization model that considers both economic and environmental costs was constructed.A two-stage adaptive multi-objective quantum particle swarm optimization algorithm aided by a Lévy flight,named MO-LQPSO,was proposed for the comprehensive optimal scheduling of a multi-energy microgrid system.This algorithm effectively balances the global and local search capabilities and enhances the solution of complex nonlinear problems.The effectiveness and superiority of the proposed scheme are verified through comparative simulations.

Price prediction of power transformer materials based on CEEMD and GRU

The rapid growth of the Chinese economy has fueled the expansion of power grids.Power transformers are key equipment in power grid projects,and their price changes have a significant impact on cost control.However,the prices of power transformer materials manifest as nonsmooth and nonlinear sequences.Hence,estimating the acquisition costs of power grid projects is difficult,hindering the normal operation of power engineering construction.To more accurately predict the price of power transformer materials,this study proposes a method based on complementary ensemble empirical mode decomposition(CEEMD)and gated recurrent unit(GRU)network.First,the CEEMD decomposed the price series into multiple intrinsic mode functions(IMFs).Multiple IMFs were clustered to obtain several aggregated sequences based on the sample entropy of each IMF.Then,an empirical wavelet transform(EWT)was applied to the aggregation sequence with a large sample entropy,and the multiple subsequences obtained from the decomposition were predicted by the GRU model.The GRU model was used to directly predict the aggregation sequences with a small sample entropy.In this study,we used authentic historical pricing data for power transformer materials to validate the proposed approach.The empirical findings demonstrated the efficacy of our method across both datasets,with mean absolute percentage errors(MAPEs)of less than 1%and 3%.This approach holds a significant reference value for future research in the field of power transformer material price prediction.

BSTFNet:An Encrypted Malicious Traffic Classification Method Integrating Global Semantic and Spatiotemporal Features

While encryption technology safeguards the security of network communications,malicious traffic also uses encryption protocols to obscure its malicious behavior.To address the issues of traditional machine learning methods relying on expert experience and the insufficient representation capabilities of existing deep learning methods for encrypted malicious traffic,we propose an encrypted malicious traffic classification method that integrates global semantic features with local spatiotemporal features,called BERT-based Spatio-Temporal Features Network(BSTFNet).At the packet-level granularity,the model captures the global semantic features of packets through the attention mechanism of the Bidirectional Encoder Representations from Transformers(BERT)model.At the byte-level granularity,we initially employ the Bidirectional Gated Recurrent Unit(BiGRU)model to extract temporal features from bytes,followed by the utilization of the Text Convolutional Neural Network(TextCNN)model with multi-sized convolution kernels to extract local multi-receptive field spatial features.The fusion of features from both granularities serves as the ultimate multidimensional representation of malicious traffic.Our approach achieves accuracy and F1-score of 99.39%and 99.40%,respectively,on the publicly available USTC-TFC2016 dataset,and effectively reduces sample confusion within the Neris and Virut categories.The experimental results demonstrate that our method has outstanding representation and classification capabilities for encrypted malicious traffic.

Deep-Ensemble Learning Method for Solar Resource Assessment of Complex Terrain Landscapes

As the global demand for renewable energy grows,solar energy is gaining attention as a clean,sustainable energy source.Accurate assessment of solar energy resources is crucial for the siting and design of photovoltaic power plants.This study proposes an integrated deep learning-based photovoltaic resource assessment method.Ensemble learning and deep learning methods are fused for photovoltaic resource assessment for the first time.The proposed method combines the random forest,gated recurrent unit,and long short-term memory to effectively improve the accuracy and reliability of photovoltaic resource assessment.The proposed method has strong adaptability and high accuracy even in the photovoltaic resource assessment of complex terrain and landscape.The experimental results show that the proposed method outperforms the comparison algorithm in all evaluation indexes,indicating that the proposed method has higher accuracy and reliability in photovoltaic resource assessment with improved generalization performance traditional single algorithm.