A gated recurrent unit model to predict Poisson’s ratio using deep learning

Static Poisson’s ratio(vs)is crucial for determining geomechanical properties in petroleum applications,namely sand production.Some models have been used to predict vs;however,the published models were limited to specific data ranges with an average absolute percentage relative error(AAPRE)of more than 10%.The published gated recurrent unit(GRU)models do not consider trend analysis to show physical behaviors.In this study,we aim to develop a GRU model using trend analysis and three inputs for predicting n s based on a broad range of data,n s(value of 0.1627-0.4492),bulk formation density(RHOB)(0.315-2.994 g/mL),compressional time(DTc)(44.43-186.9 μs/ft),and shear time(DTs)(72.9-341.2μ s/ft).The GRU model was evaluated using different approaches,including statistical error an-alyses.The GRU model showed the proper trends,and the model data ranges were wider than previous ones.The GRU model has the largest correlation coefficient(R)of 0.967 and the lowest AAPRE,average percent relative error(APRE),root mean square error(RMSE),and standard deviation(SD)of 3.228%,1.054%,4.389,and 0.013,respectively,compared to other models.The GRU model has a high accuracy for the different datasets:training,validation,testing,and the whole datasets with R and AAPRE values were 0.981 and 2.601%,0.966 and 3.274%,0.967 and 3.228%,and 0.977 and 2.861%,respectively.The group error analyses of all inputs show that the GRU model has less than 5% AAPRE for all input ranges,which is superior to other models that have different AAPRE values of more than 10% at various ranges of inputs.

Aerial target threat assessment based on gated recurrent unit and self-attention mechanism

Aerial threat assessment is a crucial link in modern air combat, whose result counts a great deal for commanders to make decisions. With the consideration that the existing threat assessment methods have difficulties in dealing with high dimensional time series target data, a threat assessment method based on self-attention mechanism and gated recurrent unit(SAGRU) is proposed. Firstly, a threat feature system including air combat situations and capability features is established. Moreover, a data augmentation process based on fractional Fourier transform(FRFT) is applied to extract more valuable information from time series situation features. Furthermore, aiming to capture key characteristics of battlefield evolution, a bidirectional GRU and SA mechanisms are designed for enhanced features.Subsequently, after the concatenation of the processed air combat situation and capability features, the target threat level will be predicted by fully connected neural layers and the softmax classifier. Finally, in order to validate this model, an air combat dataset generated by a combat simulation system is introduced for model training and testing. The comparison experiments show the proposed model has structural rationality and can perform threat assessment faster and more accurately than the other existing models based on deep learning.

A HybridManufacturing ProcessMonitoringMethod Using Stacked Gated Recurrent Unit and Random Forest

This study proposed a new real-time manufacturing process monitoring method to monitor and detect process shifts in manufacturing operations.Since real-time production process monitoring is critical in today’s smart manufacturing.The more robust the monitoring model,the more reliable a process is to be under control.In the past,many researchers have developed real-time monitoring methods to detect process shifts early.However,thesemethods have limitations in detecting process shifts as quickly as possible and handling various data volumes and varieties.In this paper,a robust monitoring model combining Gated Recurrent Unit(GRU)and Random Forest(RF)with Real-Time Contrast(RTC)called GRU-RF-RTC was proposed to detect process shifts rapidly.The effectiveness of the proposed GRU-RF-RTC model is first evaluated using multivariate normal and nonnormal distribution datasets.Then,to prove the applicability of the proposed model in a realmanufacturing setting,the model was evaluated using real-world normal and non-normal problems.The results demonstrate that the proposed GRU-RF-RTC outperforms other methods in detecting process shifts quickly with the lowest average out-of-control run length(ARL1)in all synthesis and real-world problems under normal and non-normal cases.The experiment results on real-world problems highlight the significance of the proposed GRU-RF-RTC model in modern manufacturing process monitoring applications.The result reveals that the proposed method improves the shift detection capability by 42.14%in normal and 43.64%in gamma distribution problems.

Real-time analysis and prediction of shield cutterhead torque using optimized gated recurrent unit neural network

An accurate prediction of earth pressure balance(EPB)shield moving performance is important to ensure the safety tunnel excavation.A hybrid model is developed based on the particle swarm optimization(PSO)and gated recurrent unit(GRU)neural network.PSO is utilized to assign the optimal hyperparameters of GRU neural network.There are mainly four steps:data collection and processing,hybrid model establishment,model performance evaluation and correlation analysis.The developed model provides an alternative to tackle with time-series data of tunnel project.Apart from that,a novel framework about model application is performed to provide guidelines in practice.A tunnel project is utilized to evaluate the performance of proposed hybrid model.Results indicate that geological and construction variables are significant to the model performance.Correlation analysis shows that construction variables(main thrust and foam liquid volume)display the highest correlation with the cutterhead torque(CHT).This work provides a feasible and applicable alternative way to estimate the performance of shield tunneling.

Multi-Scale Convolutional Gated Recurrent Unit Networks for Tool Wear Prediction in Smart Manufacturing

As an integrated application of modern information technologies and artificial intelligence,Prognostic and Health Management(PHM)is important for machine health monitoring.Prediction of tool wear is one of the symbolic applications of PHM technology in modern manufacturing systems and industry.In this paper,a multi-scale Convolutional Gated Recurrent Unit network(MCGRU)is proposed to address raw sensory data for tool wear prediction.At the bottom of MCGRU,six parallel and independent branches with different kernel sizes are designed to form a multi-scale convolutional neural network,which augments the adaptability to features of different time scales.These features of different scales extracted from raw data are then fed into a Deep Gated Recurrent Unit network to capture long-term dependencies and learn significant representations.At the top of the MCGRU,a fully connected layer and a regression layer are built for cutting tool wear prediction.Two case studies are performed to verify the capability and effectiveness of the proposed MCGRU network and results show that MCGRU outperforms several state-of-the-art baseline models.

Radar Quantitative Precipitation Estimation Based on the Gated Recurrent Unit Neural Network and Echo-Top Data

The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). In this study, we propose a new method, GRU_Z-ET, by introducing Z and ET as two independent variables into the GRU neural network to conduct the quantitative single-polarization radar precipitation estimation. The performance of GRU_Z-ET is compared with that of the other three methods in three heavy rainfall cases in China during 2018, namely, the traditional Z-R relationship(Z=300R1.4), the optimal Z-R relationship(Z=79R1.68) and the GRU neural network with only Z as the independent input variable(GRU_Z). The results indicate that the GRU_Z-ET performs the best, while the traditional Z-R relationship performs the worst. The performances of the rest two methods are similar.To further evaluate the performance of the GRU_Z-ET, 200 rainfall events with 21882 total samples during May–July of 2018 are used for statistical analysis. Results demonstrate that the spatial correlation coefficients, threat scores and probability of detection between the observed and estimated precipitation are the largest for the GRU_Z-ET and the smallest for the traditional Z-R relationship, and the root mean square error is just the opposite. In addition, these statistics of GRU_Z are similar to those of optimal Z-R relationship. Thus, it can be concluded that the performance of the GRU_ZET is the best in the four methods for the quantitative precipitation estimation.

Gated recurrent unit model for a sequence tagging problem

Combinatory categorial grammer(CCG) supertagging is an important subtask that takes place before full parsing and can benefit many natural language processing(NLP) tasks like question answering and machine translation. CCG supertagging can be regarded as a sequence labeling problem that remains a challenging problem where each word is assigned to a CCG lexical category and the number of the probably associated CCG supertags to each word is large. To address this, recently recurrent neural networks(RNNs), as extremely powerful sequential models, have been proposed for CCG supertagging and achieved good performances. In this paper, a variant of recurrent networks is proposed whose design makes it much easier to train and memorize information for long range dependencies based on gated recurrent units(GRUs), which have been recently introduced on some but not all tasks. Results of the experiments revealed the effectiveness of the proposed method on the CCGBank datasets and show that the model has comparable accuracy with the previously proposed models for CCG supertagging.

Micro-seismic Event Detection of Hot Dry Rock based on the Gated Recurrent Unit Model and a Support Vector Machine

Micro-seismic monitoring is one of the most critical technologies that guide hydraulic fracturing in hot dry rock resource development. Micro-seismic monitoring requires high precision detection of micro-seismic events with a low signal-to-noise ratio. Because of this requirement, we propose a recurrent neural network model named gated recurrent unit and support vector machine(GRU;VM). The proposed model ensures high accuracy while reducing the parameter number and hardware requirement in the training process. Since micro-seismic events in hot dry rock produce large wave amplitudes and strong vibrations, it is difficult to reverse the onset of each individual event. In this study, we utilize a support vector machine(SVM) as a classifier to improve the micro-seismic event detection accuracy. To validate the methodology, we compare the simulation results of the short-term-average to the long-term-average(STA/LTA) method with GRU;VM method by using hot dry rock micro-seismic event data in Qinghai Province, China. Our proposed method has an accuracy of about 95% for identifying micro-seismic events with low signal-to-noise ratios. By ignoring smaller micro-seismic events, the detection procedure can be processed more efficiently, which is able to provide a real-time observation on the types of hydraulic fracturing in the reservoirs.

Using Hybrid Penalty and Gated Linear Units to Improve Wasserstein Generative Adversarial Networks for Single-Channel Speech Enhancement

Recently,speech enhancement methods based on Generative Adversarial Networks have achieved good performance in time-domain noisy signals.However,the training of Generative Adversarial Networks has such problems as convergence difficulty,model collapse,etc.In this work,an end-to-end speech enhancement model based on Wasserstein Generative Adversarial Networks is proposed,and some improvements have been made in order to get faster convergence speed and better generated speech quality.Specifically,in the generator coding part,each convolution layer adopts different convolution kernel sizes to conduct convolution operations for obtaining speech coding information from multiple scales;a gated linear unit is introduced to alleviate the vanishing gradient problem with the increase of network depth;the gradient penalty of the discriminator is replaced with spectral normalization to accelerate the convergence rate of themodel;a hybrid penalty termcomposed of L1 regularization and a scale-invariant signal-to-distortion ratio is introduced into the loss function of the generator to improve the quality of generated speech.The experimental results on both TIMIT corpus and Tibetan corpus show that the proposed model improves the speech quality significantly and accelerates the convergence speed of the model.

Turnout fault prediction method based on gated recurrent units model

Turnout is one of the important signal infrastructure equipment,which will directly affect the safety and efficiency of driving.Base on analysis of the power curve of the turnout,we extract and select the time domain and Haar wavelet transform characteristics of the curve firstly.Then the correlation between the degradation state and the fault state is established by using the clustering algorithm and the Pearson correlation coefficient.Finally,the convolutional neural network(CNN)and the gated recurrent unit(GRU)are used to establish the state prediction model of the turnout to realize the failure prediction.The CNN can directly extract features from the original data of the turnout and reduce the dimension,which simplifies the prediction process.Due to its unique gate structure and time series processing features,GRU has certain advantages over the traditional forecasting methods in terms of prediction accuracy and time.The experimental results show that the accuracy of prediction can reach 94.2%when the feature matrix adopts 40-dimensional input and iterates 50 times.

Predicting Wavelet-Transformed Stock Prices Using a Vanishing Gradient Resilient Optimized Gated Recurrent Unit with a Time Lag

The development of accurate prediction models continues to be highly beneficial in myriad disciplines. Deep learning models have performed well in stock price prediction and give high accuracy. However, these models are largely affected by the vanishing gradient problem escalated by some activation functions. This study proposes the use of the Vanishing Gradient Resilient Optimized Gated Recurrent Unit (OGRU) model with a scaled mean Approximation Coefficient (AC) time lag which should counter slow convergence, vanishing gradient and large error metrics. This study employed the Rectified Linear Unit (ReLU), Hyperbolic Tangent (Tanh), Sigmoid and Exponential Linear Unit (ELU) activation functions. Real-life datasets including the daily Apple and 5-minute Netflix closing stock prices were used, and they were decomposed using the Stationary Wavelet Transform (SWT). The decomposed series formed a decomposed data model which was compared to an undecomposed data model with similar hyperparameters and different default lags. The Apple daily dataset performed well with a Default_1 lag, using an undecomposed data model and the ReLU, attaining 0.01312, 0.00854 and 3.67 minutes for RMSE, MAE and runtime. The Netflix data performed best with the MeanAC_42 lag, using decomposed data model and the ELU achieving 0.00620, 0.00487 and 3.01 minutes for the same metrics.

融合CNN-BiGRU和注意力机制的网络入侵检测模型

为提高网络入侵检测模型特征提取能力和分类准确率,提出了一种融合双向门控循环单元(CNN-BiGRU)和注意力机制的网络入侵检测模型.使用CNN有效提取流量数据集中的非线性特征;双向门控循环单元(BiGRU)提取数据集中的时序特征,最后融合注意力机制对不同类型流量数据通过加权的方式进行重要程度的区分,从而整体提高该模型特征提取与分类的性能.实验结果表明:其整体精确率比双向长短期记忆网络(BiLSTM)模型提升了2.25%.K折交叉验证结果表明:该模型泛化性能良好,避免了过拟合现象的发生,印证了该模型的有效性与合理性.

Speech Separation Algorithm Using Gated Recurrent Network Based on Microphone Array

Speech separation is an active research topic that plays an important role in numerous applications,such as speaker recognition,hearing pros-thesis,and autonomous robots.Many algorithms have been put forward to improve separation performance.However,speech separation in reverberant noisy environment is still a challenging task.To address this,a novel speech separation algorithm using gate recurrent unit(GRU)network based on microphone array has been proposed in this paper.The main aim of the proposed algorithm is to improve the separation performance and reduce the computational cost.The proposed algorithm extracts the sub-band steered response power-phase transform(SRP-PHAT)weighted by gammatone filter as the speech separation feature due to its discriminative and robust spatial position in formation.Since the GRU net work has the advantage of processing time series data with faster training speed and fewer training parameters,the GRU model is adopted to process the separation featuresof several sequential frames in the same sub-band to estimate the ideal Ratio Masking(IRM).The proposed algorithm decomposes the mixture signals into time-frequency(TF)units using gammatone filter bank in the frequency domain,and the target speech is reconstructed in the frequency domain by masking the mixture signal according to the estimated IRM.The operations of decomposing the mixture signal and reconstructing the target signal are completed in the frequency domain which can reduce the total computational cost.Experimental results demonstrate that the proposed algorithm realizes omnidirectional speech sep-aration in noisy and reverberant environments,provides good performance in terms of speech quality and intelligibility,and has the generalization capacity to reverberate.

基于KPCA-CNN-DBiGRU模型的短期负荷预测方法

本文针对已有神经网络模型在短期负荷预测中输入维度过高、预测误差较大等问题,提出了一种结合核主成分分析、卷积神经网络和深度双向门控循环单元的短期负荷预测方法。先运用核主成分分析法对原始高维输入变量进行降维,再通过卷积深度双向门控循环单元网络模型进行负荷预测。以第九届全国电工数学建模竞赛试题A题中的负荷数据作为实际算例,结果表明所提方法较降维之前预测误差大大降低,与已有预测方法相比也有大幅的误差降低。

基于特征选择及ISSA-CNN-BiGRU的短期风功率预测

针对风电功率随机性大、平稳性低,以及直接输入预测模型往往难以取得较高精度等问题,提出了一种基于特征选择及改进麻雀搜索算法(ISSA)优化卷积神经网络-双向门控循环单元(CNN-BiGRU)的短期风电功率预测方法。首先,利用变分模态分解(VMD)将原始功率分解为一组包含不同信息的子分量,以降低原始功率序列的非平稳性,提升可预测性,同时通过观察中心频率方式确定模态分解数。其次,对每一分量采用随机森林(RF)特征重要度的方法进行特征选择,从风速、风向、温度、空气密度等气象特征因素中,选取对各个分量预测贡献度较高的影响因素组成输入特征向量。然后,建立各分量的CNN-BiGRU预测模型,针对神经网络算法参数难调、手动配置参数随机性大的问题,利用ISSA对模型超参数寻优,自适应搜寻最优参数组合。最后,叠加各分量的预测值,得到最终的预测结果。以中国内蒙古某风电场实际数据进行仿真实验,与多种单一及组合预测方法进行对比,结果表明,本文所提方法相比于其他方法具有更高的预测精度,其平均绝对百分比误差值达到2.644 0%;在其他4个数据集上进行的模型准确性及泛化性验证结果显示,模型平均绝对百分比误差值分别为4.385 3%、3.174 9%、1.576 1%和1.358 8%,均保持在5.000 0%以内,证明本文所提方法具有较好的预测精度及泛化能力。

基于新型相似日选取和VMD-NGO-BiGRU的短期光伏功率预测

光伏功率预测在现代电力系统调度和运行中起着重要作用.针对光伏发电功率的多变性和复杂性,提出了一种基于新型相似日选取和北方苍鹰算法(Northern Goshawk Optimization,NGO)优化双向门控循环单元(Bidirectional Gated Recurrent Unit,BiGRU)的短期光伏功率预测方法.首先,利用斯皮尔曼相关系数选取主要气象因子,通过变分模态分解(Variational Mode Decomposition,VMD)将原始光伏功率和最大气象因子分解重构为一系列子信号.其次,通过构建新的评价指标筛选出相似日数据集,利用一组BiGRU建立以相似日子信号为网络输入的深度学习模型,并利用NGO对每个BiGRU网络的超参数进行有效优化.最后,对各子信号的预测结果进行综合,得到最终的光伏功率预测值.仿真结果表明,所提混合深度学习方法在预测精度和计算效率方面均优于其他方法.

基于CNN-GRU-ISSA-XGBoost的短期光伏功率预测

针对光伏功率随机性及波动性大,单一预测模型往往难以准确分析历史数据波动规律,从而导致预测精度不高的问题,提出一种基于卷积神经网络-门控循环单元(CNN-GRU)和改进麻雀搜索算法(ISSA)优化的极限梯度提升(XGBoost)模型的短期光伏功率预测组合模型.首先去除历史数据中的异常值并对其进行归一化处理,利用主成分分析法(PCA)进行特征选取,以便更好地识别影响光伏功率的关键因素.然后采用CNN网络提取数据的空间特征,再经过GRU网络提取时间特征,针对XGBoost模型手动配置参数困难、随机性大的问题,利用ISSA对模型超参数寻优.最后对两种方法预测的结果用误差倒数法减小误差的同时对权重进行更新,得到新的预测值,从而完成对光伏功率的预测.实验结果表明,所提出的CNN-GRU-ISSA-XGBoost组合模型具有更强的适应性和更高的精度.

基于双层注意力和深度自编码器的时间序列异常检测模型

目前时间序列通常具有弱周期性以及高度复杂的相关性特征,传统的时间序列异常检测方法难以检测此类异常。针对这一问题,提出了一种新的无监督时间序列异常检测模型(DA-CBG-AE)。首先,使用新型滑动窗口方法,针对时间序列周期性设置滑动窗口大小;其次,采用卷积神经网络提取时间序列高维度空间特征;然后,提出具有堆叠式Dropout双向门循环单元网络作为自编码器的基本结构,从而捕捉时间序列的相关性特征;最后,引入双层注意力机制,进一步提取特征,选择更加关键的时间序列,从而提高异常检测准确率。为了验证该模型的有效性,将DA-CBG-AE与6种基准模型在8个数据集上进行比较。最终的实验结果表明,DA-CBG-AE获得了最优的F1值(0.863),并且其检测性能相比最新的基准模型Tad-GAN高出25.25%。

基于ICEEMDAN和时变权重集成预测模型的变压器油中溶解气体含量预测

为了实现对变压器油中溶解气体体积分数的精确预测,同时克服仅使用单一预测模型导致预测精度及泛化能力不足的局限,提出了一种基于改进完全自适应噪声集合经验模态分解(improved complete ensemble empirical mode decomposition,ICEEMDAN)和灰色关联系数时变权重集成预测模型的变压器油中溶解气体预测方法。首先将溶解气体含量序列模态分解为一系列具有不同时间尺度的子序列。然后,使用门控循环神经网络和麻雀搜索算法优化支持向量机对各子序列进行训练,组合为一个集成预测模型;并比较不同预测方法的预测精度,计算灰色关联系数时变权重,形成各子系列的预测结果。最后将各子序列的预测结果叠加重构,得到最终预测结果。算例分析结果显示:该方法单步预测的均方根误差、平均绝对误差和相关系数分别为0.593、0.422和0.768,相比其他算法在预测精度上有明显提升,同时具有很强的泛化性能,可以为油浸式变压器内部状态监测提供依据。

基于CEEMD-SE的CNN&LSTM-GRU短期风电功率预测

为进一步提升短期风电功率的预测精度,提出了一种基于互补集合经验模态分解-样本熵(complementary ensemble empirical mode decomposition-sample entropy,CEEMD-SE)的卷积神经网络(convolutional neural network,CNN)和长短期记忆-门控循环单元(longshorttermmemory-gatedrecurrentunit,LSTM-GRU)的短期风电功率预测模型。首先,利用互补集合经验模态分解将原始风电功率序列分解为若干本征模态函数(intrinsic mode function,IMF)分量和一个残差(residual,RES)分量,利用样本熵算法将相近的分量进行重构;其次,搭建卷积神经网络和长短期记忆网络的并行网络结构,提取数据的局部特征和时序特征,并将特征融合后输入门控循环单元网络中进行学习预测;最后,通过算例进行验证,结果表明采用该模型后预测精度得到了有效提升,其均方根误差降低了15.06%、平均绝对误差降低了15.22%、决定系数提高了1.91%。