A HybridManufacturing ProcessMonitoringMethod Using Stacked Gated Recurrent Unit and Random Forest

This study proposed a new real-time manufacturing process monitoring method to monitor and detect process shifts in manufacturing operations.Since real-time production process monitoring is critical in today’s smart manufacturing.The more robust the monitoring model,the more reliable a process is to be under control.In the past,many researchers have developed real-time monitoring methods to detect process shifts early.However,thesemethods have limitations in detecting process shifts as quickly as possible and handling various data volumes and varieties.In this paper,a robust monitoring model combining Gated Recurrent Unit(GRU)and Random Forest(RF)with Real-Time Contrast(RTC)called GRU-RF-RTC was proposed to detect process shifts rapidly.The effectiveness of the proposed GRU-RF-RTC model is first evaluated using multivariate normal and nonnormal distribution datasets.Then,to prove the applicability of the proposed model in a realmanufacturing setting,the model was evaluated using real-world normal and non-normal problems.The results demonstrate that the proposed GRU-RF-RTC outperforms other methods in detecting process shifts quickly with the lowest average out-of-control run length(ARL1)in all synthesis and real-world problems under normal and non-normal cases.The experiment results on real-world problems highlight the significance of the proposed GRU-RF-RTC model in modern manufacturing process monitoring applications.The result reveals that the proposed method improves the shift detection capability by 42.14%in normal and 43.64%in gamma distribution problems.

Radar Quantitative Precipitation Estimation Based on the Gated Recurrent Unit Neural Network and Echo-Top Data

The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). In this study, we propose a new method, GRU_Z-ET, by introducing Z and ET as two independent variables into the GRU neural network to conduct the quantitative single-polarization radar precipitation estimation. The performance of GRU_Z-ET is compared with that of the other three methods in three heavy rainfall cases in China during 2018, namely, the traditional Z-R relationship(Z=300R1.4), the optimal Z-R relationship(Z=79R1.68) and the GRU neural network with only Z as the independent input variable(GRU_Z). The results indicate that the GRU_Z-ET performs the best, while the traditional Z-R relationship performs the worst. The performances of the rest two methods are similar.To further evaluate the performance of the GRU_Z-ET, 200 rainfall events with 21882 total samples during May–July of 2018 are used for statistical analysis. Results demonstrate that the spatial correlation coefficients, threat scores and probability of detection between the observed and estimated precipitation are the largest for the GRU_Z-ET and the smallest for the traditional Z-R relationship, and the root mean square error is just the opposite. In addition, these statistics of GRU_Z are similar to those of optimal Z-R relationship. Thus, it can be concluded that the performance of the GRU_ZET is the best in the four methods for the quantitative precipitation estimation.

Speech Separation Algorithm Using Gated Recurrent Network Based on Microphone Array

Speech separation is an active research topic that plays an important role in numerous applications,such as speaker recognition,hearing pros-thesis,and autonomous robots.Many algorithms have been put forward to improve separation performance.However,speech separation in reverberant noisy environment is still a challenging task.To address this,a novel speech separation algorithm using gate recurrent unit(GRU)network based on microphone array has been proposed in this paper.The main aim of the proposed algorithm is to improve the separation performance and reduce the computational cost.The proposed algorithm extracts the sub-band steered response power-phase transform(SRP-PHAT)weighted by gammatone filter as the speech separation feature due to its discriminative and robust spatial position in formation.Since the GRU net work has the advantage of processing time series data with faster training speed and fewer training parameters,the GRU model is adopted to process the separation featuresof several sequential frames in the same sub-band to estimate the ideal Ratio Masking(IRM).The proposed algorithm decomposes the mixture signals into time-frequency(TF)units using gammatone filter bank in the frequency domain,and the target speech is reconstructed in the frequency domain by masking the mixture signal according to the estimated IRM.The operations of decomposing the mixture signal and reconstructing the target signal are completed in the frequency domain which can reduce the total computational cost.Experimental results demonstrate that the proposed algorithm realizes omnidirectional speech sep-aration in noisy and reverberant environments,provides good performance in terms of speech quality and intelligibility,and has the generalization capacity to reverberate.

Real-time analysis and prediction of shield cutterhead torque using optimized gated recurrent unit neural network

An accurate prediction of earth pressure balance(EPB)shield moving performance is important to ensure the safety tunnel excavation.A hybrid model is developed based on the particle swarm optimization(PSO)and gated recurrent unit(GRU)neural network.PSO is utilized to assign the optimal hyperparameters of GRU neural network.There are mainly four steps:data collection and processing,hybrid model establishment,model performance evaluation and correlation analysis.The developed model provides an alternative to tackle with time-series data of tunnel project.Apart from that,a novel framework about model application is performed to provide guidelines in practice.A tunnel project is utilized to evaluate the performance of proposed hybrid model.Results indicate that geological and construction variables are significant to the model performance.Correlation analysis shows that construction variables(main thrust and foam liquid volume)display the highest correlation with the cutterhead torque(CHT).This work provides a feasible and applicable alternative way to estimate the performance of shield tunneling.

Multi-Scale Convolutional Gated Recurrent Unit Networks for Tool Wear Prediction in Smart Manufacturing

As an integrated application of modern information technologies and artificial intelligence,Prognostic and Health Management(PHM)is important for machine health monitoring.Prediction of tool wear is one of the symbolic applications of PHM technology in modern manufacturing systems and industry.In this paper,a multi-scale Convolutional Gated Recurrent Unit network(MCGRU)is proposed to address raw sensory data for tool wear prediction.At the bottom of MCGRU,six parallel and independent branches with different kernel sizes are designed to form a multi-scale convolutional neural network,which augments the adaptability to features of different time scales.These features of different scales extracted from raw data are then fed into a Deep Gated Recurrent Unit network to capture long-term dependencies and learn significant representations.At the top of the MCGRU,a fully connected layer and a regression layer are built for cutting tool wear prediction.Two case studies are performed to verify the capability and effectiveness of the proposed MCGRU network and results show that MCGRU outperforms several state-of-the-art baseline models.

融合CNN-BiGRU和注意力机制的网络入侵检测模型

为提高网络入侵检测模型特征提取能力和分类准确率,提出了一种融合双向门控循环单元(CNN-BiGRU)和注意力机制的网络入侵检测模型.使用CNN有效提取流量数据集中的非线性特征;双向门控循环单元(BiGRU)提取数据集中的时序特征,最后融合注意力机制对不同类型流量数据通过加权的方式进行重要程度的区分,从而整体提高该模型特征提取与分类的性能.实验结果表明:其整体精确率比双向长短期记忆网络(BiLSTM)模型提升了2.25%.K折交叉验证结果表明:该模型泛化性能良好,避免了过拟合现象的发生,印证了该模型的有效性与合理性.

基于KPCA-CNN-DBiGRU模型的短期负荷预测方法

本文针对已有神经网络模型在短期负荷预测中输入维度过高、预测误差较大等问题,提出了一种结合核主成分分析、卷积神经网络和深度双向门控循环单元的短期负荷预测方法。先运用核主成分分析法对原始高维输入变量进行降维,再通过卷积深度双向门控循环单元网络模型进行负荷预测。以第九届全国电工数学建模竞赛试题A题中的负荷数据作为实际算例,结果表明所提方法较降维之前预测误差大大降低,与已有预测方法相比也有大幅的误差降低。

基于特征选择及ISSA-CNN-BiGRU的短期风功率预测

针对风电功率随机性大、平稳性低,以及直接输入预测模型往往难以取得较高精度等问题,提出了一种基于特征选择及改进麻雀搜索算法(ISSA)优化卷积神经网络-双向门控循环单元(CNN-BiGRU)的短期风电功率预测方法。首先,利用变分模态分解(VMD)将原始功率分解为一组包含不同信息的子分量,以降低原始功率序列的非平稳性,提升可预测性,同时通过观察中心频率方式确定模态分解数。其次,对每一分量采用随机森林(RF)特征重要度的方法进行特征选择,从风速、风向、温度、空气密度等气象特征因素中,选取对各个分量预测贡献度较高的影响因素组成输入特征向量。然后,建立各分量的CNN-BiGRU预测模型,针对神经网络算法参数难调、手动配置参数随机性大的问题,利用ISSA对模型超参数寻优,自适应搜寻最优参数组合。最后,叠加各分量的预测值,得到最终的预测结果。以中国内蒙古某风电场实际数据进行仿真实验,与多种单一及组合预测方法进行对比,结果表明,本文所提方法相比于其他方法具有更高的预测精度,其平均绝对百分比误差值达到2.644 0%;在其他4个数据集上进行的模型准确性及泛化性验证结果显示,模型平均绝对百分比误差值分别为4.385 3%、3.174 9%、1.576 1%和1.358 8%,均保持在5.000 0%以内,证明本文所提方法具有较好的预测精度及泛化能力。

基于CNN-BiGRU-Attention的短期电力负荷预测

针对目前电力负荷数据随机性强,影响因素复杂,传统单一预测模型精度低的问题,结合卷积神经网络(Convolutional neural network,CNN)、双向门控循环单元(Bi-directional gated recurrent unit,BiGRU)以及注意力机制(Attention)在短期电力负荷预测上的不同优点,提出一种基于CNN-BiGRU-Attention的混合预测模型。该方法首先通过CNN对历史负荷和气象数据进行初步特征提取,然后利用BiGRU进一步挖掘特征数据间时序关联,再引入注意力机制,对BiGRU输出状态给与不同权重,强化关键特征,最后完成负荷预测。试验结果表明,该模型的平均绝对百分比误差(Mean absolute percentage error,MAPE)、均方根误差(Root mean square error,RMSE)、判定系数(R-square,R~2)分别为0.167%、0.057%、0.993,三项指标明显优于其他模型,具有更高的预测精度和稳定性,验证了模型在短期负荷预测中的优势。

融合1D-CNN与BiGRU的类不平衡流量异常检测

网络流量异常检测是利用各种检测技术分析判断网络流量,发现网络中潜在的攻击,是一种有效的网络安全防护方法。针对高维海量数据和不同攻击类别的网络流量数据不均衡而导致检测准确率低、误报率高的问题,提出一种融合一维卷积神经网络(1D-CNN)和双向门控循环单元(BiGRU)的类不平衡流量异常检测模型。首先,针对类不平衡数据,通过使用改进的合成少数类过采样技术(SMOTE)即Borderline-SMOTE和基于高斯混合模型(GMM)的欠采样聚类技术进行平衡处理;然后,使用1D-CNN提取数据的局部特征,并利用BiGRU更好地提取数据中的时序特征;最后,在UNSW-NB15数据集对所提模型进行验证,所提模型的准确率为98.12%,误报率为1.28%。结果表明,所提模型提高了对少数攻击的识别率,检测精度高于其他经典机器学习和深度学习模型。

基于Mogrifier-BiGRU的飞行器轨迹预测

针对当前飞行器轨迹预测准确性低的问题,引入双向传播机制和Mogrifier数据耦合模块,改进传统门控循环单元网络,提出基于Mogrifier-BiGRU的飞行器轨迹预测算法,加强网络对历史数据的学习与记忆,使得输入信息与隐藏层数据的充分耦合,提高预测准确度。仿真结果表明,所提方法对飞行器轨迹预测的准确度可达到96.26%,满足我方作战指挥人员对战场态势趋势准确预测的实际需求。

一种融合注意力机制的CNN-BiGRU磁盘故障预测方法研究

磁盘作为重要的存储介质,一旦出现故障很可能会导致存储数据丢失,给个人及企业带来难以估量的损失。现有磁盘故障预测模型存在不能很好地平衡磁盘数据样本、未充分利用磁盘数据的时序特性等问题。以Backblaze云存储公司公布的真实磁盘数据为研究对象,提出了一种融合注意力机制的卷积神经网络(CNN)和双向门控循环单元(BiGRU)网络的磁盘故障预测模型。在数据预处理方面,采用负采样与焦点损失函数来平衡正负样本,利用CNN进行特征提取,并结合BiGRU网络来有效地处理时序数据。通过融合注意力机制,能够让模型快速地捕捉更多关键特征信息,将筛选出的特征与数据输入模型进行训练。通过对比其他故障预测模型,本文提出的模型在精确率等4个评价指标上均有1%~7%的性能提升,为提高磁盘存储的可靠性提供了有力的支撑。

基于注意力机制的IWOA-BiGRU超短期风电功率预测

超短期风电功率预测对电力系统调度及大规模风电并网具有重要作用。为得到准确可靠的风电功率预测结果,针对风电功率数据非线性和时序性的特点,提出一种基于IWOA-AT-BiGRU的超短期风电功率预测方法。首先,提出改进鲸鱼优化算法(improved whale optimization algorithm,IWOA)来优化风电功率预测模型的超参数,加速模型收敛,提高预测准确度;然后,在BiGRU中加入注意力机制(AT),AT用来加强重要信息对风功率的影响,BiGRU同时考虑数据的正反向信息,充分挖掘数据的时序特征;最后,通过某风电场实测数据进行实验,结果表明提出的方法预测准确度均高于其他对比模型,具有良好的预测性能。

基于CEEMDAN-SBiGRU-OMHA的短期电力负荷预测

为了提高短期电力负荷预测的精准度,充分挖掘电力负荷数据的复杂相关性,提出了一种优化多头注意力机制的CEEMDAN-SBiGRU组合预测模型,改进了特征提取和特征融合两个模块.首先,采用自适应噪声完全集成经验模态分解(complete ensemble empirical mode decomposition with adaptive noise,CEEMDAN)将电力负荷数据分解成多个内在模态函数(IMF)和一个残差信号(RES);同时引入降噪自编码器DAE挖掘数据中受气象因素、工作日类型和温度变化的潜在特征.其次,将提取到的复杂特征输入至堆叠双向门控循环单元(stacked bidirectional gated recurrent unit,SBiGRU)模块中继续学习,以获取隐藏状态.最后,将获取的隐藏状态输入至加入残差机制和层归一化优化的多头注意力(optimized multi-head attention,OMHA)机制模块,可以准确地给重要特征分配更高的权重,解决噪声干扰问题.实验结果表明,CEEMDAN-SBiGRU-OMHA组合模型具有更高的精确性.

基于CNN-BiGRU-ResNet的网络入侵检测研究

网络入侵检测是网络安全中的一项重要工作,其主要是通过网络、系统等信息对入侵行为进行判断,它可以及时地发现网络中的攻击行为,传统的网络入侵检测方法存在准确率低并且误报率高的问题,针对上述问题,提出了一种融合双向门控循环单元(BiGRU)、卷积神经网络(CNN)以及残差网络(ResNet)的网络入侵检测方法,该方法通过双向门控循环单元对时间序列特征以及卷积神经网络和残差网络对局部空间特征的提取,利用softmax分类器获得最终的分类结果。实验表明,与基于GRU和ResNet等方法相比,该方法的网络入侵检测效果比较好,其准确率较高,误报率更低。

基于CNN-BiGRU的高压直流输电线路故障识别

针对高压直流(high voltage direct current,HVDC)输电线路故障暂态行波具有时序性和强非线性的特点,导致高过渡电阻情况下故障识别率低的问题,提出基于卷积神经网络(convolutional neural networks,CNN)和双向循环门单元(bidirectional gate recurrent unit,BiGRU)的HVDC输电线路故障识别方法。首先,采用故障后整流侧的双极暂态电流行波作为特征向量,利用CNN提取全局特征,并从中剔除噪声和不稳定成分,完成对数据的降维处理。然后,采用BiGRU来捕获CNN提取到特征的前后时间信息,进一步提取数据中的时序特征,以实现HVDC输电线路故障识别。仿真结果表明:该方法可在不同故障地点以及不同过渡电阻下对单极接地、双极短路、雷击故障、雷击干扰共四种故障实现准确识别,可靠性高,具有较强的耐受过渡电阻能力,同时具备一定的抗噪性能。

基于注意力机制的CNN-BIGRU短期电价预测

针对短期电价预测的复杂性和精确度较差的问题,本文提出一种基于注意力机制的卷积神经网络和双向门控循环单元网络的短期电价预测模型。该模型将历史电价数据经过数据预处理后作为输入,首先利用卷积神经网络提取历史电价序列中的特征;其次,将提取的特征向量构造成时间序列输入到双向门控循环单元网络,充分挖掘特征内部的变化规律进行训练;然后,引入注意力机制来突出重要信息的影响并赋予权重,利用注意力机制对双向门控循环单元网络每个时间步的输出进行加权求和;最后,在全连接层通过激活函数计算输出最终预测值。通过实例验证了本文所提模型的准确性。

基于双注意力机制的MSCN-BiGRU的滚动轴承故障诊断方法

针对滚动轴承故障诊断模型在变工况和环境噪声干扰下诊断精度降低的问题,提出一种基于双注意力机制的多尺度卷积网络(dual attention and multi-scale convolutional networks,DAMSCN)与改进的双向门控循环单元(bidirectional gated recurrent unit,BiGRU)组成的故障诊断模型DAMSCN-BiGRU。首先,多尺度特征融合模块使用不同大小的卷积核,获得多种感受野,从而提取到轴承原始振动信号的多尺度特征信息,并根据重要性对其进行自适应融合,然后利用通道注意力和空间注意力组成的双注意力模块(dual attention module,DAM)对多尺度特征进行重新标定,分配注意力权重,削弱融合特征中的冗余特征;然后,增加注意力层和利用分段激活改进BiGRU进而挖掘信号的时域特征,以提高轴承故障诊断的性能;最后,通过Softmax层完成对不同故障的分类。试验结果表明,与其他智能诊断模型相比,DAMSCN-BiGRU在变工况环境下,平均诊断精度达到98.2%,在强噪声背景下仍然有着85.3%的准确率,且在不同程度的噪声强度下效果均优于其他常用模型,有利于促进滚动轴承的智能故障诊断研究和实际应用。

基于ResNet-TSM和BiGRU网络的移动视频感知质量评价模型

考虑到卡顿、质量切换、内容特征等因素对用户体验质量的影响都会直接体现在客户端的失真视频里,提出了一种客户端的移动视频感知质量评价模型。该模型无须对每种影响因素均进行表征和度量,而是基于深度特征提取+回归的思路,直接建立失真视频与平均意见分数之间的映射模型。首先,构建了ResNet-TSM网络结构,提取失真视频片段的深度时空特征;为了避免维度灾难,采用LargeVis算法对提取的深度特征进行降维,同时提升特征的表达与区分能力。然后,采用双向门控循环单元网络对视频的长时间依赖关系进行建模,得到各视频片段的打分,再利用时间平均池化方法将各片段分数进行聚合,得到整个视频的打分结果。在WaterlooSQoE-Ⅲ和LIVE-NFLX-Ⅱ数据集上的实验结果表明,提出的模型可以获得更高的预测精度。

基于DL-BiGRU多特征融合的注塑件尺寸预测方法

为了充分挖掘注塑成型过程中模腔内的高频时序特征和注塑成型机状态特征,提出基于双层双向门控循环单元网络(DL-BiGRU)的多特征融合注塑件尺寸预测方法.分析膜腔内传感器高频时序特征与注塑件尺寸间的关联性,采用DL-BiGRU网络从高频数据中自动提取时序特征,表征注塑件成型过程状态变化特性.通过采样模腔内高频时序数据进行展成平铺,表征注塑成型的瞬时特征.融合时序特征、瞬时特征和成型机状态特征,构建端到端的深度学习多特征融合框架.将上述3种特征融合并联合训练,提升注塑件尺寸预测精度.在注塑成型数据集上进行模型验证,预测尺寸平均均方误差为4.7×10^(-4) mm^(2),最小误差波动为10^(-5) mm^(2)量级,模型具有较高的预测精度和稳定性.