Landslide displacement prediction based on optimized empirical mode decomposition and deep bidirectional long short-term memory network

There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement and time series of a landslide.The second one is the dynamic evolution of a landslide,which could not be feasibly simulated simply by traditional prediction models.In this paper,a dynamic model of displacement prediction is introduced for composite landslides based on a combination of empirical mode decomposition with soft screening stop criteria(SSSC-EMD)and deep bidirectional long short-term memory(DBi-LSTM)neural network.In the proposed model,the time series analysis and SSSC-EMD are used to decompose the observed accumulated displacements of a slope into three components,viz.trend displacement,periodic displacement,and random displacement.Then,by analyzing the evolution pattern of a landslide and its key factors triggering landslides,appropriate influencing factors are selected for each displacement component,and DBi-LSTM neural network to carry out multi-datadriven dynamic prediction for each displacement component.An accumulated displacement prediction has been obtained by a summation of each component.For accuracy verification and engineering practicability of the model,field observations from two known landslides in China,the Xintan landslide and the Bazimen landslide were collected for comparison and evaluation.The case study verified that the model proposed in this paper can better characterize the"stepwise"deformation characteristics of a slope.As compared with long short-term memory(LSTM)neural network,support vector machine(SVM),and autoregressive integrated moving average(ARIMA)model,DBi-LSTM neural network has higher accuracy in predicting the periodic displacement of slope deformation,with the mean absolute percentage error reduced by 3.063%,14.913%,and 13.960%respectively,and the root mean square error reduced by 1.951 mm,8.954 mm and 7.790 mm respectively.Conclusively,this model not only has high prediction accuracy but also is more stable,which can provide new insight for practical landslide prevention and control engineering.

A real-time prediction method for tunnel boring machine cutter-head torque using bidirectional long short-term memory networks optimized by multi-algorithm

Based on data from the Jilin Water Diversion Tunnels from the Songhua River(China),an improved and real-time prediction method optimized by multi-algorithm for tunnel boring machine(TBM)cutter-head torque is presented.Firstly,a function excluding invalid and abnormal data is established to distinguish TBM operating state,and a feature selection method based on the SelectKBest algorithm is proposed.Accordingly,ten features that are most closely related to the cutter-head torque are selected as input variables,which,in descending order of influence,include the sum of motor torque,cutter-head power,sum of motor power,sum of motor current,advance rate,cutter-head pressure,total thrust force,penetration rate,cutter-head rotational velocity,and field penetration index.Secondly,a real-time cutterhead torque prediction model’s structure is developed,based on the bidirectional long short-term memory(BLSTM)network integrating the dropout algorithm to prevent overfitting.Then,an algorithm to optimize hyperparameters of model based on Bayesian and cross-validation is proposed.Early stopping and checkpoint algorithms are integrated to optimize the training process.Finally,a BLSTMbased real-time cutter-head torque prediction model is developed,which fully utilizes the previous time-series tunneling information.The mean absolute percentage error(MAPE)of the model in the verification section is 7.3%,implying that the presented model is suitable for real-time cutter-head torque prediction.Furthermore,an incremental learning method based on the above base model is introduced to improve the adaptability of the model during the TBM tunneling.Comparison of the prediction performance between the base and incremental learning models in the same tunneling section shows that:(1)the MAPE of the predicted results of the BLSTM-based real-time cutter-head torque prediction model remains below 10%,and both the coefficient of determination(R^(2))and correlation coefficient(r)between measured and predicted values exceed 0.95;and(2)the incremental learning method is suitable for realtime cutter-head torque prediction and can effectively improve the prediction accuracy and generalization capacity of the model during the excavation process.

Power entity recognition based on bidirectional long short-term memory and conditional random fields

With the application of artificial intelligence technology in the power industry,the knowledge graph is expected to play a key role in power grid dispatch processes,intelligent maintenance,and customer service response provision.Knowledge graphs are usually constructed based on entity recognition.Specifically,based on the mining of entity attributes and relationships,domain knowledge graphs can be constructed through knowledge fusion.In this work,the entities and characteristics of power entity recognition are analyzed,the mechanism of entity recognition is clarified,and entity recognition techniques are analyzed in the context of the power domain.Power entity recognition based on the conditional random fields (CRF) and bidirectional long short-term memory (BLSTM) models is investigated,and the two methods are comparatively analyzed.The results indicated that the CRF model,with an accuracy of 83%,can better identify the power entities compared to the BLSTM.The CRF approach can thus be applied to the entity extraction for knowledge graph construction in the power field.

基于自注意力机制和改进的K-BiLSTM的水产养殖水体溶解氧含量预测模型

为精确预测水产养殖水体溶解氧含量,本研究提出一种基于自注意力机制(ATTN)和改进的K-means聚类-基于残差和批标准化(BN)的双向长短期记忆网络(BiLSTM)的水产养殖水体溶解氧含量预测模型。首先,根据环境数据的相似性,使用改进的K-means算法将数据划分成若干个类别;然后,在BiLSTM基础上构建残差连接和加入BN完成高层次特征提取,利用BiLSTM的长期记忆能力保存特征信息;最后,引入自注意力机制突出不同时间节点数据特征的重要性,进一步提升模型的性能。试验结果表明,本研究提出的基于自注意力机制和改进的K-BiLSTM模型的平均绝对误差为0.238、均方根误差为0.322、平均绝对百分比误差为0.035,与单一的BP模型、CNN-LSTM模型、传统的K-means-基于残差和BN的BiLSTM-ATTN等模型相比具有更优的预测性能和泛化能力。

基于BERT+CNN_BiLSTM的列控车载设备故障诊断

列控车载设备作为列车运行控制系统核心设备,在高速列车运行过程中发挥着重要作用。目前,其故障诊断仅依赖于现场作业人员经验,诊断效率相对较低。为了实现列控车载设备故障自动诊断并提高诊断效率,提出了BERT+CNN_BiLSTM故障诊断模型。首先,使用来自变换器的双向编码器表征量(Bidirectional encoder representations from transformers,BERT)模型将应用事件日志(Application event log,AElog)转换为计算机能够识别的可以挖掘语义信息的文本向量表示。其次,分别利用卷积神经网络(Convolutional neural network,CNN)和双向长短时记忆网络(Bidirectional long short-term memory,BiLSTM)提取故障特征并进行组合,从而增强空间和时序能力。最后,利用Softmax实现列控车载设备的故障分类与诊断。实验中,选取一列实际运行的列车为研究对象,以运行过程中产生的AElog日志作为实验数据来验证BERT+CNN_BiLSTM模型的性能。与传统机器学习算法、BERT+BiLSTM模型和BERT+CNN模型相比,BERT+CNN_BiLSTM模型的准确率、召回率和F1分别为92.27%、91.03%和91.64%,表明该模型在高速列车控制系统故障诊断中性能优良。

Analyzing Arabic Twitter-Based Patient Experience Sentiments Using Multi-Dialect Arabic Bidirectional Encoder Representations from Transformers

Healthcare organizations rely on patients’feedback and experiences to evaluate their performance and services,thereby allowing such organizations to improve inadequate services and address any shortcomings.According to the literature,social networks and particularly Twitter are effective platforms for gathering public opinions.Moreover,recent studies have used natural language processing to measure sentiments in text segments collected from Twitter to capture public opinions about various sectors,including healthcare.The present study aimed to analyze Arabic Twitter-based patient experience sentiments and to introduce an Arabic patient experience corpus.The authors collected 12,400 tweets from Arabic patients discussing patient experiences related to healthcare organizations in Saudi Arabia from 1 January 2008 to 29 January 2022.The tweets were labeled according to sentiment(positive or negative)and sector(public or private),and thereby the Hospital Patient Experiences in Saudi Arabia(HoPE-SA)dataset was produced.A simple statistical analysis was conducted to examine differences in patient views of healthcare sectors.The authors trained five models to distinguish sentiments in tweets automatically with the following schemes:a transformer-based model fine-tuned with deep learning architecture and a transformer-based model fine-tuned with simple architecture,using two different transformer-based embeddings based on Bidirectional Encoder Representations from Transformers(BERT),Multi-dialect Arabic BERT(MAR-BERT),and multilingual BERT(mBERT),as well as a pretrained word2vec model with a support vector machine classifier.This is the first study to investigate the use of a bidirectional long short-term memory layer followed by a feedforward neural network for the fine-tuning of MARBERT.The deep-learning fine-tuned MARBERT-based model—the authors’best-performing model—achieved accuracy,micro-F1,and macro-F1 scores of 98.71%,98.73%,and 98.63%,respectively.

基于奇异谱分析的CNN-BiLSTM短期空调负荷预测模型

空调负荷的精准预测对建筑空调系统优化控制具有重要意义。为提高空调负荷预测精度,提出了一种基于奇异谱分析(SSA,Singular Spectrum Analysis)的卷积神经网络(CNN,Convolutional Neural Network)和双向长短时记忆网络(BiLSTM,Bidirectional Long Short Term Memory)短期空调负荷预测模型。使用皮尔森相关系数选取与空调负荷高相关性特征。针对空调负荷的波动性和随机性,采用SSA将空调负荷分解为多个分量,同时将各个分量带入CNN-BiLSTM模型进行预测,该模型利用了CNN的特征提取和BiLSTM的双向学习能力,并将各个分量预测结果进行重构。通过不同建筑类型的空调数据对该模型进行验证分析,发现所提出模型在预测办公建筑空调负荷中RMSE、MAPE和MAE为19.47RT、14.72RT和2.33%,在预测商业建筑空调负荷中RMSE、MAPE和MAE为82.5RT、34.21RT和0.87%。结果表明,所提出的模型具有普适性且精度较高,可进行推广应用。

基于DBO-VMD和IWOA-BILSTM神经网络组合模型的短期电力负荷预测

新能源在现代电力系统中占比不断提高,其负荷不规律性、波动性远大于传统电力系统,这就导致负荷预测精度不高。针对这个问题,提出了蜣螂优化(dung beetle optimizer,DBO)算法优化变分模态分解(variational mode decomposition,VMD)与改进鲸鱼优化算法优化双向长短期记忆(improved whale optimization algorithm-bidirectional long short-term memory,IWOA-BILSTM)神经网络相结合的短期负荷预测模型。首先利用DBO优化VMD,分解时间序列数据,并根据最小包络熵对各种特征数据进行分类,增强了分解效果。通过对原始数据进行有效分解,降低了数据的波动性。然后使用非线性收敛因子、自适应权重策略与随机差分法变异策略增强鲸鱼优化算法的局部及全局搜索能力得到改进鲸鱼优化算法(improved whale optimization algorithm,IWOA),并用于优化双向长短期记忆(bidirectional long short-term memory,BILSTM)神经网络,增加了模型预测的精确度。最后将所提方法应用于某地真实的负荷数据,得到最终相对均方根误差、平均绝对误差和平均绝对百分比误差分别为0.0084、48.09、0.66%,证明了提出的模型对于短期负荷预测的有效性。

融合SARIMA与BiLSTM的水利设施形变预测

水利设施形变预测可以有效地判断水利设施的运行状态。水利设施安全监测数据是时间序列数据,既有趋势性又有季节性。为了获得更准确的预测结果,文中提出一种基于季节自回归差分移动平均(SARIMA)模型和双向长短时记忆(BiLSTM)网络的预测模型,以解决无法充分挖掘数据中正向与反向的关联进行预测的问题。该模型采用SARIMA模型预测变形数据中的线性分量,采用BiLSTM模型预测变形数据中的非线性分量,使得模型能够更好地提取历史数据中的非线性关系以及正向与反向关系从而提高预测准确度。结合某水电站4#引水涵洞监测数据,使用SARIMA-BiLSTM模型对裂缝计开合度时间序列进行了预测,并与反向传播神经网络模型、SARIMA模型和SARIMA-LSTM模型的预测结果进行对比,比对结果证明所提方法有效地提高了预测精度。

基于改进BiLSTM-RF的短期负荷预测研究

电力负荷的准确预测能有效保持电网运行的稳定性,提高经济效益和社会效益。为了提高负荷预测的精准度,首先利用麻雀搜索算法(SSA)和变分模态分解(VMD)对输入的原始负荷进行模态分解,降低了电力负荷数据随机性与非平稳性;然后利用双向长短期记忆-随机森林(BiLSTM-RF)组合模型对分解后的子模态进行特征提取和预测。对某地区公开数据的性能验证与模型对比分析结果表明,改进的BiLSTM-RF(+BiLSTM-RF)组合模型在决定系数(R^(2))、均方根误差(RMSE)、平均绝对误差(MAE)、平均绝对百分误差(MAPE)这四个预测精度指标方面分别达到了0.973、300.31、134.91、0.037。与传统的支持向量机(SVM)、长短期记忆(LSTM)网络、门控循环单元(GRU)等单一预测方法,以及未改进的BiLSTM-RF组合预测方法相比,+BiLSTM-RF组合模型有更好的预测表现。

基于SSA-CNN-BiLSTM组合模型的短时交通流量预测

为改善城市道路交通拥堵状况,并为智能交通系统决策提供辅助手段,针对短时交通流的非线性和时序性特点,构建了一种基于麻雀搜索算法(SSA)优化的卷积神经网络(CNN)联合双向长短时记忆神经网络(BiLSTM)的组合模型以预测短时交通流量。首先,对原始交通流数据进行异常值清洗、小波阈值去噪和归一化处理。然后,利用SSA算法对CNN与BiLSTM组合网络中的隐藏层单元数、初始学习率和L2正则化系数三个超参数迭代寻优。最后,将搜索得到的最优超参数组合输入搭建好的组合网络中进行训练和预测。实验结果显示:与粒子群优化(PSO)和灰狼优化(GWO)算法相比,SSA算法在网络超参数寻优过程中的收敛速度更快,全局寻优能力更强;与3种对比模型(CNNBiLSTM、BiLSTM和LSTM)相比,在5 min时间尺度划分下,SSA-CNN-BiLSTM组合模型的均方根误差(RMSE)分别降低了5.46、12.78、20.38,平均绝对百分比误差(MAPE)分别降低了0.49%、2.24%、3.11%;在15 min时间尺度划分下,SSA-CNN-BiLSTM组合模型的RMSE分别降低了9.70、28.42、41.18,MAPE分别降低了0.50%、1.98%、2.59%。研究表明,相比既有算法,该短时交通流量预测组合模型在精度和稳定性上都有所提升,可通过提供更精准的短时交通出行信息来改善道路交通状况。

基于1DCNN-BiLSTM-BiGRU的电能质量扰动分类方法

为了应对电能质量扰动(PQD)识别中噪声干扰导致的识别率下降问题,本文提出一种基于一维卷积神经网络(1DCNN)-双向长短期记忆(BiLSTM)网络-双向门控循环单元(BiGRU)的PQD分类方法。该方法首先借助1DCNN有效地提取原始信号的浅层局部特征,然后通过BiLSTM和BiGRU组合模块对时序信息和上下文关系进行深入处理,从而实现深层时序特征的提取。最后,将所提取的特征经分类模块用于PQD识别。仿真结果表明,与传统方法相比,本文所提方法在准确性方面更具优势,且抗噪声能力更强。

基于BERT-BiLSTM-CRF模型的油气领域命名实体识别

针对油气领域知识图谱构建过程中命名实体识别使用传统方法存在实体特征信息提取不准确、识别效率低的问题,提出了一种基于BERT-BiLSTM-CRF模型的命名实体识别研究方法。该方法首先利用BERT(bidirectional encoder representations from transformers)预训练模型得到输入序列语义的词向量;然后将训练后的词向量输入双向长短期记忆网络(bi-directional long short-term memory,BiLSTM)模型进一步获取上下文特征;最后根据条件随机场(conditional random fields,CRF)的标注规则和序列解码能力输出最大概率序列标注结果,构建油气领域命名实体识别模型框架。将BERT-BiLSTM-CRF模型与其他2种命名实体识别模型(BiLSTM-CRF、BiLSTM-Attention-CRF)在包括3万多条文本语料数据、4类实体的自建数据集上进行了对比实验。实验结果表明,BERT-BiLSTM-CRF模型的准确率(P)、召回率(R)和F_(1)值分别达到91.3%、94.5%和92.9%,实体识别效果优于其他2种模型。

基于Transformer与BiLSTM的网络流量入侵检测

网络流量入侵检测技术对主机和平台安全起着重要作用。目前常采用机器学习和深度学习技术进行网络流量入侵检测,然而相关数据集的不平衡问题导致模型偏向于学习多数类数据的特征而忽视少数类数据的特征,严重影响了检测准确率。结合SMOTE算法和生成对抗网络(GAN)构建OSW模型对训练数据进行预处理,通过Wasserstein GAN学习少数类数据分布情况,避免边缘分布问题,构造平衡数据集。建立基于Transformer与双向长短时记忆-深度神经网络(BiLSTM-DNN)的TBD入侵检测模型,使用Transformer中的编码器捕捉全局联系并对输入数据进行初步特征提取,利用BiLSTM网络进行长距离依赖特征提取保留数据的序列化特征,采用DNN进一步提取深层次特征,最终通过Softmax分类器获得分类结果。在NSL_KDD数据集上的实验结果表明,在进行数据平衡处理后TBD模型的二分类和五分类任务检测准确率分别达到90.3%和79.8%,均高于对比的深度神经网络模型以及机器学习算法。

基于VMD-BO-BiLSTM的猪肉价格预测模型

基于猪肉价格的非线性与波动性特性,提出一种基于变分模态分解(variational modal decomposition,VMD)和贝叶斯优化(Bayesian optimization,BO)的双向长短时记忆(bidirectional long short-term memory,BiLSTM)网络的猪肉价格预测方法。首先采用变分模态分解对数据进行预处理,将数据分解为具有相对简单波动的子序列;然后通过贝叶斯算法对双向长短时记忆网络模型的第1、2隐含层神经元数目、学习率和批次大小进行寻优,根据寻优的结果建立预测模型。实验结果表明:VMD-BO-BiLSTM方法的平均绝对误差、均方根误差、平均绝对百分比误差和确定系数分别为1.101214、1.466100、0.040631、0.987760,相比传统单一的LSTM,BiLSTM模型精确度更高,有更高的适用性,适合对猪肉价格预测。

基于数据增强技术与CNN-BiLSTM-Attention的油田注水流量预测及效果

准确识别地层注水情况是油田开发的重要前提,对制定合理的注水发展规划也具有重要的指导意义。为准确预测注水,提出一种结合卷积神经网络、双向长短期记忆网络与注意力机制的油田注水流量预测方法,该方法首先将卷积神经网络(convolutional neural network,CNN)和双向长短期记忆网络(bidirection long short-term memory,BiLSTM)进行联合,用于捕获注水流量的复杂非线性时空关系,然后采用注意力机制来关注输入的重要特征。并针对油田历史数据匮乏问题,提出使用数据增强技术来增加一维时间序列的数据量。采用国内某油田注水井真实历史注水数据进行实验。结果表明:所提出的CNN-BiLSTM-Attention预测模型的平均绝对误差(mean absolute error,MAE)、均方根误差(root mean square error,RMSE)、平均绝对百分比误差(mean absolute percentage error,MAPE)和决定系数(coefficient of determination,R 2)分别为0.027、0.043、9.936和0.968,通过多种模型对比,表明该方法具有较高的预测精度,可以更准确地预测注水流量。采用数据增强技术可以有效提高模型的预测精度。研究成果可为油田精细化注水提供调整方案与高质量数据,从而为油田智能化开发提供理论依据。

基于ResNet-ABiLSTM的滚动轴承剩余寿命预测

传统数据驱动的方法过度依赖先验知识且特征提取能力不足,从而导致预测精度不高等后果。针对这一问题,提出了一种带有自注意力机制(SAM)的残差网络(ResNet)与双向长短时记忆网络(BiLSTM)结合的剩余使用寿命(RUL)预测方法(ResNet-ABiLSTM)。首先,对采集的原始监测信号进行了标准化处理,并采用滑窗法对处理后的数据进行了重采样,以实现数据的扩充目标;然后,通过采用残差网络和双向长短时记忆网络,分别提取了数据空间维度和时间维度上的深层特征,同时引入了自注意力机制,关注了时空维度上反映设备退化趋势的更重要的特征;最后,采用PHM2012轴承数据集对预测效果进行了验证,并将其结果与CNN-LSTM、ResNet-BiLSTM、HI-GRNN、CNN-HI、ResNet-CBAM、DRN-BiGRU等方法的预测结果进行了对比分析。研究结果表明:采用ResNet-ABiLSTM方法的两项误差值(RMSE、MAE)分别取得了0.037、0.029的最低值,其效果显著优于其他对比方法;该结果验证了ResNet-ABiLSTM方法对轴承RUL预测的准确性和有效性。

基于DRN-BiLSTM模型的矿井涌水量预测

针对矿井涌水量预测中存在的深度学习模型预测精度不高和适用性不强的问题,提出了一种基于深度残差网络(Deep Residual Network,DRN)和双向长短记忆网络(Bidirectional short and long memory network,BiLSTM)的矿井涌水量预测方法。首先,将矿井涌水量数据进行小波分解和归一化处理,得到趋势项数据和细节项数据;其次,采用DRN网络方法对趋势项数据进行预测,采用BiLSTM网络方法对细节项数据进行预测;最后,将2部分预测结果进行重构得到矿井涌水量预测结果。研究结果表明:DRN-BiLSTM模型相比于单一模型预测精度更高,说明该模型具有更好的泛化性。

基于BERT-BILSTM-CRF模型的电力行业事故文本智能分析

为解决电力行业事故报告文本较长、语义复杂,难以进行有效文本识别问题,提出1种以BERT作为底层的预训练模型,并设计1种双重注意力机制编码器,结合BILSTM-CRF深度挖掘事故文本语义特征,从而实现文本智能分析。首先构建电力词典,通过对BERT预训练,进行BIO标注,然后引入BILSTM-CRF模型实现对文本标签智能分类,最后将该模型与现行其他4种深度学习模型进行对比。研究结果表明:该模型智能识别精确率、召回率及F 1值(查准率)均达到约97%,较其他4种模型中效果最好的模型分别提高0.02,0.03,0.02。研究结果可为电力行业事故报告文本分析提供1种新思路。

基于KPCA和BiLSTM的分解炉出口温度预测

水泥生产过程中,分解炉出口温度是非常重要的工艺参数,为了应对出口温度变量的多样性,文章提出一种核主成分分析(kernel principal component analysis,KPCA)与双向长短期记忆(bidirectional long short-term memory,BiLSTM)神经网络相结合的温度预测组合模型用来预测分解炉的出口温度。通过KPCA筛选出影响因素的主成分从而达到数据降维目的,将降维后的主成分作为BiLSTM神经网络的输入,分解炉出口温度作为BiLSTM神经网络的输出。经BiLSTM神经网络训练,得到分解炉出口温度预测模型。通过对比验证表明,使用KPCA-BiLSTM相结合的温度预测模型具有较好的预测精度。