Integrating Transformer and Bidirectional Long Short-Term Memory for Intelligent Breast Cancer Detection from Histopathology Biopsy Images

Breast cancer is a significant threat to the global population,affecting not only women but also a threat to the entire population.With recent advancements in digital pathology,Eosin and hematoxylin images provide enhanced clarity in examiningmicroscopic features of breast tissues based on their staining properties.Early cancer detection facilitates the quickening of the therapeutic process,thereby increasing survival rates.The analysis made by medical professionals,especially pathologists,is time-consuming and challenging,and there arises a need for automated breast cancer detection systems.The upcoming artificial intelligence platforms,especially deep learning models,play an important role in image diagnosis and prediction.Initially,the histopathology biopsy images are taken from standard data sources.Further,the gathered images are given as input to the Multi-Scale Dilated Vision Transformer,where the essential features are acquired.Subsequently,the features are subjected to the Bidirectional Long Short-Term Memory(Bi-LSTM)for classifying the breast cancer disorder.The efficacy of the model is evaluated using divergent metrics.When compared with other methods,the proposed work reveals that it offers impressive results for detection.

Landslide displacement prediction based on optimized empirical mode decomposition and deep bidirectional long short-term memory network

There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement and time series of a landslide.The second one is the dynamic evolution of a landslide,which could not be feasibly simulated simply by traditional prediction models.In this paper,a dynamic model of displacement prediction is introduced for composite landslides based on a combination of empirical mode decomposition with soft screening stop criteria(SSSC-EMD)and deep bidirectional long short-term memory(DBi-LSTM)neural network.In the proposed model,the time series analysis and SSSC-EMD are used to decompose the observed accumulated displacements of a slope into three components,viz.trend displacement,periodic displacement,and random displacement.Then,by analyzing the evolution pattern of a landslide and its key factors triggering landslides,appropriate influencing factors are selected for each displacement component,and DBi-LSTM neural network to carry out multi-datadriven dynamic prediction for each displacement component.An accumulated displacement prediction has been obtained by a summation of each component.For accuracy verification and engineering practicability of the model,field observations from two known landslides in China,the Xintan landslide and the Bazimen landslide were collected for comparison and evaluation.The case study verified that the model proposed in this paper can better characterize the"stepwise"deformation characteristics of a slope.As compared with long short-term memory(LSTM)neural network,support vector machine(SVM),and autoregressive integrated moving average(ARIMA)model,DBi-LSTM neural network has higher accuracy in predicting the periodic displacement of slope deformation,with the mean absolute percentage error reduced by 3.063%,14.913%,and 13.960%respectively,and the root mean square error reduced by 1.951 mm,8.954 mm and 7.790 mm respectively.Conclusively,this model not only has high prediction accuracy but also is more stable,which can provide new insight for practical landslide prevention and control engineering.

A real-time prediction method for tunnel boring machine cutter-head torque using bidirectional long short-term memory networks optimized by multi-algorithm

Based on data from the Jilin Water Diversion Tunnels from the Songhua River(China),an improved and real-time prediction method optimized by multi-algorithm for tunnel boring machine(TBM)cutter-head torque is presented.Firstly,a function excluding invalid and abnormal data is established to distinguish TBM operating state,and a feature selection method based on the SelectKBest algorithm is proposed.Accordingly,ten features that are most closely related to the cutter-head torque are selected as input variables,which,in descending order of influence,include the sum of motor torque,cutter-head power,sum of motor power,sum of motor current,advance rate,cutter-head pressure,total thrust force,penetration rate,cutter-head rotational velocity,and field penetration index.Secondly,a real-time cutterhead torque prediction model’s structure is developed,based on the bidirectional long short-term memory(BLSTM)network integrating the dropout algorithm to prevent overfitting.Then,an algorithm to optimize hyperparameters of model based on Bayesian and cross-validation is proposed.Early stopping and checkpoint algorithms are integrated to optimize the training process.Finally,a BLSTMbased real-time cutter-head torque prediction model is developed,which fully utilizes the previous time-series tunneling information.The mean absolute percentage error(MAPE)of the model in the verification section is 7.3%,implying that the presented model is suitable for real-time cutter-head torque prediction.Furthermore,an incremental learning method based on the above base model is introduced to improve the adaptability of the model during the TBM tunneling.Comparison of the prediction performance between the base and incremental learning models in the same tunneling section shows that:(1)the MAPE of the predicted results of the BLSTM-based real-time cutter-head torque prediction model remains below 10%,and both the coefficient of determination(R^(2))and correlation coefficient(r)between measured and predicted values exceed 0.95;and(2)the incremental learning method is suitable for realtime cutter-head torque prediction and can effectively improve the prediction accuracy and generalization capacity of the model during the excavation process.

Power entity recognition based on bidirectional long short-term memory and conditional random fields

With the application of artificial intelligence technology in the power industry,the knowledge graph is expected to play a key role in power grid dispatch processes,intelligent maintenance,and customer service response provision.Knowledge graphs are usually constructed based on entity recognition.Specifically,based on the mining of entity attributes and relationships,domain knowledge graphs can be constructed through knowledge fusion.In this work,the entities and characteristics of power entity recognition are analyzed,the mechanism of entity recognition is clarified,and entity recognition techniques are analyzed in the context of the power domain.Power entity recognition based on the conditional random fields (CRF) and bidirectional long short-term memory (BLSTM) models is investigated,and the two methods are comparatively analyzed.The results indicated that the CRF model,with an accuracy of 83%,can better identify the power entities compared to the BLSTM.The CRF approach can thus be applied to the entity extraction for knowledge graph construction in the power field.

Seismic-inversion method for nonlinear mapping multilevel well–seismic matching based on bidirectional long short-term memory networks

In this paper,the recurrent neural network structure of a bidirectional long shortterm memory network(Bi-LSTM)with special memory cells that store information is used to characterize the deep features of the variation pattern between logging and seismic data.A mapping relationship model between high-frequency logging data and low-frequency seismic data is established via nonlinear mapping.The seismic waveform is infinitely approximated using the logging curve in the low-frequency band to obtain a nonlinear mapping model of this scale,which then stepwise approach the logging curve in the high-frequency band.Finally,a seismic-inversion method of nonlinear mapping multilevel well–seismic matching based on the Bi-LSTM network is developed.The characteristic of this method is that by applying the multilevel well–seismic matching process,the seismic data are stepwise matched to the scale range that is consistent with the logging curve.Further,the matching operator at each level can be stably obtained to effectively overcome the problems that occur in the well–seismic matching process,such as the inconsistency in the scale of two types of data,accuracy in extracting the seismic wavelet of the well-side seismic traces,and multiplicity of solutions.Model test and practical application demonstrate that this method improves the vertical resolution of inversion results,and at the same time,the boundary and the lateral characteristics of the sand body are well maintained to improve the accuracy of thin-layer sand body prediction and achieve an improved practical application effect.

基于BiLSTM-XGBoost混合模型的储层岩性识别

储层岩性分类是地质研究基础,基于数据驱动的机器学习模型虽然能较好地识别储层岩性,但由于测井数据是特殊的序列数据,模型很难有效提取数据的空间相关性,造成模型对储层识别仍存在不足.针对此问题,本文结合双向长短期循环神经网络(bidirectional long short-term memory,BiLSTM)和极端梯度提升决策树(extreme gradient boosting decision tree,XGBoost),提出双向记忆极端梯度提升(BiLSTM-XGBoost,BiXGB)模型预测储层岩性.该模型在传统XGBoost基础上融入了BiLSTM,大大增强了模型对测井数据的特征提取能力.BiXGB模型使用BiLSTM对测井数据进行特征提取,将提取到的特征传递给XGBoost分类模型进行训练和预测.将BiXGB模型应用于储层岩性数据集时,模型预测的总体精度达到了91%.为了进一步验证模型的准确性和稳定性,将模型应用于UCI公开的Occupancy序列数据集,结果显示模型的预测总体精度也高达93%.相较于其他机器学习模型,BiXGB模型能准确地对序列数据进行分类,提高了储层岩性的识别精度,满足了油气勘探的实际需要,为储层岩性识别提供了新的方法.

基于自注意力机制和改进的K-BiLSTM的水产养殖水体溶解氧含量预测模型

为精确预测水产养殖水体溶解氧含量,本研究提出一种基于自注意力机制(ATTN)和改进的K-means聚类-基于残差和批标准化(BN)的双向长短期记忆网络(BiLSTM)的水产养殖水体溶解氧含量预测模型。首先,根据环境数据的相似性,使用改进的K-means算法将数据划分成若干个类别;然后,在BiLSTM基础上构建残差连接和加入BN完成高层次特征提取,利用BiLSTM的长期记忆能力保存特征信息;最后,引入自注意力机制突出不同时间节点数据特征的重要性,进一步提升模型的性能。试验结果表明,本研究提出的基于自注意力机制和改进的K-BiLSTM模型的平均绝对误差为0.238、均方根误差为0.322、平均绝对百分比误差为0.035,与单一的BP模型、CNN-LSTM模型、传统的K-means-基于残差和BN的BiLSTM-ATTN等模型相比具有更优的预测性能和泛化能力。

融合CNN与BiLSTM模型的短期电能负荷预测

针对卷积神经网络(CNN)在捕捉预测序列间历史相关性方面的不足以及在变量复杂情况下出现的无法精准提取预测关键信息的问题,提出一种将双向长短期记忆网络(BiLSTM)与卷积神经网络结合的CNN-BiLSTM模型.首先,采用数据预处理方法保证数据的正确性和完整性,并对数据进行分析以探究多变量之间的相关性;其次,通过CNN与L1正则化对多维输入特征进行特征筛选,选取与预测相关的重要性特征向量;最后,使用BiLSTM对CNN输出的关键特征信息进行保存,形成向量与预测序列,并通过分析时序特征的潜在特点,提取用户的内在消费模式.实验比较了该模型与其他时序模型在不同时间分辨率下的预测效果,实验结果表明,CNN-BiLSTM模型在不同的回望时间间隔下表现出了最佳的预测性能,能够实现更好的短期负荷预测.

基于BERT+CNN_BiLSTM的列控车载设备故障诊断

列控车载设备作为列车运行控制系统核心设备,在高速列车运行过程中发挥着重要作用。目前,其故障诊断仅依赖于现场作业人员经验,诊断效率相对较低。为了实现列控车载设备故障自动诊断并提高诊断效率,提出了BERT+CNN_BiLSTM故障诊断模型。首先,使用来自变换器的双向编码器表征量(Bidirectional encoder representations from transformers,BERT)模型将应用事件日志(Application event log,AElog)转换为计算机能够识别的可以挖掘语义信息的文本向量表示。其次,分别利用卷积神经网络(Convolutional neural network,CNN)和双向长短时记忆网络(Bidirectional long short-term memory,BiLSTM)提取故障特征并进行组合,从而增强空间和时序能力。最后,利用Softmax实现列控车载设备的故障分类与诊断。实验中,选取一列实际运行的列车为研究对象,以运行过程中产生的AElog日志作为实验数据来验证BERT+CNN_BiLSTM模型的性能。与传统机器学习算法、BERT+BiLSTM模型和BERT+CNN模型相比,BERT+CNN_BiLSTM模型的准确率、召回率和F1分别为92.27%、91.03%和91.64%,表明该模型在高速列车控制系统故障诊断中性能优良。

结合Word2vec和BiLSTM的民航非计划事件分析方法

安全是民航业的核心主题。针对目前民航非计划事件分析严重依赖专家经验及分析效率低下的问题,文章提出一种结合Word2vec和双向长短期记忆(bidirectional long short-term memory,BiLSTM)神经网络模型的民航非计划事件分析方法。首先采用Word2vec模型针对事件文本语料进行词向量训练,缩小空间向量维度;然后通过BiLSTM模型自动提取特征,获取事件文本的完整序列信息和上下文特征向量;最后采用softmax函数对民航非计划事件进行分类。实验结果表明,所提出的方法分类效果更好,能达到更优的准确率和F 1值,对不平衡数据样本同样具有较稳定的分类性能,证明了该方法在民航非计划事件分析上的适用性和有效性。

Analyzing Arabic Twitter-Based Patient Experience Sentiments Using Multi-Dialect Arabic Bidirectional Encoder Representations from Transformers

Healthcare organizations rely on patients’feedback and experiences to evaluate their performance and services,thereby allowing such organizations to improve inadequate services and address any shortcomings.According to the literature,social networks and particularly Twitter are effective platforms for gathering public opinions.Moreover,recent studies have used natural language processing to measure sentiments in text segments collected from Twitter to capture public opinions about various sectors,including healthcare.The present study aimed to analyze Arabic Twitter-based patient experience sentiments and to introduce an Arabic patient experience corpus.The authors collected 12,400 tweets from Arabic patients discussing patient experiences related to healthcare organizations in Saudi Arabia from 1 January 2008 to 29 January 2022.The tweets were labeled according to sentiment(positive or negative)and sector(public or private),and thereby the Hospital Patient Experiences in Saudi Arabia(HoPE-SA)dataset was produced.A simple statistical analysis was conducted to examine differences in patient views of healthcare sectors.The authors trained five models to distinguish sentiments in tweets automatically with the following schemes:a transformer-based model fine-tuned with deep learning architecture and a transformer-based model fine-tuned with simple architecture,using two different transformer-based embeddings based on Bidirectional Encoder Representations from Transformers(BERT),Multi-dialect Arabic BERT(MAR-BERT),and multilingual BERT(mBERT),as well as a pretrained word2vec model with a support vector machine classifier.This is the first study to investigate the use of a bidirectional long short-term memory layer followed by a feedforward neural network for the fine-tuning of MARBERT.The deep-learning fine-tuned MARBERT-based model—the authors’best-performing model—achieved accuracy,micro-F1,and macro-F1 scores of 98.71%,98.73%,and 98.63%,respectively.

基于CRITIC和多策略秃鹰优化BiLSTM的水质预测研究

科学有效地预测水质对于水环境的可持续发展和人类健康具有重要意义,为此以固原市某黄河断面的水质监测数据为研究对象,提出了基于指标客观性的权重赋权(Criteria Importance Though Intercriteria Correlation,CRITIC)法和改进的秃鹰搜索(Improved Bald Eagle Search,IBES)算法优化双向长短时记忆网络(Bidirectional Long Short-Term Memory Network,BiLSTM)的组合水质等级预测模型。首先,采用CRITIC法确定各水质指标的权重,加权求和获得一项综合水质指标,从而提出一种改进的水质评价指标体系,以为BiLSTM提供更丰富、更可靠的水质特征信息。其次,在训练过程中引入Logistic映射和莱维飞行策略,并设计交叉共享及准反向搜索策略优化秃鹰搜索(Bald Eagle Search,BES)算法,以提升其种群多样性,增强寻优能力。最后,通过IBES算法迭代寻找BiLSTM的最佳学习率、隐藏层节点数以及正则化系数的超参数组合,进一步提高其预测水平。结果显示:与IBES-BiLSTM、BES-BiLSTM、GA-BiLSTM、PSO-BiLSTM和BiLSTM等模型相比,CRITIC-IBES-BiLSTM模型进行水质等级预测的准确率、精准率、召回率及F_(1)均最高,且具有更好的稳定性。

基于DBO-VMD和IWOA-BILSTM神经网络组合模型的短期电力负荷预测

新能源在现代电力系统中占比不断提高,其负荷不规律性、波动性远大于传统电力系统,这就导致负荷预测精度不高。针对这个问题,提出了蜣螂优化(dung beetle optimizer,DBO)算法优化变分模态分解(variational mode decomposition,VMD)与改进鲸鱼优化算法优化双向长短期记忆(improved whale optimization algorithm-bidirectional long short-term memory,IWOA-BILSTM)神经网络相结合的短期负荷预测模型。首先利用DBO优化VMD,分解时间序列数据,并根据最小包络熵对各种特征数据进行分类,增强了分解效果。通过对原始数据进行有效分解,降低了数据的波动性。然后使用非线性收敛因子、自适应权重策略与随机差分法变异策略增强鲸鱼优化算法的局部及全局搜索能力得到改进鲸鱼优化算法(improved whale optimization algorithm,IWOA),并用于优化双向长短期记忆(bidirectional long short-term memory,BILSTM)神经网络,增加了模型预测的精确度。最后将所提方法应用于某地真实的负荷数据,得到最终相对均方根误差、平均绝对误差和平均绝对百分比误差分别为0.0084、48.09、0.66%,证明了提出的模型对于短期负荷预测的有效性。

基于奇异谱分析的CNN-BiLSTM短期空调负荷预测模型

空调负荷的精准预测对建筑空调系统优化控制具有重要意义。为提高空调负荷预测精度,提出了一种基于奇异谱分析(SSA,Singular Spectrum Analysis)的卷积神经网络(CNN,Convolutional Neural Network)和双向长短时记忆网络(BiLSTM,Bidirectional Long Short Term Memory)短期空调负荷预测模型。使用皮尔森相关系数选取与空调负荷高相关性特征。针对空调负荷的波动性和随机性,采用SSA将空调负荷分解为多个分量,同时将各个分量带入CNN-BiLSTM模型进行预测,该模型利用了CNN的特征提取和BiLSTM的双向学习能力,并将各个分量预测结果进行重构。通过不同建筑类型的空调数据对该模型进行验证分析,发现所提出模型在预测办公建筑空调负荷中RMSE、MAPE和MAE为19.47RT、14.72RT和2.33%,在预测商业建筑空调负荷中RMSE、MAPE和MAE为82.5RT、34.21RT和0.87%。结果表明,所提出的模型具有普适性且精度较高,可进行推广应用。

基于BERT-BiLSTM-CRF模型的油气领域命名实体识别

针对油气领域知识图谱构建过程中命名实体识别使用传统方法存在实体特征信息提取不准确、识别效率低的问题,提出了一种基于BERT-BiLSTM-CRF模型的命名实体识别研究方法。该方法首先利用BERT(bidirectional encoder representations from transformers)预训练模型得到输入序列语义的词向量;然后将训练后的词向量输入双向长短期记忆网络(bi-directional long short-term memory,BiLSTM)模型进一步获取上下文特征;最后根据条件随机场(conditional random fields,CRF)的标注规则和序列解码能力输出最大概率序列标注结果,构建油气领域命名实体识别模型框架。将BERT-BiLSTM-CRF模型与其他2种命名实体识别模型(BiLSTM-CRF、BiLSTM-Attention-CRF)在包括3万多条文本语料数据、4类实体的自建数据集上进行了对比实验。实验结果表明,BERT-BiLSTM-CRF模型的准确率(P)、召回率(R)和F_(1)值分别达到91.3%、94.5%和92.9%,实体识别效果优于其他2种模型。

基于ISABO-IBiLSTM模型的刀具磨损预测方法

针对现有的刀具磨损预测方法因为缺少优化算法及网络结构不完善而导致预测精度不高的问题,提出了一种将改进的减法优化器(SABO)算法和改进的双向长短时记忆(BiLSTM)网络相结合的刀具磨损状态预测模型(ISABO-IBiLSTM模型)。首先,采用截断法、Hampel滤波法、改进的完全自适应噪声集合经验模态分解(ICEEMDAN)-改进的小波阈值降噪法对加速度振动信号与力信号数据进行预处理。然后,提取预处理后的信号数据的时域、频域、时频域特征,并通过斯皮尔曼和最大互信息相关系数筛选特征,构建模型的输入。最后,利用改进的SABO算法对改进后的BiLSTM网络进行参数寻优,基于所得到的优化参数训练网络实现磨损预测。实验数据分析结果表明,所提出的ISABO-IBiLSTM模型对刀具磨损量的预测精度为98.49%~98.83%,较BiLSTM模型、改进的BiLSTM模型、改进的卷积神经网络(ICNN)-BiLSTM模型有了较大的提高。

Infrasound Event Classification Fusion Model Based on Multiscale SE-CNN and BiLSTM

The classification of infrasound events has considerable importance in improving the capability to identify the types of natural disasters.The traditional infrasound classification mainly relies on machine learning algorithms after artificial feature extraction.However,guaranteeing the effectiveness of the extracted features is difficult.The current trend focuses on using a convolution neural network to automatically extract features for classification.This method can be used to extract signal spatial features automatically through a convolution kernel;however,infrasound signals contain not only spatial information but also temporal information when used as a time series.These extracted temporal features are also crucial.If only a convolution neural network is used,then the time dependence of the infrasound sequence will be missed.Using long short-term memory networks can compensate for the missing time-series features but induces spatial feature information loss of the infrasound signal.A multiscale squeeze excitation–convolution neural network–bidirectional long short-term memory network infrasound event classification fusion model is proposed in this study to address these problems.This model automatically extracted temporal and spatial features,adaptively selected features,and also realized the fusion of the two types of features.Experimental results showed that the classification accuracy of the model was more than 98%,thus verifying the effectiveness and superiority of the proposed model.

A New Industrial Intrusion Detection Method Based on CNN-BiLSTM

Nowadays,with the rapid development of industrial Internet technology,on the one hand,advanced industrial control systems(ICS)have improved industrial production efficiency.However,there are more and more cyber-attacks targeting industrial control systems.To ensure the security of industrial networks,intrusion detection systems have been widely used in industrial control systems,and deep neural networks have always been an effective method for identifying cyber attacks.Current intrusion detection methods still suffer from low accuracy and a high false alarm rate.Therefore,it is important to build a more efficient intrusion detection model.This paper proposes a hybrid deep learning intrusion detection method based on convolutional neural networks and bidirectional long short-term memory neural networks(CNN-BiLSTM).To address the issue of imbalanced data within the dataset and improve the model’s detection capabilities,the Synthetic Minority Over-sampling Technique-Edited Nearest Neighbors(SMOTE-ENN)algorithm is applied in the preprocessing phase.This algorithm is employed to generate synthetic instances for the minority class,simultaneously mitigating the impact of noise in the majority class.This approach aims to create a more equitable distribution of classes,thereby enhancing the model’s ability to effectively identify patterns in both minority and majority classes.In the experimental phase,the detection performance of the method is verified using two data sets.Experimental results show that the accuracy rate on the CICIDS-2017 data set reaches 97.7%.On the natural gas pipeline dataset collected by Lan Turnipseed from Mississippi State University in the United States,the accuracy rate also reaches 85.5%.

基于VMD-SSA-BiLSTM网络下的短期电力负荷预测

短期电力负荷预测是电力系统运控的重要部分,为提高负荷预测精度,针对实际负荷数据非线性、随机性等特征,建立了一种基于变分模态分解(VMD)下麻雀搜索算法(SSA)优化的双向长短期记忆网络(BiLSTM)的短期电力负荷预测模型。采用VMD对电力负荷数据进行分解,提取多个不同频率特征的模态分量,并引入SSA算法对BiLSTM网络参数进行优化,根据输入的模态分量建立SSA-BiLSTM预测模型进行预测。结果表明:相比于BiLSTM模型和VMD-BiLSTM模型,所建立的模型预测精度更高,拟合效果更好。

基于SSAE-IARO-BiLSTM的工业过程故障诊断研究

针对工业过程故障诊断精度低的问题,提出了一种基于栈式稀疏自编码网络(stacked sparse auto-encoder network, SSAE)和改进人工兔算法优化双向长短时记忆神经网络(improved artificial rabbit algorithm optimized bidirectional long short-term memory neural network, IARO-BiLSTM)的故障诊断方法。首先,利用SSAE网络强大的特征提取能力,实现对原始数据进行降维处理;其次,引入Circle混沌映射以达到丰富种群数量的目的,提出权重系数和Levy飞行机制改进人工兔算法的位置更新公式,提高人工兔算法的寻优能力,进而对BiLSTM网络的参数进行优化。最后,利用优化后的BiLSTM网络实现对故障的识别和分类。通过选取多组数据集进行验证,结果表明,基于SSAE-IARO-BiLSTM故障诊断方法能够准确地对故障进行识别和分类,且诊断准确率可达98%以上。