Integrating Transformer and Bidirectional Long Short-Term Memory for Intelligent Breast Cancer Detection from Histopathology Biopsy Images

Breast cancer is a significant threat to the global population,affecting not only women but also a threat to the entire population.With recent advancements in digital pathology,Eosin and hematoxylin images provide enhanced clarity in examiningmicroscopic features of breast tissues based on their staining properties.Early cancer detection facilitates the quickening of the therapeutic process,thereby increasing survival rates.The analysis made by medical professionals,especially pathologists,is time-consuming and challenging,and there arises a need for automated breast cancer detection systems.The upcoming artificial intelligence platforms,especially deep learning models,play an important role in image diagnosis and prediction.Initially,the histopathology biopsy images are taken from standard data sources.Further,the gathered images are given as input to the Multi-Scale Dilated Vision Transformer,where the essential features are acquired.Subsequently,the features are subjected to the Bidirectional Long Short-Term Memory(Bi-LSTM)for classifying the breast cancer disorder.The efficacy of the model is evaluated using divergent metrics.When compared with other methods,the proposed work reveals that it offers impressive results for detection.

Landslide displacement prediction based on optimized empirical mode decomposition and deep bidirectional long short-term memory network

There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement and time series of a landslide.The second one is the dynamic evolution of a landslide,which could not be feasibly simulated simply by traditional prediction models.In this paper,a dynamic model of displacement prediction is introduced for composite landslides based on a combination of empirical mode decomposition with soft screening stop criteria(SSSC-EMD)and deep bidirectional long short-term memory(DBi-LSTM)neural network.In the proposed model,the time series analysis and SSSC-EMD are used to decompose the observed accumulated displacements of a slope into three components,viz.trend displacement,periodic displacement,and random displacement.Then,by analyzing the evolution pattern of a landslide and its key factors triggering landslides,appropriate influencing factors are selected for each displacement component,and DBi-LSTM neural network to carry out multi-datadriven dynamic prediction for each displacement component.An accumulated displacement prediction has been obtained by a summation of each component.For accuracy verification and engineering practicability of the model,field observations from two known landslides in China,the Xintan landslide and the Bazimen landslide were collected for comparison and evaluation.The case study verified that the model proposed in this paper can better characterize the"stepwise"deformation characteristics of a slope.As compared with long short-term memory(LSTM)neural network,support vector machine(SVM),and autoregressive integrated moving average(ARIMA)model,DBi-LSTM neural network has higher accuracy in predicting the periodic displacement of slope deformation,with the mean absolute percentage error reduced by 3.063%,14.913%,and 13.960%respectively,and the root mean square error reduced by 1.951 mm,8.954 mm and 7.790 mm respectively.Conclusively,this model not only has high prediction accuracy but also is more stable,which can provide new insight for practical landslide prevention and control engineering.

A real-time prediction method for tunnel boring machine cutter-head torque using bidirectional long short-term memory networks optimized by multi-algorithm

Based on data from the Jilin Water Diversion Tunnels from the Songhua River(China),an improved and real-time prediction method optimized by multi-algorithm for tunnel boring machine(TBM)cutter-head torque is presented.Firstly,a function excluding invalid and abnormal data is established to distinguish TBM operating state,and a feature selection method based on the SelectKBest algorithm is proposed.Accordingly,ten features that are most closely related to the cutter-head torque are selected as input variables,which,in descending order of influence,include the sum of motor torque,cutter-head power,sum of motor power,sum of motor current,advance rate,cutter-head pressure,total thrust force,penetration rate,cutter-head rotational velocity,and field penetration index.Secondly,a real-time cutterhead torque prediction model’s structure is developed,based on the bidirectional long short-term memory(BLSTM)network integrating the dropout algorithm to prevent overfitting.Then,an algorithm to optimize hyperparameters of model based on Bayesian and cross-validation is proposed.Early stopping and checkpoint algorithms are integrated to optimize the training process.Finally,a BLSTMbased real-time cutter-head torque prediction model is developed,which fully utilizes the previous time-series tunneling information.The mean absolute percentage error(MAPE)of the model in the verification section is 7.3%,implying that the presented model is suitable for real-time cutter-head torque prediction.Furthermore,an incremental learning method based on the above base model is introduced to improve the adaptability of the model during the TBM tunneling.Comparison of the prediction performance between the base and incremental learning models in the same tunneling section shows that:(1)the MAPE of the predicted results of the BLSTM-based real-time cutter-head torque prediction model remains below 10%,and both the coefficient of determination(R^(2))and correlation coefficient(r)between measured and predicted values exceed 0.95;and(2)the incremental learning method is suitable for realtime cutter-head torque prediction and can effectively improve the prediction accuracy and generalization capacity of the model during the excavation process.

Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis

A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deeplearning algorithm based on a fully convolutional neural network(FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory(LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.

Power entity recognition based on bidirectional long short-term memory and conditional random fields

With the application of artificial intelligence technology in the power industry,the knowledge graph is expected to play a key role in power grid dispatch processes,intelligent maintenance,and customer service response provision.Knowledge graphs are usually constructed based on entity recognition.Specifically,based on the mining of entity attributes and relationships,domain knowledge graphs can be constructed through knowledge fusion.In this work,the entities and characteristics of power entity recognition are analyzed,the mechanism of entity recognition is clarified,and entity recognition techniques are analyzed in the context of the power domain.Power entity recognition based on the conditional random fields (CRF) and bidirectional long short-term memory (BLSTM) models is investigated,and the two methods are comparatively analyzed.The results indicated that the CRF model,with an accuracy of 83%,can better identify the power entities compared to the BLSTM.The CRF approach can thus be applied to the entity extraction for knowledge graph construction in the power field.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

A forecasting model for wave heights based on a long short-term memory neural network

To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with sea surface wind and wave heights as training samples.The prediction performance of the model is evaluated,and the error analysis shows that when using the same set of numerically predicted sea surface wind as input,the prediction error produced by the proposed LSTM model at Sta.N01 is 20%,18%and 23%lower than the conventional numerical wave models in terms of the total root mean square error(RMSE),scatter index(SI)and mean absolute error(MAE),respectively.Particularly,for significant wave height in the range of 3–5 m,the prediction accuracy of the LSTM model is improved the most remarkably,with RMSE,SI and MAE all decreasing by 24%.It is also evident that the numbers of hidden neurons,the numbers of buoys used and the time length of training samples all have impact on the prediction accuracy.However,the prediction does not necessary improve with the increase of number of hidden neurons or number of buoys used.The experiment trained by data with the longest time length is found to perform the best overall compared to other experiments with a shorter time length for training.Overall,long short-term memory neural network was proved to be a very promising method for future development and applications in wave forecasting.

Preliminary abnormal electrocardiogram segment screening method for Holter data based on long short-term memory networks

Holter usually monitors electrocardiogram(ECG)signals for more than 24 hours to capture short-lived cardiac abnormalities.In view of the large amount of Holter data and the fact that the normal part accounts for the majority,it is reasonable to design an algorithm that can automatically eliminate normal data segments as much as possible without missing any abnormal data segments,and then take the left segments to the doctors or the computer programs for further diagnosis.In this paper,we propose a preliminary abnormal segment screening method for Holter data.Based on long short-term memory(LSTM)networks,the prediction model is established and trained with the normal data of a monitored object.Then,on the basis of kernel density estimation,we learn the distribution law of prediction errors after applying the trained LSTM model to the regular data.Based on these,the preliminary abnormal ECG segment screening analysis is carried out without R wave detection.Experiments on the MIT-BIH arrhythmia database show that,under the condition of ensuring that no abnormal point is missed,53.89% of normal segments can be effectively obviated.This work can greatly reduce the workload of subsequent further processing.

Dynamic prediction of landslide displacement using singular spectrum analysis and stack long short-term memory network

An accurate landslide displacement prediction is an important part of landslide warning system. Aiming at the dynamic characteristics of landslide evolution and the shortcomings of traditional static prediction models, this paper proposes a dynamic prediction model of landslide displacement based on singular spectrum analysis(SSA) and stack long short-term memory(SLSTM) network. The SSA is used to decompose the landslide accumulated displacement time series data into trend term and periodic term displacement subsequences. A cubic polynomial function is used to predict the trend term displacement subsequence, and the SLSTM neural network is used to predict the periodic term displacement subsequence. At the same time, the Bayesian optimization algorithm is used to determine that the SLSTM network input sequence length is 12 and the number of hidden layer nodes is 18. The SLSTM network is updated by adding predicted values to the training set to achieve dynamic displacement prediction. Finally, the accumulated landslide displacement is obtained by superimposing the predicted value of each displacement subsequence. The proposed model was verified on the Xintan landslide in Hubei Province, China. The results show that when predicting the displacement of the periodic term, the SLSTM network has higher prediction accuracy than the support vector machine(SVM) and auto regressive integrated moving average(ARIMA). The mean relative error(MRE) is reduced by 4.099% and 3.548% respectively, while the root mean square error(RMSE) is reduced by 5.830 mm and 3.854 mm respectively. It is concluded that the SLSTM network model can better simulate the dynamic characteristics of landslides.

GNSS拒止时基于并行CNN-BiLSTM回归和残差补偿的UAV导航误差校正方法

全球导航卫星系统(GNSS)拒止时,GNSS/惯性导航系统(INS)组合导航系统的性能严重下降,导致无人机集群导航误差快速发散.目前,利用神经网络预测位置与速度代替GNSS导航信息可校正无人机INS误差,但该方法仍存在定位误差较高且在轨迹突变时预测精度急剧下降的问题.因此,提出了一种基于卷积-双向长短时记忆网络联合残差补偿的位置与速度预测方法,用于提高位置与速度预测精度.首先,针对GNSS拒止后GNSS/INS组合导航系统定位误差较高的问题,提出卷积神经网络(CNN)与双向长短时记忆网络(BiLSTM)的融合模型,该模型可建立惯性测量单元(IMU)动力学测量数据与GNSS导航信息之间的关系,实现较准确的位置和速度预测.其次,针对轨迹突变时预测效果急剧下降的问题,提出并行CNNBiLSTM回归架构,在预测位置与速度的同时,挖掘IMU动力学测量数据、预测值与预测残差之间的关系,预测并补偿预测残差,增强模型在轨迹突变时的预测精度.仿真结果表明,所提模型在预测准确性、有效性和稳定性方面都优于CNN-LSTM、LSTM网络模型.

基于自注意力机制和改进的K-BiLSTM的水产养殖水体溶解氧含量预测模型

为精确预测水产养殖水体溶解氧含量,本研究提出一种基于自注意力机制(ATTN)和改进的K-means聚类-基于残差和批标准化(BN)的双向长短期记忆网络(BiLSTM)的水产养殖水体溶解氧含量预测模型。首先,根据环境数据的相似性,使用改进的K-means算法将数据划分成若干个类别;然后,在BiLSTM基础上构建残差连接和加入BN完成高层次特征提取,利用BiLSTM的长期记忆能力保存特征信息;最后,引入自注意力机制突出不同时间节点数据特征的重要性,进一步提升模型的性能。试验结果表明,本研究提出的基于自注意力机制和改进的K-BiLSTM模型的平均绝对误差为0.238、均方根误差为0.322、平均绝对百分比误差为0.035,与单一的BP模型、CNN-LSTM模型、传统的K-means-基于残差和BN的BiLSTM-ATTN等模型相比具有更优的预测性能和泛化能力。

基于VMD-FE-CNN-BiLSTM的短期光伏发电功率预测

为提高光伏功率的预测精度,提出一种变分模态分解(VMD)、模糊熵(FE)、卷积神经网络(CNN)和双向长短期记忆神经网络(BiLSTM)的光伏功率组合预测模型。该方法首先采用VMD将原始光伏序列数据分解成多个子序列,从而减少随机波动分量和噪声干扰对预测模型的影响,通过FE对每个子序列进行重组,使用一维CNN的局部连接及权值共享提取不同分量的特征,将CNN输出的特征融合并输入到BiLSTM模型中;利用BiLSTM模型建立历史数据之间的时间特征关系,得到光伏发电功率预测结果。与BiLSTM、CNN-BiLSTM、EEMD-CNN-BiLSTM、VMD-CNN-BiLSTM这4种模型进行比较,该文提出的VMD-FE-CNN-BiLSTM模型在光伏发电功率预测中具有较高的精确度和稳定性,满足光伏发电短期预测的要求。

基于BERT-BiLSTM-CRF模型的畜禽疫病文本分词研究

针对畜禽疫病文本语料匮乏、文本内包含大量疫病名称及短语等未登录词问题,提出了一种结合词典匹配的BERT-BiLSTM-CRF畜禽疫病文本分词模型。以羊疫病为研究对象,构建了常见疫病文本数据集,将其与通用语料PKU结合,利用BERT(Bidirectional encoder representation from transformers)预训练语言模型进行文本向量化表示;通过双向长短时记忆网络(Bidirectional long short-term memory network,BiLSTM)获取上下文语义特征;由条件随机场(Conditional random field,CRF)输出全局最优标签序列。基于此,在CRF层后加入畜禽疫病领域词典进行分词匹配修正,减少在分词过程中出现的疫病名称及短语等造成的歧义切分,进一步提高了分词准确率。实验结果表明,结合词典匹配的BERT-BiLSTM-CRF模型在羊常见疫病文本数据集上的F1值为96.38%,与jieba分词器、BiLSTM-Softmax模型、BiLSTM-CRF模型、未结合词典匹配的本文模型相比,分别提升11.01、10.62、8.3、0.72个百分点,验证了方法的有效性。与单一语料相比,通用语料PKU和羊常见疫病文本数据集结合的混合语料,能够同时对畜禽疫病专业术语及疫病文本中常用词进行准确切分,在通用语料及疫病文本数据集上F1值都达到95%以上,具有较好的模型泛化能力。该方法可用于畜禽疫病文本分词。

基于BiLSTM-XGBoost混合模型的储层岩性识别

储层岩性分类是地质研究基础,基于数据驱动的机器学习模型虽然能较好地识别储层岩性,但由于测井数据是特殊的序列数据,模型很难有效提取数据的空间相关性,造成模型对储层识别仍存在不足.针对此问题,本文结合双向长短期循环神经网络(bidirectional long short-term memory,BiLSTM)和极端梯度提升决策树(extreme gradient boosting decision tree,XGBoost),提出双向记忆极端梯度提升(BiLSTM-XGBoost,BiXGB)模型预测储层岩性.该模型在传统XGBoost基础上融入了BiLSTM,大大增强了模型对测井数据的特征提取能力.BiXGB模型使用BiLSTM对测井数据进行特征提取,将提取到的特征传递给XGBoost分类模型进行训练和预测.将BiXGB模型应用于储层岩性数据集时,模型预测的总体精度达到了91%.为了进一步验证模型的准确性和稳定性,将模型应用于UCI公开的Occupancy序列数据集,结果显示模型的预测总体精度也高达93%.相较于其他机器学习模型,BiXGB模型能准确地对序列数据进行分类,提高了储层岩性的识别精度,满足了油气勘探的实际需要,为储层岩性识别提供了新的方法.

基于BERT+CNN_BiLSTM的列控车载设备故障诊断

列控车载设备作为列车运行控制系统核心设备,在高速列车运行过程中发挥着重要作用。目前,其故障诊断仅依赖于现场作业人员经验,诊断效率相对较低。为了实现列控车载设备故障自动诊断并提高诊断效率,提出了BERT+CNN_BiLSTM故障诊断模型。首先,使用来自变换器的双向编码器表征量(Bidirectional encoder representations from transformers,BERT)模型将应用事件日志(Application event log,AElog)转换为计算机能够识别的可以挖掘语义信息的文本向量表示。其次,分别利用卷积神经网络(Convolutional neural network,CNN)和双向长短时记忆网络(Bidirectional long short-term memory,BiLSTM)提取故障特征并进行组合,从而增强空间和时序能力。最后,利用Softmax实现列控车载设备的故障分类与诊断。实验中,选取一列实际运行的列车为研究对象,以运行过程中产生的AElog日志作为实验数据来验证BERT+CNN_BiLSTM模型的性能。与传统机器学习算法、BERT+BiLSTM模型和BERT+CNN模型相比,BERT+CNN_BiLSTM模型的准确率、召回率和F1分别为92.27%、91.03%和91.64%,表明该模型在高速列车控制系统故障诊断中性能优良。

基于VMD-BiLSTM-WOA的短期风电功率预测

风力发电对于解决全球能源短缺问题有重要意义,准确预测风电功率有助于风电并网的合理调度和可靠的电网运行.文章提出了一种基于变分模态分解(Variational Mode Decomposition, VMD)、双向长短期记忆网络(Bidirectional Long Short-term Memory Network, BiLSTM)以及鲸鱼优化算法(Whale Optimization Algorithm, WOA)的混合深度学习模型,以用于短期风电功率预测.首先,VMD将原始风电功率分解为多个子模态,有效减少了序列的波动性;然后对每个子模态分别建立BiLSTM模型,使用WOA对BiLSTM中的参数进行优化,以提高混合模型的效率和预测性能;最后将各个子模型的结果叠加得到最终预测结果.在实验中通过建立不同的比较模型来说明改进策略的有效性和优越性,结果表明所提的混合模型在风电功率预测中具有较高的预测精度.

基于RF-BiLSTM模型的河流水质预测

水环境中过量的氮、磷和高锰酸盐会对流域造成严重污染,准确预测这三类指标的含量对流域污染治理具有重要意义。然而,现有的模型预测精度低,输入因子的选择缺乏数理依据。基于此,以邕江为研究区域,提出一种RF-BiLSTM的混合网络模型。该模型具有利用RF算法提取水质指标最优特征和利用BiLSTM模型提取输入数据的时间特征的优势,采用先降维后预测的方式对TN、TP和COD Mn进行预测,并将深度学习中的CNN、LSTM、BiLSTM和RF-LSTM作为基准模型与本研究所提模型作对比研究。研究结果表明,本研究模型预测TN、TP和COD Mn的平均绝对百分比误差(MAPE)分别达到了4.330%、6.781%和7.384%,均低于其他基准模型,预测结果具有较高的准确性和实用性,可为水环境的污染治理提供有效的技术支持。

基于改进BiLSTM网络的地铁车轮磨耗预测模型

针对地铁车轮磨耗数据时间跨度较长引起的长期依赖问题,为了进一步提升预测精度,提出一种将麻雀搜索算法(sparrow search algorithm,SSA)优化双向长短期记忆网络(bidirectional long short term memory,Bi LSTM)的改进BiLSTM(SSA-BiLSTM)网络模型,用于地铁车轮磨耗预测。首先,利用麻雀搜索算法对双向长短期记忆网络算法的神经元个数、迭代次数、输入批量和学习率等超参数在给定范围内进行寻优,得到参数最优值;然后,以参数最优值来构建改进BiLSTM网络模型,对车轮磨耗进行预测分析;最后,以车轮踏面磨耗和轮缘磨耗作为研究对象,将某地铁1车厢1号车轮的现场实测历史磨耗数据作为输入,对该模型进行训练及验证分析,并与多层感知机(multilayer perceptron,MLP)、LSTM、BiLSTM以及SSA-LSTM模型的预测结果进行对比。研究结果表明:SSA-Bi-LSTM模型的车轮磨耗预测精度更高,与LSTM、BiLSTM以及SSA-LSTM网络模型相比,踏面磨耗的平均绝对百分误差(mean absolute percentage error,MAPE)分别降低了13.28%、10.32%、1.47%,轮缘磨耗分别降低了9.5%、0.46%、0.02%;分别对同一地铁2号、4号车厢的1号位置车轮磨耗进行预测,并与磨耗实测数据进行对比,踏面磨耗的平均绝对百分比误差分别为1.34%、1.42%,轮缘磨耗的平均绝对百分比误差分别为0.18%、0.19%,验证了本文所提模型具有良好的泛化性,为地铁轮对智能化管理提供理论支持,延长车轮使用寿命。

基于Bo-BiLSTM网络的IGBT老化失效预测方法

针对绝缘栅双极型晶体管(IGBT)受热应力冲击后对其进行老化失效预测精度不高的情况,提出一种基于贝叶斯优化(Bo)-双向长短期记忆(BiLSTM)网络的IGBT老化失效预测方法。首先分析IGBT模块老化失效原理,然后基于NASA老化实验数据集建立失效特征数据库,最后利用Matlab软件构造Bo-BiLSTM网络预测失效特征参数数据。选取常用回归预测性能评估指标将长短期记忆(LSTM)网络模型、BiLSTM网络模型与Bo-BiLSTM网络模型的预测结果进行对比分析。结果表明,Bo-BiLSTM网络的模型拟合精度更高,基于Bo-BiLSTM网络的IGBT老化失效预测方法具有较好的预测效果,能够应用于IGBT的失效预测。

结合Word2vec和BiLSTM的民航非计划事件分析方法

安全是民航业的核心主题。针对目前民航非计划事件分析严重依赖专家经验及分析效率低下的问题,文章提出一种结合Word2vec和双向长短期记忆(bidirectional long short-term memory,BiLSTM)神经网络模型的民航非计划事件分析方法。首先采用Word2vec模型针对事件文本语料进行词向量训练,缩小空间向量维度;然后通过BiLSTM模型自动提取特征,获取事件文本的完整序列信息和上下文特征向量;最后采用softmax函数对民航非计划事件进行分类。实验结果表明,所提出的方法分类效果更好,能达到更优的准确率和F 1值,对不平衡数据样本同样具有较稳定的分类性能,证明了该方法在民航非计划事件分析上的适用性和有效性。