Slope stability prediction based on a long short-term memory neural network:comparisons with convolutional neural networks,support vector machines and random forest models

The numerical simulation and slope stability prediction are the focus of slope disaster research.Recently,machine learning models are commonly used in the slope stability prediction.However,these machine learning models have some problems,such as poor nonlinear performance,local optimum and incomplete factors feature extraction.These issues can affect the accuracy of slope stability prediction.Therefore,a deep learning algorithm called Long short-term memory(LSTM)has been innovatively proposed to predict slope stability.Taking the Ganzhou City in China as the study area,the landslide inventory and their characteristics of geotechnical parameters,slope height and slope angle are analyzed.Based on these characteristics,typical soil slopes are constructed using the Geo-Studio software.Five control factors affecting slope stability,including slope height,slope angle,internal friction angle,cohesion and volumetric weight,are selected to form different slope and construct model input variables.Then,the limit equilibrium method is used to calculate the stability coefficients of these typical soil slopes under different control factors.Each slope stability coefficient and its corresponding control factors is a slope sample.As a result,a total of 2160 training samples and 450 testing samples are constructed.These sample sets are imported into LSTM for modelling and compared with the support vector machine(SVM),random forest(RF)and convo-lutional neural network(CNN).The results show that the LSTM overcomes the problem that the commonly used machine learning models have difficulty extracting global features.Furthermore,LSTM has a better prediction performance for slope stability compared to SVM,RF and CNN models.

Device-Free Through-the-Wall Activity Recognition Using Bi-Directional Long Short-Term Memory and WiFi Channel State Information

Activity recognition plays a key role in health management and security.Traditional approaches are based on vision or wearables,which only work under the line of sight(LOS)or require the targets to carry dedicated devices.As human bodies and their movements have influences on WiFi propagation,this paper proposes the recognition of human activities by analyzing the channel state information(CSI)from the WiFi physical layer.The method requires only the commodity:WiFi transmitters and receivers that can operate through a wall,under LOS and non-line of sight(NLOS),while the targets are not required to carry dedicated devices.After collecting CSI,the discrete wavelet transform is applied to reduce the noise,followed by outlier detection based on the local outlier factor to extract the activity segment.Activity recognition is fulfilled by using the bi-directional long short-term memory that takes the sequential features into consideration.Experiments in through-the-wall environments achieve recognition accuracy>95%for six common activities,such as standing up,squatting down,walking,running,jumping,and falling,outperforming existing work in this field.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

融合CNN与BiLSTM模型的短期电能负荷预测

针对卷积神经网络(CNN)在捕捉预测序列间历史相关性方面的不足以及在变量复杂情况下出现的无法精准提取预测关键信息的问题,提出一种将双向长短期记忆网络(BiLSTM)与卷积神经网络结合的CNN-BiLSTM模型.首先,采用数据预处理方法保证数据的正确性和完整性,并对数据进行分析以探究多变量之间的相关性;其次,通过CNN与L1正则化对多维输入特征进行特征筛选,选取与预测相关的重要性特征向量;最后,使用BiLSTM对CNN输出的关键特征信息进行保存,形成向量与预测序列,并通过分析时序特征的潜在特点,提取用户的内在消费模式.实验比较了该模型与其他时序模型在不同时间分辨率下的预测效果,实验结果表明,CNN-BiLSTM模型在不同的回望时间间隔下表现出了最佳的预测性能,能够实现更好的短期负荷预测.

基于BERT-BiLSTM-CRF模型的油气领域命名实体识别

针对油气领域知识图谱构建过程中命名实体识别使用传统方法存在实体特征信息提取不准确、识别效率低的问题,提出了一种基于BERT-BiLSTM-CRF模型的命名实体识别研究方法。该方法首先利用BERT(bidirectional encoder representations from transformers)预训练模型得到输入序列语义的词向量;然后将训练后的词向量输入双向长短期记忆网络(bi-directional long short-term memory,BiLSTM)模型进一步获取上下文特征;最后根据条件随机场(conditional random fields,CRF)的标注规则和序列解码能力输出最大概率序列标注结果,构建油气领域命名实体识别模型框架。将BERT-BiLSTM-CRF模型与其他2种命名实体识别模型(BiLSTM-CRF、BiLSTM-Attention-CRF)在包括3万多条文本语料数据、4类实体的自建数据集上进行了对比实验。实验结果表明,BERT-BiLSTM-CRF模型的准确率(P)、召回率(R)和F_(1)值分别达到91.3%、94.5%和92.9%,实体识别效果优于其他2种模型。

CEEMDAN-CNN-BiLSTM混合模型矿区地表沉降预测

为了进一步发挥全球卫星导航系统(GNSS)实时监测优势,对时序数据中的潜藏特征与隐藏信息进行深度挖掘,提高地表沉降预测精度,提出基于自适应噪声完备集合经验模态分解(CEEMDAN)、卷积神经网络(CNN)和双向长短期记忆网络(BiLSTM)的CEEMDAN-CNN-BiLSTM混合地表沉降预测方法:以皖北某大型煤矿开采工作面与工业广场区域为验证对象,对比分析稳定区域和重点监测区域数据形态;然后基于CEEMDAN重构监测站高程数据分量,输入CNN模型提取分量隐含信息;最后构建BiLSTM模型,实现对沉降监测点位数据的短期预测。实验结果表明,相较于传统的CNN和长短期记忆模型,CEEMDAN-CNN-BiLSTM混合模型可有效降低预测误差,其中平均绝对百分比误差(MAPE)的降低范围为40%~90%,而均方根(RMS)误差的降低范围为52%~87%;该模型在时空特征捕捉和泛化能力方面表现性能较好,可为GNSS时间序列短期预测提供更为精准和可靠的解决方案。

DPAL-BERT:A Faster and Lighter Question Answering Model

Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.

Prediction of discharge in a tidal river using the LSTM-based sequence-to-sequence models

The complexity of river-tide interaction poses a significant challenge in predicting discharge in tidal rivers.Long short-term memory(LSTM)networks excel in processing and predicting crucial events with extended intervals and time delays in time series data.Additionally,the sequence-to-sequence(Seq2Seq)model,known for handling temporal relationships,adapting to variable-length sequences,effectively capturing historical information,and accommodating various influencing factors,emerges as a robust and flexible tool in discharge forecasting.In this study,we introduce the application of LSTM-based Seq2Seq models for the first time in forecasting the discharge of a tidal reach of the Changjiang River(Yangtze River)Estuary.This study focuses on discharge forecasting using three key input characteristics:flow velocity,water level,and discharge,which means the structure of multiple input and single output is adopted.The experiment used the discharge data of the whole year of 2020,of which the first 80%is used as the training set,and the last 20%is used as the test set.This means that the data covers different tidal cycles,which helps to test the forecasting effect of different models in different tidal cycles and different runoff.The experimental results indicate that the proposed models demonstrate advantages in long-term,mid-term,and short-term discharge forecasting.The Seq2Seq models improved by 6%-60%and 5%-20%of the relative standard deviation compared to the harmonic analysis models and improved back propagation neural network models in discharge prediction,respectively.In addition,the relative accuracy of the Seq2Seq model is 1%to 3%higher than that of the LSTM model.Analytical assessment of the prediction errors shows that the Seq2Seq models are insensitive to the forecast lead time and they can capture characteristic values such as maximum flood tide flow and maximum ebb tide flow in the tidal cycle well.This indicates the significance of the Seq2Seq models.

Comparison of Two Recurrent Neural Networks for Rainfall-Runoff Modeling in the Zou River Basin at Atchérigbé (Bénin)

Hydrological models are developed to simulate river flows over a watershed for many practical applications in the field of water resource management. The present paper compares the performance of two recurrent neural networks for rainfall-runoff modeling in the Zou River basin at Atchérigbé outlet. To this end, we used daily precipitation data over the period 1988-2010 as input of the models, such as the Long Short-Term Memory (LSTM) and Recurrent Gate Networks (GRU) to simulate river discharge in the study area. The investigated models give good results in calibration (R2 = 0.888, NSE = 0.886, and RMSE = 0.42 for LSTM;R2 = 0.9, NSE = 0.9 and RMSE = 0.397 for GRU) and in validation (R2 = 0.865, NSE = 0.851, and RMSE = 0.329 for LSTM;R2 = 0.9, NSE = 0.865 and RMSE = 0.301 for GRU). This good performance of LSTM and GRU models confirms the importance of models based on machine learning in modeling hydrological phenomena for better decision-making.

基于DRN-BiLSTM模型的矿井涌水量预测

针对矿井涌水量预测中存在的深度学习模型预测精度不高和适用性不强的问题,提出了一种基于深度残差网络(Deep Residual Network,DRN)和双向长短记忆网络(Bidirectional short and long memory network,BiLSTM)的矿井涌水量预测方法。首先,将矿井涌水量数据进行小波分解和归一化处理,得到趋势项数据和细节项数据;其次,采用DRN网络方法对趋势项数据进行预测,采用BiLSTM网络方法对细节项数据进行预测;最后,将2部分预测结果进行重构得到矿井涌水量预测结果。研究结果表明:DRN-BiLSTM模型相比于单一模型预测精度更高,说明该模型具有更好的泛化性。

Joint On-Demand Pruning and Online Distillation in Automatic Speech Recognition Language Model Optimization

Automatic speech recognition(ASR)systems have emerged as indispensable tools across a wide spectrum of applications,ranging from transcription services to voice-activated assistants.To enhance the performance of these systems,it is important to deploy efficient models capable of adapting to diverse deployment conditions.In recent years,on-demand pruning methods have obtained significant attention within the ASR domain due to their adaptability in various deployment scenarios.However,these methods often confront substantial trade-offs,particularly in terms of unstable accuracy when reducing the model size.To address challenges,this study introduces two crucial empirical findings.Firstly,it proposes the incorporation of an online distillation mechanism during on-demand pruning training,which holds the promise of maintaining more consistent accuracy levels.Secondly,it proposes the utilization of the Mogrifier long short-term memory(LSTM)language model(LM),an advanced iteration of the conventional LSTM LM,as an effective alternative for pruning targets within the ASR framework.Through rigorous experimentation on the ASR system,employing the Mogrifier LSTM LM and training it using the suggested joint on-demand pruning and online distillation method,this study provides compelling evidence.The results exhibit that the proposed methods significantly outperform a benchmark model trained solely with on-demand pruning methods.Impressively,the proposed strategic configuration successfully reduces the parameter count by approximately 39%,all the while minimizing trade-offs.

Research on Welding Quality Traceability Model of Offshore Platform Block Construction Process

Quality traceability plays an essential role in assembling and welding offshore platform blocks.The improvement of the welding quality traceability system is conducive to improving the durability of the offshore platform and the process level of the offshore industry.Currently,qualitymanagement remains in the era of primary information,and there is a lack of effective tracking and recording of welding quality data.When welding defects are encountered,it is difficult to rapidly and accurately determine the root cause of the problem from various complexities and scattered quality data.In this paper,a composite welding quality traceability model for offshore platform block construction process is proposed,it contains the quality early-warning method based on long short-term memory and quality data backtracking query optimization algorithm.By fulfilling the training of the early-warning model and the implementation of the query optimization algorithm,the quality traceability model has the ability to assist enterprises in realizing the rapid identification and positioning of quality problems.Furthermore,the model and the quality traceability algorithm are checked by cases in actual working conditions.Verification analyses suggest that the proposed early-warningmodel for welding quality and the algorithmfor optimizing backtracking requests are effective and can be applied to the actual construction process.

基于改进金豺算法的短期负荷预测

针对电力负荷序列波动性和预测精度不高的问题,提出一种基于变分模态分解、排列熵和改进金豺算法优化双向长短期记忆网络的预测模型。首先,利用变分模态分解重构原始负荷序列,再采用排列熵理论对分解后的子序列进行熵值重组;然后,利用改进金豺算法对双向长短期记忆网络的参数进行优化,并对每个子序列建立预测模型;最后,组合各模型结果得到最终预测值。实验结果表明,本文模型预测精度更高,与真实值拟合度更好。

基于MacBERT与对抗训练的机器阅读理解模型

机器阅读理解旨在让机器像人类一样理解自然语言文本,并据此进行问答任务。近年来,随着深度学习和大规模数据集的发展,机器阅读理解引起了广泛关注,但是在实际应用中输入的问题通常包含各种噪声和干扰,这些噪声和干扰会影响模型的预测结果。为了提高模型的泛化能力和鲁棒性,提出一种基于掩码校正的来自Transformer的双向编码器表示(Mac BERT)与对抗训练(AT)的机器阅读理解模型。首先利用Mac BERT对输入的问题和文本进行词嵌入转化为向量表示;然后根据原始样本反向传播的梯度变化在原始词向量上添加微小扰动生成对抗样本;最后将原始样本和对抗样本输入双向长短期记忆(Bi LSTM)网络进一步提取文本的上下文特征,输出预测答案。实验结果表明,该模型在简体中文数据集CMRC2018上的F1值和精准匹配(EM)值分别较基线模型提高了1.39和3.85个百分点,在繁体中文数据集DRCD上的F1值和EM值分别较基线模型提高了1.22和1.71个百分点,在英文数据集SQu ADv1.1上的F1值和EM值分别较基线模型提高了2.86和1.85个百分点,优于已有的大部分机器阅读理解模型,并且在真实问答结果上与基线模型进行对比,结果验证了该模型具有更强的鲁棒性和泛化能力,在输入的问题存在噪声的情况下性能更好。

整合BiLSTM-CRF网络和词典资源的中文电子病历实体识别

[目的/意义]通过整合BiLSTM-CRF神经网络和具有先验领域知识的词典资源,提高中文电子病历领域中的实体识别效果。[方法/过程]采用BiLSTM-CRF神经网络模型,以CCKS-2017测评任务提供的脱敏中文电子病历数据为实验数据集,结合Word2Vec和外部词典构造神经网络的词嵌入输入改进实体识别模型。[结果/结论]与传统的CRF和单纯的BiLSTM-CRF模型相比,引入先验知识的词典资源可以取得更好的实体识别效果,F1值达到最高的90.41%。深度学习模型BiLSTM-CRF能够显著提升传统CRF方法的实体识别效果,同时先验的词典知识能进一步增强神经网络的性能。

基于XLNET模型的开阳磷矿成矿条件相关地质实体识别与应用

随着磷矿找矿难度越来越大,地质勘探成果报告也愈来愈多,通过人工识别海量文档中与磷矿成矿相关地质信息耗时低效,无法满足知识共享传播和地质报告智能管理的需求。为快速获得磷矿地质文档报告中隐藏的成矿地质知识,基于XLNET模型建立了磷矿成矿地质实体自动识别的方法。首先对实体进行BIO标注建立地质实体字典,利用XLNET作为底层预处理模型学习句子双向语义;然后使用BILSTM-Attention-CRF模型实现文本多标签的智能分类;最后通过定位磷矿实体在报告中的分布位置大致推测该处磷矿成矿条件和成矿模式。将该模型与其余3种模型比较得出结果,该模型识别的准确率(P)、召回率(R)及F1值都接近了90%,较前3种模型分别调高了2%,5%,6%。该研究为开阳磷矿地质研究人员提供了更加高效的地质实体自动识别的方法。

面向行政执法案件文本的事件抽取研究

行政执法的智能化水平是国家治理能力现代化的体现,数据是智能化发展的重要依托。在行政执法领域,各行政机关存储大量以文本形式记录的历史案件,这种非结构化的数据价值密度较低、可利用性不强。利用事件抽取技术从行政执法案件文本中快速高效地抽取案件职权类型、案发时间、案发地点等结构化信息,可推动行政机关对历史案件信息的利用和智能化执法办案研究。收集整理某城市的真实案例数据,并通过人工标注构建一个行政执法领域的数据集,根据行政执法案件文本的无触发词、文档级、格式不固定等文本特征,提出结合基于Transformer的双向编码器表示(BERT)和基于条件随机场的双向长短期记忆网络(BiLSTM-CRF)模型的两阶段事件抽取方法,通过文本多分类和序列标注依次完成事件类型检测和事件论元抽取任务。实验结果表明,事件类型检测任务的F1值达到99.54%,事件论元抽取任务的F1值达到97.36%,实现了对案件信息的有效抽取。

基于深度学习的教材德目教育文本分类方法

对上海中小学教材德目教育文本分类进行研究,提出了基于转换器的双向编码表征(BERT)预训练模型、双向长短期记忆(BiLSTM)网络和注意力机制的模型IoMET_BBA.通过合成少数类过采样技术(SMOTE)与探索性数据分析(EDA)技术进行数据增强,使用BERT模型生成富含语境信息的语义向量,通过BiLSTM提取特征,并结合注意力机制来获得词语权重信息,通过全连接层进行分类.对比实验的结果表明,IoMET_BBA的F1度量值达到了86.14%,优于其他模型,可以精确地评估教材德目教育文本.

基于上下文知识增强型Transformer网络的抑郁检测

抑郁症作为一种常见的心理健康问题,严重影响人们的日常生活甚至是生命安全。鉴于目前的抑郁症检测存在主观性和人工干预等缺点,基于深度学习的自动检测方式成为热门研究方向。对于最易获取的文本模态而言,主要的挑战在于如何建模抑郁文本中的长距离依赖与序列依赖。为解决该问题,提出一种基于上下文知识的增强型Transformer网络模型RoBERTa-BiLSTM,旨在从抑郁文本序列中充分提取和利用上下文特征。结合序列模型与Transformer模型优点,建模单词间上下文交互,为抑郁类别揭示与信息表征提供参考。首先,利用RoBERTa方法将词汇嵌入到语义向量空间;其次,利用双向长短期记忆网络(BiLSTM)模型有效捕获长距离上下文语义;最后,在DAIC-WOZ和EATD-Corpus 2个大规模数据集上进行实证研究。实验结果显示,RoBERTa-BiLSTM模型的准确率分别达到0.74和0.93以上,召回率分别达到0.66和0.56以上,能够准确地检测抑郁症。

融合MAML与BiLSTM的微博负面情感多分类方法

随着社交网络的不断发展,微博成为人们日常生活中分享观点和感情的重要平台,分析用户的情感倾向可以有效地应用于舆情控制、民意调查、商品推荐等工作。传统的深度学习算法在面对新的工作任务时,往往需要大量数据重新训练才能得到较好准确率。针对这一情况,提出了一种基于MAML(model-agnostic meta-learning)与BiLSTM(双向长短时记忆网络)的微博负面情感多分类方法。对微博文本进行词向量化表示,构建MAML与BiLSTM结合的模型,其中BiLSTM实现对微博负面情感的分类,通过随机梯度下降更新参数;MAML中的元学习器则通过计算多次训练的损失总和,进行第二次梯度下降,更新元学习器参数。通过更新后得到的元学习器可以在面对新的微博负面情感分类任务时快速迭代。实验结果表明:相较于目前流行的模型,在微博负面情感数据集上,准确率、召回率和F1值分别提高了1.68个百分点、2.86个百分点和2.27个百分点。