Estimation of unloading relaxation depth of Baihetan Arch Dam foundation using long-short term memory network

The unloading relaxation caused by excavation for construction of high arch dams is an important factor influencing the foundation’s integrity and strength.To evaluate the degree of unloading relaxation,the long-short term memory(LSTM)network was used to estimate the depth of unloading relaxation zones on the left bank foundation of the Baihetan Arch Dam.Principal component analysis indicates that rock charac-teristics,the structural plane,the protection layer,lithology,and time are the main factors.The LSTM network results demonstrate the unloading relaxation characteristics of the left bank,and the relationships with the factors were also analyzed.The structural plane has the most significant influence on the distribution of unloading relaxation zones.Compared with massive basalt,the columnar jointed basalt experiences a more significant unloading relaxation phenomenon with a clear time effect,with the average unloading relaxation period being 50 d.The protection layer can effectively reduce the unloading relaxation depth by approximately 20%.

Online multi-target intelligent tracking using a deep long-short term memory network

Multi-target tracking is facing the difficulties of modeling uncertain motion and observation noise.Traditional tracking algorithms are limited by specific models and priors that may mismatch a real-world scenario.In this paper,considering the model-free purpose,we present an online Multi-Target Intelligent Tracking(MTIT)algorithm based on a Deep Long-Short Term Memory(DLSTM)network for complex tracking requirements,named the MTIT-DLSTM algorithm.Firstly,to distinguish trajectories and concatenate the tracking task in a time sequence,we define a target tuple set that is the labeled Random Finite Set(RFS).Then,prediction and update blocks based on the DLSTM network are constructed to predict and estimate the state of targets,respectively.Further,the prediction block can learn the movement trend from the historical state sequence,while the update block can capture the noise characteristic from the historical measurement sequence.Finally,a data association scheme based on Hungarian algorithm and the heuristic track management strategy are employed to assign measurements to targets and adapt births and deaths.Experimental results manifest that,compared with the existing tracking algorithms,our proposed MTIT-DLSTM algorithm can improve effectively the accuracy and robustness in estimating the state of targets appearing at random positions,and be applied to linear and nonlinear multi-target tracking scenarios.

Short-TermWind Power Prediction Based on Combinatorial Neural Networks

Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.

Conditional Random Field Tracking Model Based on a Visual Long Short Term Memory Network

In dense pedestrian tracking,frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories.In this study,a conditional random field tracking model is established by using a visual long short term memory network in the three-dimensional(3D)space and the motion estimations jointly performed on object trajectory segments.Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation.To address the uncertainty of the length and interval of trajectory segments,a multimode long short term memory network is proposed for the object motion estimation.The tracking performance is evaluated using the PETS2009 dataset.The experimental results show that the proposed method achieves better performance than the tracking methods based on the independent motion estimation.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

ART-2 neural network based on eternal term memory vector:Architecture and algorithm

Aimed at the problem that the traditional ART-2 neural network can not recognize a gradually changing course, an eternal term memory (ETM) vector is introduced into ART-2 to simulate the function of human brain, i.e. the deep remembrance for the initial impression.. The eternal term memory vector is determined only by the initial vector that establishes category neuron node and is used to keep the remembrance for this vector for ever. Two times of vigilance algorithm are put forward, and the posterior input vector must first pass the first vigilance of this eternal term memory vector, only succeeded has it the qualification to begin the second vigilance of long term memory vector. The long term memory vector can be revised only when both of the vigilances are passed. Results of recognition examples show that the improved ART-2 overcomes the defect of traditional ART-2 and can recognize a gradually changing course effectively.

Memory Analysis for Memristors and Memristive Recurrent Neural Networks

Traditional recurrent neural networks are composed of capacitors, inductors, resistors, and operational amplifiers.Memristive neural networks are constructed by replacing resistors with memristors. This paper focuses on the memory analysis,i.e. the initial value computation, of memristors. Firstly, we present the memory analysis for a single memristor based on memristors’ mathematical models with linear and nonlinear drift.Secondly, we present the memory analysis for two memristors in series and parallel. Thirdly, we point out the difference between traditional neural networks and those that are memristive. Based on the current and voltage relationship of memristors, we use mathematical analysis and SPICE simulations to demonstrate the validity of our methods.

A Novel MegaBAT Optimized Intelligent Intrusion Detection System in Wireless Sensor Networks

Wireless Sensor Network(WSN),whichfinds as one of the major components of modern electronic and wireless systems.A WSN consists of numerous sensor nodes for the discovery of sensor networks to leverage features like data sensing,data processing,and communication.In thefield of medical health care,these network plays a very vital role in transmitting highly sensitive data from different geographic regions and collecting this information by the respective network.But the fear of different attacks on health care data typically increases day by day.In a very short period,these attacks may cause adversarial effects to the WSN nodes.Furthermore,the existing Intrusion Detection System(IDS)suffers from the drawbacks of limited resources,low detection rate,and high computational overhead and also increases the false alarm rates in detecting the different attacks.Given the above-mentioned problems,this paper proposes the novel MegaBAT optimized Long Short Term Memory(MBOLT)-IDS for WSNs for the effective detection of different attacks.In the proposed framework,hyperpara-meters of deep Long Short-Term Memory(LSTM)were optimized by the meta-heuristic megabat algorithm to obtain a low computational overhead and high performance.The experimentations have been carried out using(Wireless Sensor NetworkDetection System)WSN-DS datasets and performance metrics such as accuracy,recall,precision,specificity,and F1-score are calculated and compared with the other existing intelligent IDS.The proposed framework provides outstanding results in detecting the black hole,gray hole,scheduling,flooding attacks and significantly reduces the time complexity,which makes this system suitable for resource-constraint WSNs.

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.

Short-Term Relay Quality Prediction Algorithm Based on Long and Short-Term Memory

The fraction defective of semi-finished products is predicted to optimize the process of relay production lines, by which production quality and productivity are increased, and the costs are decreased. The process parameters of relay production lines are studied based on the long-and-short-term memory network. Then, the Keras deep learning framework is utilized to build up a short-term relay quality prediction algorithm for the semi-finished product. A simulation model is used to study prediction algorithm. The simulation results show that the average prediction absolute error of the fraction is less than 5%. This work displays great application potential in the relay production lines.

Feature identification in complex fluid flows by convolutional neural networks

Recent advancements have established machine learning's utility in predicting nonlinear fluid dynamics,with predictive accuracy being a central motivation for employing neural networks.However,the pattern recognition central to the networks function is equally valuable for enhancing our dynamical insight into the complex fluid dynamics.In this paper,a single-layer convolutional neural network(CNN)was trained to recognize three qualitatively different subsonic buffet flows(periodic,quasi-periodic and chaotic)over a high-incidence airfoil,and a near-perfect accuracy was obtained with only a small training dataset.The convolutional kernels and corresponding feature maps,developed by the model with no temporal information provided,identified large-scale coherent structures in agreement with those known to be associated with buffet flows.Sensitivity to hyperparameters including network architecture and convolutional kernel size was also explored.The coherent structures identified by these models enhance our dynamical understanding of subsonic buffet over high-incidence airfoils over a wide range of Reynolds numbers.

Soil NOx Emission Prediction via Recurrent Neural Networks

This paper presents designing sequence-to-sequence recurrent neural network(RNN)architectures for a novel study to predict soil NOx emissions,driven by the imperative of understanding and mitigating environmental impact.The study utilizes data collected by the Environmental Protection Agency(EPA)to develop two distinct RNN predictive models:one built upon the long-short term memory(LSTM)and the other utilizing the gated recurrent unit(GRU).These models are fed with a combination of historical and anticipated air temperature,air moisture,and NOx emissions as inputs to forecast future NOx emissions.Both LSTM and GRU models can capture the intricate pulse patterns inherent in soil NOx emissions.Notably,the GRU model emerges as the superior performer,surpassing the LSTM model in predictive accuracy while demonstrating efficiency by necessitating less training time.Intriguingly,the investigation into varying input features reveals that relying solely on past NOx emissions as input yields satisfactory performance,highlighting the dominant influence of this factor.The study also delves into the impact of altering input series lengths and training data sizes,yielding insights into optimal configurations for enhanced model performance.Importantly,the findings promise to advance our grasp of soil NOx emission dynamics,with implications for environmental management strategies.Looking ahead,the anticipated availability of additional measurements is poised to bolster machine-learning model efficacy.Furthermore,the future study will explore physical-based RNNs,a promising avenue for deeper insights into soil NOx emission prediction.

An Improved Time Feedforward Connections Recurrent Neural Networks

Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to the strict time serial dependency,making it difficult to realize a long-term memory function.On the other hand,RNNs cells are highly complex,which will signifi-cantly increase computational complexity and cause waste of computational resources during model training.In this paper,an improved Time Feedforward Connections Recurrent Neural Networks(TFC-RNNs)model was first proposed to address the gradient issue.A parallel branch was introduced for the hidden state at time t−2 to be directly transferred to time t without the nonlinear transforma-tion at time t−1.This is effective in improving the long-term dependence of RNNs.Then,a novel cell structure named Single Gate Recurrent Unit(SGRU)was presented.This cell structure can reduce the number of parameters for RNNs cell,consequently reducing the computational complexity.Next,applying SGRU to TFC-RNNs as a new TFC-SGRU model solves the above two difficulties.Finally,the performance of our proposed TFC-SGRU was verified through sev-eral experiments in terms of long-term memory and anti-interference capabilities.Experimental results demonstrated that our proposed TFC-SGRU model can cap-ture helpful information with time step 1500 and effectively filter out the noise.The TFC-SGRU model accuracy is better than the LSTM and GRU models regarding language processing ability.

State of Health Estimation of Lithium-Ion Batteries Using Support Vector Regression and Long Short-Term Memory

Lithium-ion batteries are the most widely accepted type of battery in the electric vehicle industry because of some of their positive inherent characteristics. However, the safety problems associated with inaccurate estimation and prediction of the state of health of these batteries have attracted wide attention due to the adverse negative effect on vehicle safety. In this paper, both machine and deep learning models were used to estimate the state of health of lithium-ion batteries. The paper introduces the definition of battery health status and its importance in the electric vehicle industry. Based on the data preprocessing and visualization analysis, three features related to actual battery capacity degradation are extracted from the data. Two learning models, SVR and LSTM were employed for the state of health estimation and their respective results are compared in this paper. The mean square error and coefficient of determination were the two metrics for the performance evaluation of the models. The experimental results indicate that both models have high estimation results. However, the metrics indicated that the SVR was the overall best model.

基于改进麻雀搜索算法优化LSTM的滚动轴承故障诊断

为了对滚动轴承的工作状态及故障类别进行准确的诊断,本文采用长短时记忆(LSTM)神经网络作为分类器对滚动轴承数据集进行分类诊断。首先,从滚动轴承原始运行振动信号中提取时域和频域特征参数,组成具有高维特征参数的数据集;使用核主成分分析(KPCA)方法对高维特征集进行降维处理,选取重要性程度高的特征构成输入特征向量。然后,针对LSTM神经网络在滚动轴承故障诊断中存在的超参数难以确定的问题,提出一种基于自适应t分布策略的麻雀搜索算法优化LSTM神经网络的故障诊断方法(tSSA–LSTM)。最后,使用凯斯西储大学滚动轴承数据中心的数据进行故障诊断精度测试、泛化性能测试及噪声环境下故障诊断性能测试等多个仿真实验,并将本文提出的诊断模型与麻雀搜索算法优化长短时记忆神经网络(SSA–LSTM)、遗传算法优化长短时记忆神经网络(GA–LSTM)、粒子群算法优化长短时记忆神经网络(PSO–LSTM)及传统LSTM诊断模型进行对比。结果表明:tSSA可以更有效地对LSTM的隐含层神经元数量、周期次数、学习率等超参数进行合理优化;所提方法的平均诊断准确率达到98.86%,交叉验证平均诊断结果为98.57%;所提方法在噪声干扰下的故障诊断准确率也优于对比方法。因此,本文提出的tSSA–LSTM模型不仅可以更精准地诊断滚动轴承故障状态,而且具有更强的泛化能力及抗干扰能力,有效地提高了滚动轴承故障诊断的性能。

结合太阳辐射量计算与CNN-LSTM组合的光伏功率预测方法研究

为了提高模型预测性能,提出一种综合太阳辐射模型及深度学习的光伏功率预测模型。首先,利用太阳辐射机理建立太阳辐射模型(SRM),估算出水平面上总辐射值,再由斜面辐照度转换方法计算出光伏组件所接收的斜面辐射值。其次,通过皮尔逊相关分析法筛选出对光伏功率影响较大的主要因素,将斜面辐射计算值及主要影响因素作为输入,采用卷积神经网络(CNN)和长短期记忆网络(LSTM)建立光伏功率SRM-CNN-LSTM预测模型。分别利用春夏秋冬四季典型日的数据开展对比实验,结果表明:与几种其他方法相比,该文方法具有更好的预测效果。

基于BERT-BiLSTM-CRF模型的畜禽疫病文本分词研究

针对畜禽疫病文本语料匮乏、文本内包含大量疫病名称及短语等未登录词问题,提出了一种结合词典匹配的BERT-BiLSTM-CRF畜禽疫病文本分词模型。以羊疫病为研究对象,构建了常见疫病文本数据集,将其与通用语料PKU结合,利用BERT(Bidirectional encoder representation from transformers)预训练语言模型进行文本向量化表示;通过双向长短时记忆网络(Bidirectional long short-term memory network,BiLSTM)获取上下文语义特征;由条件随机场(Conditional random field,CRF)输出全局最优标签序列。基于此,在CRF层后加入畜禽疫病领域词典进行分词匹配修正,减少在分词过程中出现的疫病名称及短语等造成的歧义切分,进一步提高了分词准确率。实验结果表明,结合词典匹配的BERT-BiLSTM-CRF模型在羊常见疫病文本数据集上的F1值为96.38%,与jieba分词器、BiLSTM-Softmax模型、BiLSTM-CRF模型、未结合词典匹配的本文模型相比,分别提升11.01、10.62、8.3、0.72个百分点,验证了方法的有效性。与单一语料相比,通用语料PKU和羊常见疫病文本数据集结合的混合语料,能够同时对畜禽疫病专业术语及疫病文本中常用词进行准确切分,在通用语料及疫病文本数据集上F1值都达到95%以上,具有较好的模型泛化能力。该方法可用于畜禽疫病文本分词。

基于CEEMD-SE的CNN&LSTM-GRU短期风电功率预测

为进一步提升短期风电功率的预测精度,提出了一种基于互补集合经验模态分解-样本熵(complementary ensemble empirical mode decomposition-sample entropy,CEEMD-SE)的卷积神经网络(convolutional neural network,CNN)和长短期记忆-门控循环单元(longshorttermmemory-gatedrecurrentunit,LSTM-GRU)的短期风电功率预测模型。首先,利用互补集合经验模态分解将原始风电功率序列分解为若干本征模态函数(intrinsic mode function,IMF)分量和一个残差(residual,RES)分量,利用样本熵算法将相近的分量进行重构;其次,搭建卷积神经网络和长短期记忆网络的并行网络结构,提取数据的局部特征和时序特征,并将特征融合后输入门控循环单元网络中进行学习预测;最后,通过算例进行验证,结果表明采用该模型后预测精度得到了有效提升,其均方根误差降低了15.06%、平均绝对误差降低了15.22%、决定系数提高了1.91%。

机器学习算法在降水和气温多模式集成中的应用

选取CMIP6中5种全球气候模式,利用算术平均、权重平均、多元线性回归、BP神经网络、长短期记忆(LSTM)神经网络和随机森林(RF)等6种多模式集成方法,基于黄河流域水源涵养区历史降水量和气温数据,评估不同集成方法的模拟效果,并选取模拟效果最好的多模式集成方法预估未来SSP1-2.6、SSP2-4.5和SSP5-8.53种情景下黄河流域水源涵养区的降水和气温变化趋势。结果表明:多模式集成能很好地再现基准期降水和气温变化,3种机器学习算法表现相对较好,其中LSTM神经网络最好;在未来3种情景下,多年平均降水量均有所增加,四季降水量变化各有差异;SSP1-2.6情景下年降水量峰值出现在各时段初期,SSP2-4.5和SSP5-8.5情景下的年降水量呈增长趋势,远期下降趋势较明显;3种情景下气温都呈上升趋势,但变化差异较大,增温幅度和速率由小到大为SSP1-2.6、SSP2-4.5、SSP5-8.5,秋季气温增幅最大,冬季最小;多模式集成方法对未来降水量和气温的预估存在较大的不确定性,均表现为中远期大于近期,降水量预估的不确定性比气温大,其中降水量秋冬季不确定性明显大于春夏季。

基于字词向量融合的民航智慧监管短文本分类

为解决民航监管事项所产生的检查记录仅依靠人工进行分类分析导致效率低的问题,提出一种基于数据增强与字词向量融合的双通道特征提取的短文本分类模型,探讨民航监管事项的分类,包括与人、设备设施环境、制度程序和机构职责等相关问题。为解决类别不平衡问题,采用数据增强算法在原始文本上进行变换,生成新的样本,使各个类别的样本数量更加均衡。将字向量和词向量按字融合拼接,得到具有词特征信息的字向量。将字词融合的向量分别送入到文本卷积神经网络(TextCNN)和双向长短期记忆(BiLSTM)模型中进行不同维度的特征提取,从局部的角度和全局的角度分别提取特征,并在民航监管事项检查记录数据集上进行试验。结果表明:该模型准确率为0.9837,F 1值为0.9836。与一些字嵌入模型和词嵌入模型相对比,准确率提升0.4%。和一些常用的单通道模型相比,准确率提升3%,验证了双通道模型提取的特征具有全面性和有效性。