Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

A forecasting model for wave heights based on a long short-term memory neural network

To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with sea surface wind and wave heights as training samples.The prediction performance of the model is evaluated,and the error analysis shows that when using the same set of numerically predicted sea surface wind as input,the prediction error produced by the proposed LSTM model at Sta.N01 is 20%,18%and 23%lower than the conventional numerical wave models in terms of the total root mean square error(RMSE),scatter index(SI)and mean absolute error(MAE),respectively.Particularly,for significant wave height in the range of 3–5 m,the prediction accuracy of the LSTM model is improved the most remarkably,with RMSE,SI and MAE all decreasing by 24%.It is also evident that the numbers of hidden neurons,the numbers of buoys used and the time length of training samples all have impact on the prediction accuracy.However,the prediction does not necessary improve with the increase of number of hidden neurons or number of buoys used.The experiment trained by data with the longest time length is found to perform the best overall compared to other experiments with a shorter time length for training.Overall,long short-term memory neural network was proved to be a very promising method for future development and applications in wave forecasting.

Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis

A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deeplearning algorithm based on a fully convolutional neural network(FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory(LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.

Landslide displacement prediction based on optimized empirical mode decomposition and deep bidirectional long short-term memory network

There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement and time series of a landslide.The second one is the dynamic evolution of a landslide,which could not be feasibly simulated simply by traditional prediction models.In this paper,a dynamic model of displacement prediction is introduced for composite landslides based on a combination of empirical mode decomposition with soft screening stop criteria(SSSC-EMD)and deep bidirectional long short-term memory(DBi-LSTM)neural network.In the proposed model,the time series analysis and SSSC-EMD are used to decompose the observed accumulated displacements of a slope into three components,viz.trend displacement,periodic displacement,and random displacement.Then,by analyzing the evolution pattern of a landslide and its key factors triggering landslides,appropriate influencing factors are selected for each displacement component,and DBi-LSTM neural network to carry out multi-datadriven dynamic prediction for each displacement component.An accumulated displacement prediction has been obtained by a summation of each component.For accuracy verification and engineering practicability of the model,field observations from two known landslides in China,the Xintan landslide and the Bazimen landslide were collected for comparison and evaluation.The case study verified that the model proposed in this paper can better characterize the"stepwise"deformation characteristics of a slope.As compared with long short-term memory(LSTM)neural network,support vector machine(SVM),and autoregressive integrated moving average(ARIMA)model,DBi-LSTM neural network has higher accuracy in predicting the periodic displacement of slope deformation,with the mean absolute percentage error reduced by 3.063%,14.913%,and 13.960%respectively,and the root mean square error reduced by 1.951 mm,8.954 mm and 7.790 mm respectively.Conclusively,this model not only has high prediction accuracy but also is more stable,which can provide new insight for practical landslide prevention and control engineering.

Seismic-inversion method for nonlinear mapping multilevel well–seismic matching based on bidirectional long short-term memory networks

In this paper,the recurrent neural network structure of a bidirectional long shortterm memory network(Bi-LSTM)with special memory cells that store information is used to characterize the deep features of the variation pattern between logging and seismic data.A mapping relationship model between high-frequency logging data and low-frequency seismic data is established via nonlinear mapping.The seismic waveform is infinitely approximated using the logging curve in the low-frequency band to obtain a nonlinear mapping model of this scale,which then stepwise approach the logging curve in the high-frequency band.Finally,a seismic-inversion method of nonlinear mapping multilevel well–seismic matching based on the Bi-LSTM network is developed.The characteristic of this method is that by applying the multilevel well–seismic matching process,the seismic data are stepwise matched to the scale range that is consistent with the logging curve.Further,the matching operator at each level can be stably obtained to effectively overcome the problems that occur in the well–seismic matching process,such as the inconsistency in the scale of two types of data,accuracy in extracting the seismic wavelet of the well-side seismic traces,and multiplicity of solutions.Model test and practical application demonstrate that this method improves the vertical resolution of inversion results,and at the same time,the boundary and the lateral characteristics of the sand body are well maintained to improve the accuracy of thin-layer sand body prediction and achieve an improved practical application effect.

Dynamic Hand Gesture Recognition Based on Short-Term Sampling Neural Networks

Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.

Infrasound Event Classification Fusion Model Based on Multiscale SE-CNN and BiLSTM

The classification of infrasound events has considerable importance in improving the capability to identify the types of natural disasters.The traditional infrasound classification mainly relies on machine learning algorithms after artificial feature extraction.However,guaranteeing the effectiveness of the extracted features is difficult.The current trend focuses on using a convolution neural network to automatically extract features for classification.This method can be used to extract signal spatial features automatically through a convolution kernel;however,infrasound signals contain not only spatial information but also temporal information when used as a time series.These extracted temporal features are also crucial.If only a convolution neural network is used,then the time dependence of the infrasound sequence will be missed.Using long short-term memory networks can compensate for the missing time-series features but induces spatial feature information loss of the infrasound signal.A multiscale squeeze excitation–convolution neural network–bidirectional long short-term memory network infrasound event classification fusion model is proposed in this study to address these problems.This model automatically extracted temporal and spatial features,adaptively selected features,and also realized the fusion of the two types of features.Experimental results showed that the classification accuracy of the model was more than 98%,thus verifying the effectiveness and superiority of the proposed model.

Real-time UAV path planning based on LSTM network

To address the shortcomings of single-step decision making in the existing deep reinforcement learning based unmanned aerial vehicle(UAV)real-time path planning problem,a real-time UAV path planning algorithm based on long shortterm memory(RPP-LSTM)network is proposed,which combines the memory characteristics of recurrent neural network(RNN)and the deep reinforcement learning algorithm.LSTM networks are used in this algorithm as Q-value networks for the deep Q network(DQN)algorithm,which makes the decision of the Q-value network has some memory.Thanks to LSTM network,the Q-value network can use the previous environmental information and action information which effectively avoids the problem of single-step decision considering only the current environment.Besides,the algorithm proposes a hierarchical reward and punishment function for the specific problem of UAV real-time path planning,so that the UAV can more reasonably perform path planning.Simulation verification shows that compared with the traditional feed-forward neural network(FNN)based UAV autonomous path planning algorithm,the RPP-LSTM proposed in this paper can adapt to more complex environments and has significantly improved robustness and accuracy when performing UAV real-time path planning.

Prediction of discharge in a tidal river using the LSTM-based sequence-to-sequence models

The complexity of river-tide interaction poses a significant challenge in predicting discharge in tidal rivers.Long short-term memory(LSTM)networks excel in processing and predicting crucial events with extended intervals and time delays in time series data.Additionally,the sequence-to-sequence(Seq2Seq)model,known for handling temporal relationships,adapting to variable-length sequences,effectively capturing historical information,and accommodating various influencing factors,emerges as a robust and flexible tool in discharge forecasting.In this study,we introduce the application of LSTM-based Seq2Seq models for the first time in forecasting the discharge of a tidal reach of the Changjiang River(Yangtze River)Estuary.This study focuses on discharge forecasting using three key input characteristics:flow velocity,water level,and discharge,which means the structure of multiple input and single output is adopted.The experiment used the discharge data of the whole year of 2020,of which the first 80%is used as the training set,and the last 20%is used as the test set.This means that the data covers different tidal cycles,which helps to test the forecasting effect of different models in different tidal cycles and different runoff.The experimental results indicate that the proposed models demonstrate advantages in long-term,mid-term,and short-term discharge forecasting.The Seq2Seq models improved by 6%-60%and 5%-20%of the relative standard deviation compared to the harmonic analysis models and improved back propagation neural network models in discharge prediction,respectively.In addition,the relative accuracy of the Seq2Seq model is 1%to 3%higher than that of the LSTM model.Analytical assessment of the prediction errors shows that the Seq2Seq models are insensitive to the forecast lead time and they can capture characteristic values such as maximum flood tide flow and maximum ebb tide flow in the tidal cycle well.This indicates the significance of the Seq2Seq models.

Wind Speed Prediction Based on Improved VMD-BP-CNN-LSTM Model

Amid the randomness and volatility of wind speed, an improved VMD-BP-CNN-LSTM model for short-term wind speed prediction was proposed to assist in power system planning and operation in this paper. Firstly, the wind speed time series data was processed using Variational Mode Decomposition (VMD) to obtain multiple frequency components. Then, each individual frequency component was channeled into a combined prediction framework consisting of BP neural network (BPNN), Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM) after the execution of differential and normalization operations. Thereafter, the predictive outputs for each component underwent integration through a fully-connected neural architecture for data fusion processing, resulting in the final prediction. The VMD decomposition technique was introduced in a generalized CNN-LSTM prediction model;a BPNN model was utilized to predict high-frequency components obtained from VMD, and incorporated a fully connected neural network for data fusion of individual component predictions. Experimental results demonstrated that the proposed improved VMD-BP-CNN-LSTM model outperformed other combined prediction models in terms of prediction accuracy, providing a solid foundation for optimizing the safe operation of wind farms.

Deep Learning-Based Stock Price Prediction Using LSTM Model

The stock market is a vital component of the broader financial system,with its dynamics closely linked to economic growth.The challenges associated with analyzing and forecasting stock prices have persisted since the inception of financial markets.By examining historical transaction data,latent opportunities for profit can be uncovered,providing valuable insights for both institutional and individual investors to make more informed decisions.This study focuses on analyzing historical transaction data from four banks to predict closing price trends.Various models,including decision trees,random forests,and Long Short-Term Memory(LSTM)networks,are employed to forecast stock price movements.Historical stock transaction data serves as the input for training these models,which are then used to predict upward or downward stock price trends.The study’s empirical results indicate that these methods are effective to a degree in predicting stock price movements.The LSTM-based deep neural network model,in particular,demonstrates a commendable level of predictive accuracy.This conclusion is reached following a thorough evaluation of model performance,highlighting the potential of LSTM models in stock market forecasting.The findings offer significant implications for advancing financial forecasting approaches,thereby improving the decision-making capabilities of investors and financial institutions.

Audiovisual speech recognition based on a deep convolutional neural network

Audiovisual speech recognition is an emerging research topic.Lipreading is the recognition of what someone is saying using visual information,primarily lip movements.In this study,we created a custom dataset for Indian English linguistics and categorized it into three main categories:(1)audio recognition,(2)visual feature extraction,and(3)combined audio and visual recognition.Audio features were extracted using the mel-frequency cepstral coefficient,and classification was performed using a one-dimension convolutional neural network.Visual feature extraction uses Dlib and then classifies visual speech using a long short-term memory type of recurrent neural networks.Finally,integration was performed using a deep convolutional network.The audio speech of Indian English was successfully recognized with accuracies of 93.67%and 91.53%,respectively,using testing data from 200 epochs.The training accuracy for visual speech recognition using the Indian English dataset was 77.48%and the test accuracy was 76.19%using 60 epochs.After integration,the accuracies of audiovisual speech recognition using the Indian English dataset for training and testing were 94.67%and 91.75%,respectively.

Classification of Arrhythmia Based on Convolutional Neural Networks and Encoder-Decoder Model

As a common and high-risk type of disease,heart disease seriously threatens people’s health.At the same time,in the era of the Internet of Thing(IoT),smart medical device has strong practical significance for medical workers and patients because of its ability to assist in the diagnosis of diseases.Therefore,the research of real-time diagnosis and classification algorithms for arrhythmia can help to improve the diagnostic efficiency of diseases.In this paper,we design an automatic arrhythmia classification algorithm model based on Convolutional Neural Network(CNN)and Encoder-Decoder model.The model uses Long Short-Term Memory(LSTM)to consider the influence of time series features on classification results.Simultaneously,it is trained and tested by the MIT-BIH arrhythmia database.Besides,Generative Adversarial Networks(GAN)is adopted as a method of data equalization for solving data imbalance problem.The simulation results show that for the inter-patient arrhythmia classification,the hybrid model combining CNN and Encoder-Decoder model has the best classification accuracy,of which the accuracy can reach 94.05%.Especially,it has a better advantage for the classification effect of supraventricular ectopic beats(class S)and fusion beats(class F).

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.

Application of LSTM and CONV1D LSTM Network in Stock Forecasting Model

Predicting the direction of the stock market has always been a huge challenge.Also,the way of forecasting the stock market reduces the risk in the financial market,thus ensuring that brokers can make normal returns.Despite the complexities of the stock market,the challenge has been increasingly addressed by experts in a variety of disciplines,including economics,statistics,and computer science.The introduction of machine learning,in-depth understanding of the prospects of the financial market,thus doing many experiments to predict the future so that the stock price trend has different degrees of success.In this paper,we propose a method to predict stocks from different industries and markets,as well as trend prediction using traditional machine learning algorithms such as linear regression,polynomial regression and learning techniques in time series prediction using two forms of special types of recursive neural networks:long and short time memory(LSTM)and spoken short-term memory.

State of Health Estimation of Lithium-Ion Batteries Using Support Vector Regression and Long Short-Term Memory

Lithium-ion batteries are the most widely accepted type of battery in the electric vehicle industry because of some of their positive inherent characteristics. However, the safety problems associated with inaccurate estimation and prediction of the state of health of these batteries have attracted wide attention due to the adverse negative effect on vehicle safety. In this paper, both machine and deep learning models were used to estimate the state of health of lithium-ion batteries. The paper introduces the definition of battery health status and its importance in the electric vehicle industry. Based on the data preprocessing and visualization analysis, three features related to actual battery capacity degradation are extracted from the data. Two learning models, SVR and LSTM were employed for the state of health estimation and their respective results are compared in this paper. The mean square error and coefficient of determination were the two metrics for the performance evaluation of the models. The experimental results indicate that both models have high estimation results. However, the metrics indicated that the SVR was the overall best model.

基于多源信息融合和WOA-CNN-LSTM的外脚手架隐患分类预警研究

面对施工现场外脚手架隐患信息的多样性,传统的基于传感器监测的单一信号预警研究存在容错力不佳、含有信息有限等问题。针对施工现场外脚手架“图像+监测”数据,提出一种基于数据层和特征层信息融合的脚手架隐患分类预警方法。首先,利用Revit三维建模软件建立外脚手架实体模型,对不同初始隐患下的外脚手架进行有限元分析,划分隐患预警等级;其次,利用无迹卡尔曼滤波算法(Unscented Kalman Filter,UKF)及卷积长短时记忆网络(Convolutional Neural Network-Long Short Term Memory Network,CNN-LSTM)实现脚手架同类信息数据层融合及异类信息特征层融合;最后,通过实时收集西安市某在建项目落地式双排扣件式钢管脚手架隐患信息,对其进行分类预警,并使用鲸鱼优化算法(Whale Optimization Algorithm,WOA)对CNN-LSTM网络进行参数优化,发现隐藏节点个数为30、学习率为0.0072、正则化系数为1×10^(-4)时分类效果最佳,优化后预警精度达到了91.4526%。通过可视化WOA-CNN-LSTM、CNN-LSTM、CNN-SVM(Support Vector Machine,支持向量机)及CNN-GRU(Gate Recurrent Unit,门控循环单元)分类预警结果,证实了优化后的CNN-LSTM网络在脚手架分类预警方面的优越性。

基于注意力机制的CNN-BiLSTM的IGBT剩余使用寿命预测

针对绝缘栅双极型晶体管(IGBT)可靠性问题,提出了一种融合卷积神经网络(CNN)、双向长短期记忆(BiLSTM)网络和注意力机制的剩余使用寿命(RUL)预测模型,可用于IGBT的寿命预测。模型中使用CNN提取特征参数,BiLSTM提取时序信息,注意力机制加权处理特征参数。使用IGBT加速老化数据集对提出的模型进行验证。结果表明,对比自回归差分移动平均(ARIMA)、长短期记忆(LSTM)、多层LSTM(Multi-LSTM)、 BiLSTM预测模型,在均方根误差和决定系数等评价指标方面该模型的性能最优。验证了提出的寿命预测模型对IGBT失效预测是有效的。

基于CNN-LSTM和卷复制方法的高可用系统设计方法

针对单机服务器存在的单点故障问题,以及主备双机中存在的逻辑故障导致数据丢失的问题,设计了一种基于卷积和长短期记忆神经网络(CNN-LSTM)和卷复制方法的HA(High Availability)系统。系统至少包含两个节点,一个主节点以及一个或多个备用节点,主节点和备节点之间支持主备切换。每个服务器节点上包含4个模块,分别是负责接收配置信息与读写请求的代理模块;进行磁盘读写操作和重定向读写的磁盘I/O(输入输出)模块;负责主备节点间备份快照、映射表、数据块复制的卷复制模块以及基于CNN-LSTM进行状态检测的高可用模块。实验表明,该系统不仅可以解决单点故障问题,也可以解决主备双机集群中无法解决的逻辑错误问题;同时基于CNN-LSTM方法,自动针对服务器的运行健康状态进行分析和预测,可以根据预测结果自动通知管理员进行处理或自动进行主备切换。