Modeling injection-induced fault slip using long short-term memory networks

Stress changes due to changes in fluid pressure and temperature in a faulted formation may lead to the opening/shearing of the fault.This can be due to subsurface(geo)engineering activities such as fluid injections and geologic disposal of nuclear waste.Such activities are expected to rise in the future making it necessary to assess their short-and long-term safety.Here,a new machine learning(ML)approach to model pore pressure and fault displacements in response to high-pressure fluid injection cycles is developed.The focus is on fault behavior near the injection borehole.To capture the temporal dependencies in the data,long short-term memory(LSTM)networks are utilized.To prevent error accumulation within the forecast window,four critical measures to train a robust LSTM model for predicting fault response are highlighted:(i)setting an appropriate value of LSTM lag,(ii)calibrating the LSTM cell dimension,(iii)learning rate reduction during weight optimization,and(iv)not adopting an independent injection cycle as a validation set.Several numerical experiments were conducted,which demonstrated that the ML model can capture peaks in pressure and associated fault displacement that accompany an increase in fluid injection.The model also captured the decay in pressure and displacement during the injection shut-in period.Further,the ability of an ML model to highlight key changes in fault hydromechanical activation processes was investigated,which shows that ML can be used to monitor risk of fault activation and leakage during high pressure fluid injections.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

A forecasting model for wave heights based on a long short-term memory neural network

To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with sea surface wind and wave heights as training samples.The prediction performance of the model is evaluated,and the error analysis shows that when using the same set of numerically predicted sea surface wind as input,the prediction error produced by the proposed LSTM model at Sta.N01 is 20%,18%and 23%lower than the conventional numerical wave models in terms of the total root mean square error(RMSE),scatter index(SI)and mean absolute error(MAE),respectively.Particularly,for significant wave height in the range of 3–5 m,the prediction accuracy of the LSTM model is improved the most remarkably,with RMSE,SI and MAE all decreasing by 24%.It is also evident that the numbers of hidden neurons,the numbers of buoys used and the time length of training samples all have impact on the prediction accuracy.However,the prediction does not necessary improve with the increase of number of hidden neurons or number of buoys used.The experiment trained by data with the longest time length is found to perform the best overall compared to other experiments with a shorter time length for training.Overall,long short-term memory neural network was proved to be a very promising method for future development and applications in wave forecasting.

Slope stability prediction based on a long short-term memory neural network:comparisons with convolutional neural networks,support vector machines and random forest models

The numerical simulation and slope stability prediction are the focus of slope disaster research.Recently,machine learning models are commonly used in the slope stability prediction.However,these machine learning models have some problems,such as poor nonlinear performance,local optimum and incomplete factors feature extraction.These issues can affect the accuracy of slope stability prediction.Therefore,a deep learning algorithm called Long short-term memory(LSTM)has been innovatively proposed to predict slope stability.Taking the Ganzhou City in China as the study area,the landslide inventory and their characteristics of geotechnical parameters,slope height and slope angle are analyzed.Based on these characteristics,typical soil slopes are constructed using the Geo-Studio software.Five control factors affecting slope stability,including slope height,slope angle,internal friction angle,cohesion and volumetric weight,are selected to form different slope and construct model input variables.Then,the limit equilibrium method is used to calculate the stability coefficients of these typical soil slopes under different control factors.Each slope stability coefficient and its corresponding control factors is a slope sample.As a result,a total of 2160 training samples and 450 testing samples are constructed.These sample sets are imported into LSTM for modelling and compared with the support vector machine(SVM),random forest(RF)and convo-lutional neural network(CNN).The results show that the LSTM overcomes the problem that the commonly used machine learning models have difficulty extracting global features.Furthermore,LSTM has a better prediction performance for slope stability compared to SVM,RF and CNN models.

State-of-health estimation for fast-charging lithium-ion batteries based on a short charge curve using graph convolutional and long short-term memory networks

A fast-charging policy is widely employed to alleviate the inconvenience caused by the extended charging time of electric vehicles. However, fast charging exacerbates battery degradation and shortens battery lifespan. In addition, there is still a lack of tailored health estimations for fast-charging batteries;most existing methods are applicable at lower charging rates. This paper proposes a novel method for estimating the health of lithium-ion batteries, which is tailored for multi-stage constant current-constant voltage fast-charging policies. Initially, short charging segments are extracted by monitoring current switches,followed by deriving voltage sequences using interpolation techniques. Subsequently, a graph generation layer is used to transform the voltage sequence into graphical data. Furthermore, the integration of a graph convolution network with a long short-term memory network enables the extraction of information related to inter-node message transmission, capturing the key local and temporal features during the battery degradation process. Finally, this method is confirmed by utilizing aging data from 185 cells and 81 distinct fast-charging policies. The 4-minute charging duration achieves a balance between high accuracy in estimating battery state of health and low data requirements, with mean absolute errors and root mean square errors of 0.34% and 0.66%, respectively.

Comparison of Two Recurrent Neural Networks for Rainfall-Runoff Modeling in the Zou River Basin at Atchérigbé (Bénin)

Hydrological models are developed to simulate river flows over a watershed for many practical applications in the field of water resource management. The present paper compares the performance of two recurrent neural networks for rainfall-runoff modeling in the Zou River basin at Atchérigbé outlet. To this end, we used daily precipitation data over the period 1988-2010 as input of the models, such as the Long Short-Term Memory (LSTM) and Recurrent Gate Networks (GRU) to simulate river discharge in the study area. The investigated models give good results in calibration (R2 = 0.888, NSE = 0.886, and RMSE = 0.42 for LSTM;R2 = 0.9, NSE = 0.9 and RMSE = 0.397 for GRU) and in validation (R2 = 0.865, NSE = 0.851, and RMSE = 0.329 for LSTM;R2 = 0.9, NSE = 0.865 and RMSE = 0.301 for GRU). This good performance of LSTM and GRU models confirms the importance of models based on machine learning in modeling hydrological phenomena for better decision-making.

Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis

A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deeplearning algorithm based on a fully convolutional neural network(FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory(LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.

Landslide displacement prediction based on optimized empirical mode decomposition and deep bidirectional long short-term memory network

There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement and time series of a landslide.The second one is the dynamic evolution of a landslide,which could not be feasibly simulated simply by traditional prediction models.In this paper,a dynamic model of displacement prediction is introduced for composite landslides based on a combination of empirical mode decomposition with soft screening stop criteria(SSSC-EMD)and deep bidirectional long short-term memory(DBi-LSTM)neural network.In the proposed model,the time series analysis and SSSC-EMD are used to decompose the observed accumulated displacements of a slope into three components,viz.trend displacement,periodic displacement,and random displacement.Then,by analyzing the evolution pattern of a landslide and its key factors triggering landslides,appropriate influencing factors are selected for each displacement component,and DBi-LSTM neural network to carry out multi-datadriven dynamic prediction for each displacement component.An accumulated displacement prediction has been obtained by a summation of each component.For accuracy verification and engineering practicability of the model,field observations from two known landslides in China,the Xintan landslide and the Bazimen landslide were collected for comparison and evaluation.The case study verified that the model proposed in this paper can better characterize the"stepwise"deformation characteristics of a slope.As compared with long short-term memory(LSTM)neural network,support vector machine(SVM),and autoregressive integrated moving average(ARIMA)model,DBi-LSTM neural network has higher accuracy in predicting the periodic displacement of slope deformation,with the mean absolute percentage error reduced by 3.063%,14.913%,and 13.960%respectively,and the root mean square error reduced by 1.951 mm,8.954 mm and 7.790 mm respectively.Conclusively,this model not only has high prediction accuracy but also is more stable,which can provide new insight for practical landslide prevention and control engineering.

Dynamic prediction of landslide displacement using singular spectrum analysis and stack long short-term memory network

An accurate landslide displacement prediction is an important part of landslide warning system. Aiming at the dynamic characteristics of landslide evolution and the shortcomings of traditional static prediction models, this paper proposes a dynamic prediction model of landslide displacement based on singular spectrum analysis(SSA) and stack long short-term memory(SLSTM) network. The SSA is used to decompose the landslide accumulated displacement time series data into trend term and periodic term displacement subsequences. A cubic polynomial function is used to predict the trend term displacement subsequence, and the SLSTM neural network is used to predict the periodic term displacement subsequence. At the same time, the Bayesian optimization algorithm is used to determine that the SLSTM network input sequence length is 12 and the number of hidden layer nodes is 18. The SLSTM network is updated by adding predicted values to the training set to achieve dynamic displacement prediction. Finally, the accumulated landslide displacement is obtained by superimposing the predicted value of each displacement subsequence. The proposed model was verified on the Xintan landslide in Hubei Province, China. The results show that when predicting the displacement of the periodic term, the SLSTM network has higher prediction accuracy than the support vector machine(SVM) and auto regressive integrated moving average(ARIMA). The mean relative error(MRE) is reduced by 4.099% and 3.548% respectively, while the root mean square error(RMSE) is reduced by 5.830 mm and 3.854 mm respectively. It is concluded that the SLSTM network model can better simulate the dynamic characteristics of landslides.

Preliminary abnormal electrocardiogram segment screening method for Holter data based on long short-term memory networks

Holter usually monitors electrocardiogram(ECG)signals for more than 24 hours to capture short-lived cardiac abnormalities.In view of the large amount of Holter data and the fact that the normal part accounts for the majority,it is reasonable to design an algorithm that can automatically eliminate normal data segments as much as possible without missing any abnormal data segments,and then take the left segments to the doctors or the computer programs for further diagnosis.In this paper,we propose a preliminary abnormal segment screening method for Holter data.Based on long short-term memory(LSTM)networks,the prediction model is established and trained with the normal data of a monitored object.Then,on the basis of kernel density estimation,we learn the distribution law of prediction errors after applying the trained LSTM model to the regular data.Based on these,the preliminary abnormal ECG segment screening analysis is carried out without R wave detection.Experiments on the MIT-BIH arrhythmia database show that,under the condition of ensuring that no abnormal point is missed,53.89% of normal segments can be effectively obviated.This work can greatly reduce the workload of subsequent further processing.

A hybrid model for short-term rainstorm forecasting based on a back-propagation neural network and synoptic diagnosis

Rainstorms are one of the most important types of natural disaster in China.In order to enhance the ability to forecast rainstorms in the short term,this paper explores how to combine a back-propagation neural network(BPNN)with synoptic diagnosis for predicting rainstorms,and analyzes the hit rates of rainstorms for the above two methods using the county of Tianquan as a case study.Results showed that the traditional synoptic diagnosis method still has an important referential meaning for most rainstorm types through synoptic typing and statistics of physical quantities based on historical cases,and the threat score(TS)of rainstorms was more than 0.75.However,the accuracy for two rainstorm types influenced by low-level easterly inverted troughs was less than 40%.The BPNN method efficiently forecasted these two rainstorm types;the TS and equitable threat score(ETS)of rainstorms were 0.80 and 0.79,respectively.The TS and ETS of the hybrid model that combined the BPNN and synoptic diagnosis methods exceeded the forecast score of multi-numerical simulations over the Sichuan Basin without exception.This kind of hybrid model enhanced the forecasting accuracy of rainstorms.The findings of this study provide certain reference value for the future development of refined forecast models with local features.

A Hopfield-like hippocampal CA3 neural network model for studying associative memory in Alzheimer's disease

Associative memory, one of the major cognitive functions in the hippocampal CA3 region, includes auto-associative memory and hetero-associative memory. Many previous studies have shown that Alzheimer＇s disease （AD） can lead to loss of functional synapses in the central nervous system, and associative memory functions in patients with AD are often impaired, but few studies have addressed the effect of AD on hetero-associative memory in the hippocampal CA3 region. In this study, based on a simplified anatomical structure and synaptic connections in the hippocampal CA3 region, a three-layered Hopfield-like neural network model of hippocampal CA3 was proposed and then used to simulate associative memory functions in three circumstances： normal, synaptic deletion and synaptic compensation, according to Ruppin＇s synaptic deletion and compensation theory. The influences of AD on hetero-associative memory were further analyzed. The simulated results showed that the established three-layered Hopfield-like neural network model of hippocampal CA3 has both auto-associative and hetero-associative memory functions. With increasing synaptic deletion level, both associative memory functions were gradually impaired and the mean firing rates of the neurons within the network model were decreased. With gradual increasing synaptic compensation, the associative memory functions of the network were improved and the mean firing rates were increased. The simulated results suggest that the Hopfield-like neural network model can effectively simulate both associative memory functions of the hippocampal CA3 region. Synaptic deletion affects both auto-associative and hetero-associative memory functions in the hippocampal CA3 region, and can also result in memory dysfunction. To some extent, synaptic compensation measures can offset two kinds of associative memory dysfunction caused by synaptic deletion in the hippocampal CA3 area.

Seismic-inversion method for nonlinear mapping multilevel well–seismic matching based on bidirectional long short-term memory networks

In this paper,the recurrent neural network structure of a bidirectional long shortterm memory network(Bi-LSTM)with special memory cells that store information is used to characterize the deep features of the variation pattern between logging and seismic data.A mapping relationship model between high-frequency logging data and low-frequency seismic data is established via nonlinear mapping.The seismic waveform is infinitely approximated using the logging curve in the low-frequency band to obtain a nonlinear mapping model of this scale,which then stepwise approach the logging curve in the high-frequency band.Finally,a seismic-inversion method of nonlinear mapping multilevel well–seismic matching based on the Bi-LSTM network is developed.The characteristic of this method is that by applying the multilevel well–seismic matching process,the seismic data are stepwise matched to the scale range that is consistent with the logging curve.Further,the matching operator at each level can be stably obtained to effectively overcome the problems that occur in the well–seismic matching process,such as the inconsistency in the scale of two types of data,accuracy in extracting the seismic wavelet of the well-side seismic traces,and multiplicity of solutions.Model test and practical application demonstrate that this method improves the vertical resolution of inversion results,and at the same time,the boundary and the lateral characteristics of the sand body are well maintained to improve the accuracy of thin-layer sand body prediction and achieve an improved practical application effect.

Dynamic Hand Gesture Recognition Based on Short-Term Sampling Neural Networks

Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.

Application of preoperative artificial neural network based on blood biomarkers and clinicopathological parameters for predicting longterm survival of patients with gastric cancer

BACKGROUND Because of the powerful abilities of self-learning and handling complex biological information,artificial neural network(ANN)models have been widely applied to disease diagnosis,imaging analysis,and prognosis prediction.However,there has been no trained preoperative ANN(preope-ANN)model to preoperatively predict the prognosis of patients with gastric cancer(GC).AIM To establish a neural network model that can predict long-term survival of GC patients before surgery to evaluate the tumor condition before the operation.METHODS The clinicopathological data of 1608 GC patients treated from January 2011 to April 2015 at the Department of Gastric Surgery,Fujian Medical University Union Hospital were analyzed retrospectively.The patients were randomly divided into a training set(70%)for establishing a preope-ANN model and a testing set(30%).The prognostic evaluation ability of the preope-ANN model was compared with that of the American Joint Commission on Cancer(8th edition)clinical TNM(cTNM)and pathological TNM(pTNM)staging through the receiver operating characteristic curve,Akaike information criterion index,Harrell's C index,and likelihood ratio chi-square.RESULTS We used the variables that were statistically significant factors for the 3-year overall survival as input-layer variables to develop a preope-ANN in the training set.The survival curves within each score of the preope-ANN had good discrimination(P<0.05).Comparing the preope-ANN model,cTNM,and pTNM in both the training and testing sets,the preope-ANN model was superior to cTNM in predictive discrimination(C index),predictive homogeneity(likelihood ratio chi-square),and prediction accuracy(area under the curve).The prediction efficiency of the preope-ANN model is similar to that of pTNM.CONCLUSION The preope-ANN model can accurately predict the long-term survival of GC patients,and its predictive efficiency is not inferior to that of pTNM stage.

Stability analysis of discrete-time BAM neural networks based on standard neural network models

To facilitate stability analysis of discrete-time bidirectional associative memory (BAM) neural networks, they were converted into novel neural network models, termed standard neural network models (SNNMs), which interconnect linear dynamic systems and bounded static nonlinear operators. By combining a number of different Lyapunov functionals with S-procedure, some useful criteria of global asymptotic stability and global exponential stability of the equilibrium points of SNNMs were derived. These stability conditions were formulated as linear matrix inequalities (LMIs). So global stability of the discrete-time BAM neural networks could be analyzed by using the stability results of the SNNMs. Compared to the existing stability analysis methods, the proposed approach is easy to implement, less conservative, and is applicable to other recurrent neural networks.

Short-Term Relay Quality Prediction Algorithm Based on Long and Short-Term Memory

The fraction defective of semi-finished products is predicted to optimize the process of relay production lines, by which production quality and productivity are increased, and the costs are decreased. The process parameters of relay production lines are studied based on the long-and-short-term memory network. Then, the Keras deep learning framework is utilized to build up a short-term relay quality prediction algorithm for the semi-finished product. A simulation model is used to study prediction algorithm. The simulation results show that the average prediction absolute error of the fraction is less than 5%. This work displays great application potential in the relay production lines.

Audiovisual speech recognition based on a deep convolutional neural network

Audiovisual speech recognition is an emerging research topic.Lipreading is the recognition of what someone is saying using visual information,primarily lip movements.In this study,we created a custom dataset for Indian English linguistics and categorized it into three main categories:(1)audio recognition,(2)visual feature extraction,and(3)combined audio and visual recognition.Audio features were extracted using the mel-frequency cepstral coefficient,and classification was performed using a one-dimension convolutional neural network.Visual feature extraction uses Dlib and then classifies visual speech using a long short-term memory type of recurrent neural networks.Finally,integration was performed using a deep convolutional network.The audio speech of Indian English was successfully recognized with accuracies of 93.67%and 91.53%,respectively,using testing data from 200 epochs.The training accuracy for visual speech recognition using the Indian English dataset was 77.48%and the test accuracy was 76.19%using 60 epochs.After integration,the accuracies of audiovisual speech recognition using the Indian English dataset for training and testing were 94.67%and 91.75%,respectively.

A MODEL FOR UNFULLY INTERCONNECTED NEURAL NETWORK

In order to reduce the complexity of neural network connectivity,a dy-namical model for unfully interconnected neural network,including its energy func-tion,local area field and learning rule,is presented.The basic idea is to decompose aHopfield network into several subnetworks and set up some interconnections betweenthem.The statistical analysis of the associative memory process shows that the num-ber of interconnections after the first decomposition is reduced by 25% comparedwith that of the Hopfield network,but the storage capacity and the associative abilityof the network remain unchanged.With the decomposition continued,the number ofinterconnections is considerably reduced.Despite the reduction in storage capacityand associative ability with continuous decomposition,the average information capac-ity per interconnection has increased nearly by 100%.Finally the relationship be-tween high-order interconnection and multilayer network architecture is discussed.