Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis

A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deeplearning algorithm based on a fully convolutional neural network(FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory(LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.

A Levenberg–Marquardt Based Neural Network for Short-Term Load Forecasting

Short-term load forecasting (STLF) is part and parcel of theefficient working of power grid stations. Accurate forecasts help to detect thefault and enhance grid reliability for organizing sufficient energy transactions.STLF ranges from an hour ahead prediction to a day ahead prediction. Variouselectric load forecasting methods have been used in literature for electricitygeneration planning to meet future load demand. A perfect balance regardinggeneration and utilization is still lacking to avoid extra generation and misusageof electric load. Therefore, this paper utilizes Levenberg–Marquardt(LM) based Artificial Neural Network (ANN) technique to forecast theshort-term electricity load for smart grids in a much better, more precise,and more accurate manner. For proper load forecasting, we take the mostcritical weather parameters along with historical load data in the form of timeseries grouped into seasons, i.e., winter and summer. Further, the presentedmodel deals with each season’s load data by splitting it into weekdays andweekends. The historical load data of three years have been used to forecastweek-ahead and day-ahead load demand after every thirty minutes makingload forecast for a very short period. The proposed model is optimized usingthe Levenberg-Marquardt backpropagation algorithm to achieve results withcomparable statistics. Mean Absolute Percent Error (MAPE), Root MeanSquared Error (RMSE), R2, and R are used to evaluate the model. Comparedwith other recent machine learning-based mechanisms, our model presentsthe best experimental results with MAPE and R2 scores of 1.3 and 0.99,respectively. The results prove that the proposed LM-based ANN modelperforms much better in accuracy and has the lowest error rates as comparedto existing work.

Landslide displacement prediction based on optimized empirical mode decomposition and deep bidirectional long short-term memory network

There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement and time series of a landslide.The second one is the dynamic evolution of a landslide,which could not be feasibly simulated simply by traditional prediction models.In this paper,a dynamic model of displacement prediction is introduced for composite landslides based on a combination of empirical mode decomposition with soft screening stop criteria(SSSC-EMD)and deep bidirectional long short-term memory(DBi-LSTM)neural network.In the proposed model,the time series analysis and SSSC-EMD are used to decompose the observed accumulated displacements of a slope into three components,viz.trend displacement,periodic displacement,and random displacement.Then,by analyzing the evolution pattern of a landslide and its key factors triggering landslides,appropriate influencing factors are selected for each displacement component,and DBi-LSTM neural network to carry out multi-datadriven dynamic prediction for each displacement component.An accumulated displacement prediction has been obtained by a summation of each component.For accuracy verification and engineering practicability of the model,field observations from two known landslides in China,the Xintan landslide and the Bazimen landslide were collected for comparison and evaluation.The case study verified that the model proposed in this paper can better characterize the"stepwise"deformation characteristics of a slope.As compared with long short-term memory(LSTM)neural network,support vector machine(SVM),and autoregressive integrated moving average(ARIMA)model,DBi-LSTM neural network has higher accuracy in predicting the periodic displacement of slope deformation,with the mean absolute percentage error reduced by 3.063%,14.913%,and 13.960%respectively,and the root mean square error reduced by 1.951 mm,8.954 mm and 7.790 mm respectively.Conclusively,this model not only has high prediction accuracy but also is more stable,which can provide new insight for practical landslide prevention and control engineering.

Baseline Isolated Printed Text Image Database for Pashto Script Recognition

The optical character recognition for the right to left and cursive languages such as Arabic is challenging and received little attention from researchers in the past compared to the other Latin languages.Moreover,the absence of a standard publicly available dataset for several low-resource lan-guages,including the Pashto language remained a hurdle in the advancement of language processing.Realizing that,a clean dataset is the fundamental and core requirement of character recognition,this research begins with dataset generation and aims at a system capable of complete language understanding.Keeping in view the complete and full autonomous recognition of the cursive Pashto script.The first achievement of this research is a clean and standard dataset for the isolated characters of the Pashto script.In this paper,a database of isolated Pashto characters for forty four alphabets using various font styles has been introduced.In order to overcome the font style shortage,the graphical software Inkscape has been used to generate sufficient image data samples for each character.The dataset has been pre-processed and reduced in dimensions to 32×32 pixels,and further converted into the binary format with a black background and white text so that it resembles the Modified National Institute of Standards and Technology(MNIST)database.The benchmark database is publicly available for further research on the standard GitHub and Kaggle database servers both in pixel and Comma Separated Values(CSV)formats.

Real-time UAV path planning based on LSTM network

To address the shortcomings of single-step decision making in the existing deep reinforcement learning based unmanned aerial vehicle(UAV)real-time path planning problem,a real-time UAV path planning algorithm based on long shortterm memory(RPP-LSTM)network is proposed,which combines the memory characteristics of recurrent neural network(RNN)and the deep reinforcement learning algorithm.LSTM networks are used in this algorithm as Q-value networks for the deep Q network(DQN)algorithm,which makes the decision of the Q-value network has some memory.Thanks to LSTM network,the Q-value network can use the previous environmental information and action information which effectively avoids the problem of single-step decision considering only the current environment.Besides,the algorithm proposes a hierarchical reward and punishment function for the specific problem of UAV real-time path planning,so that the UAV can more reasonably perform path planning.Simulation verification shows that compared with the traditional feed-forward neural network(FNN)based UAV autonomous path planning algorithm,the RPP-LSTM proposed in this paper can adapt to more complex environments and has significantly improved robustness and accuracy when performing UAV real-time path planning.

Short-Term Household Load Forecasting Based on Attention Mechanism and CNN-ICPSO-LSTM

Accurate load forecasting forms a crucial foundation for implementing household demand response plans andoptimizing load scheduling. When dealing with short-term load data characterized by substantial fluctuations,a single prediction model is hard to capture temporal features effectively, resulting in diminished predictionaccuracy. In this study, a hybrid deep learning framework that integrates attention mechanism, convolution neuralnetwork (CNN), improved chaotic particle swarm optimization (ICPSO), and long short-term memory (LSTM), isproposed for short-term household load forecasting. Firstly, the CNN model is employed to extract features fromthe original data, enhancing the quality of data features. Subsequently, the moving average method is used for datapreprocessing, followed by the application of the LSTM network to predict the processed data. Moreover, the ICPSOalgorithm is introduced to optimize the parameters of LSTM, aimed at boosting the model’s running speed andaccuracy. Finally, the attention mechanism is employed to optimize the output value of LSTM, effectively addressinginformation loss in LSTM induced by lengthy sequences and further elevating prediction accuracy. According tothe numerical analysis, the accuracy and effectiveness of the proposed hybrid model have been verified. It canexplore data features adeptly, achieving superior prediction accuracy compared to other forecasting methods forthe household load exhibiting significant fluctuations across different seasons.

Analysis of Mobile and Internet Network Coverage: Propagation of Electromagnetic Waves and Concept of Digital Divide in Burundi

Mobile and Internet network coverage plays an important role in digital transformation and the exploitation of new services. The evolution of mobile networks from the first generation (1G) to the 5th generation is still a long process. 2G networks have developed the messaging service, which complements the already operational voice service. 2G technology has rapidly progressed to the third generation (3G), incorporating multimedia data transmission techniques. It then progressed to fourth generation (4G) and LTE (Long Term Evolution), increasing the transmission speed to improve 3G. Currently, developed countries have already moved to 5G. In developing countries, including Burundi, a member of the East African Community (ECA) where more than 80% are connected to 2G technologies, 40% are connected to the 3G network and 25% to the 4G network and are not yet connected to the 5G network and then still a process. The objective of this article is to analyze the coverage of 2G, 3G and 4G networks in Burundi. This analysis will make it possible to identify possible deficits in order to reduce the digital divide between connected urban areas and remote rural areas. Furthermore, this analysis will draw the attention of decision-makers to the need to deploy networks and coverage to allow the population to access mobile and Internet services and thus enable the digitalization of the population. Finally, this article shows the level of coverage, the digital divide and an overview of the deployment of base stations (BTS) throughout the country to promote the transformation and digital inclusion of services.

Dynamic Hand Gesture Recognition Based on Short-Term Sampling Neural Networks

Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Active Vibration Isolation of Micro-Manufacturing Platform Based on Neural Network

The micromation and precision of the Micro-Electromechanical System demand that its manufacturing, measuring and assembling must work in a micro-manufacturing platform with good ability to isolate vibrations. This paper develops a vibration isolation system of micro-manufacturing platform. The brains of many kinds of birds can isolate vibrations well, such as woodpecker’s brain. When a woodpecker pecks the wood at the speed as 1.6 times as the velocity of sound, its brain will tolerate the wallop 1 500 times of the weight of itself without any damage. The isolation mechanics and organic texture of woodpecker’s brain that has good isolation characteristics were studied. A structure model of vibration isolation system for the micro-manufacturing platform is established based on the bionics of the bird’s brain vibration isolation mechanism. In order to isolate effectively the high frequency vibrations from the ground, a rubber layer is used to isolate vibrations passively between the micro-manufacturing platform’s pedestal and the ground. This layer corresponds to the cartilage and muscles in the outer meninges of the bird’s brain. The active vibration isolation technique is adopted to isolate vibrations between the micro-manufacturing platform and the pedestal. Air springs are used as elastic components, which correspond to the interspaces between the outer meninges and the encephala of the bird’s brain. Actuators are made of giant magnetostrictive material, and it corresponds to the nerves and neural muscles linking the meninges and the encephala. The actuators and air springs are arranged vertically in parallel to make use of the giant magnetostrictive actuators effectively. The air springs support almost all weight of the micro-manufacturing platform and the giant magnetostrictive actuators support almost no weight. In order to realize high performance to isolate complex micro-vibration, the control method using a three-layer neural network is presented. This vibration control system takes into account the floor disturbance and the direct disturbance acting on the micro-manufacturing platform. The absolute acceleration of the micro-manufacturing platform is used as the performance index of vibration control. The performance of the control system is tested by numerical simulation. Simulation results show that the active vibration isolation system has good isolation performance against the floor disturbance and the direct disturbance acting on the micro-manufacturing platform in all the frequency range.

A Short-Term Climate Prediction Model Based on a Modular Fuzzy Neural Network

In terms of the modular fuzzy neural network (MFNN) combining fuzzy c-mean (FCM) cluster and single-layer neural network, a short-term climate prediction model is developed. It is found from modeling results that the MFNN model for short-term climate prediction has advantages of simple structure, no hidden layer and stable network parameters because of the assembling of sound functions of the self-adaptive learning, association and fuzzy information processing of fuzzy mathematics and neural network methods. The case computational results of Guangxi flood season (JJA) rainfall show that the mean absolute error (MAE) and mean relative error (MRE) of the prediction during 1998-2002 are 68.8 mm and 9.78%, and in comparison with the regression method, under the conditions of the same predictors and period they are 97.8 mm and 12.28% respectively. Furthermore, it is also found from the stability analysis of the modular model that the change of the prediction results of independent samples with training times in the stably convergent interval of the model is less than 1.3 mm. The obvious oscillation phenomenon of prediction results with training times, such as in the common back-propagation neural network (BPNN) model, does not occur, indicating a better practical application potential of the MFNN model.

A forecasting model for wave heights based on a long short-term memory neural network

To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with sea surface wind and wave heights as training samples.The prediction performance of the model is evaluated,and the error analysis shows that when using the same set of numerically predicted sea surface wind as input,the prediction error produced by the proposed LSTM model at Sta.N01 is 20%,18%and 23%lower than the conventional numerical wave models in terms of the total root mean square error(RMSE),scatter index(SI)and mean absolute error(MAE),respectively.Particularly,for significant wave height in the range of 3–5 m,the prediction accuracy of the LSTM model is improved the most remarkably,with RMSE,SI and MAE all decreasing by 24%.It is also evident that the numbers of hidden neurons,the numbers of buoys used and the time length of training samples all have impact on the prediction accuracy.However,the prediction does not necessary improve with the increase of number of hidden neurons or number of buoys used.The experiment trained by data with the longest time length is found to perform the best overall compared to other experiments with a shorter time length for training.Overall,long short-term memory neural network was proved to be a very promising method for future development and applications in wave forecasting.

Fault Detection and Isolation Based on Neural Networks Case Study: Steam Turbine

The real-time fault diagnosis system is very important for steam turbine generator set due serious fault results in a reduced amount of electricity supply in power plant. A novel real-time fault diagnosis system is proposed by using Levenberg-Marquardt algorithm related to tuning parameters of Artificial Neural Network (ANN). The model of novel fault diagnosis system by using ANN are built and analyzed. Cases of the diagnosis are simulated. The results show that the real-time fault diagnosis system is of high accuracy and quick convergence. It is also found that this model is feasible in real-time fault diagnosis. The steam turbine is used as a power generator by SONELGAZ, an Algerian company located at Cap Djinet town in Boumerdes district. We used this turbine as our main target for the purpose of this analysis. After deep investigation, while keeping our focus on the most sensitive parts within the turbine, the weakest and the strongest points of the system were identified. Those are the points mostly adequate for failure simulations and at which the designed system will be better positioned for irregularities detection during the production process.

Preliminary abnormal electrocardiogram segment screening method for Holter data based on long short-term memory networks

Holter usually monitors electrocardiogram(ECG)signals for more than 24 hours to capture short-lived cardiac abnormalities.In view of the large amount of Holter data and the fact that the normal part accounts for the majority,it is reasonable to design an algorithm that can automatically eliminate normal data segments as much as possible without missing any abnormal data segments,and then take the left segments to the doctors or the computer programs for further diagnosis.In this paper,we propose a preliminary abnormal segment screening method for Holter data.Based on long short-term memory(LSTM)networks,the prediction model is established and trained with the normal data of a monitored object.Then,on the basis of kernel density estimation,we learn the distribution law of prediction errors after applying the trained LSTM model to the regular data.Based on these,the preliminary abnormal ECG segment screening analysis is carried out without R wave detection.Experiments on the MIT-BIH arrhythmia database show that,under the condition of ensuring that no abnormal point is missed,53.89% of normal segments can be effectively obviated.This work can greatly reduce the workload of subsequent further processing.

Improved RBF network application in analog circuit fault isolation

One kind of steepest descent incremental projection learning algorithm for improving the training of radial basis function(RBF)neural network is proposed,which is applied to analog circuit fault isolation.This algorithm simplified the structure of network through optimum output layer coefficient with incremental projection learning(IPL)algorithm,and adjusted the parameters of the neural activation function to control the network scale and improve the network approximation ability.Compared to the traditional algorithm,the improved algorithm has quicker convergence rate and higher isolation precision.Simulation results show that this improved RBF network has much better performance,which can be used in analog circuit fault isolation field.

Dynamic prediction of landslide displacement using singular spectrum analysis and stack long short-term memory network

An accurate landslide displacement prediction is an important part of landslide warning system. Aiming at the dynamic characteristics of landslide evolution and the shortcomings of traditional static prediction models, this paper proposes a dynamic prediction model of landslide displacement based on singular spectrum analysis(SSA) and stack long short-term memory(SLSTM) network. The SSA is used to decompose the landslide accumulated displacement time series data into trend term and periodic term displacement subsequences. A cubic polynomial function is used to predict the trend term displacement subsequence, and the SLSTM neural network is used to predict the periodic term displacement subsequence. At the same time, the Bayesian optimization algorithm is used to determine that the SLSTM network input sequence length is 12 and the number of hidden layer nodes is 18. The SLSTM network is updated by adding predicted values to the training set to achieve dynamic displacement prediction. Finally, the accumulated landslide displacement is obtained by superimposing the predicted value of each displacement subsequence. The proposed model was verified on the Xintan landslide in Hubei Province, China. The results show that when predicting the displacement of the periodic term, the SLSTM network has higher prediction accuracy than the support vector machine(SVM) and auto regressive integrated moving average(ARIMA). The mean relative error(MRE) is reduced by 4.099% and 3.548% respectively, while the root mean square error(RMSE) is reduced by 5.830 mm and 3.854 mm respectively. It is concluded that the SLSTM network model can better simulate the dynamic characteristics of landslides.

Continuous Sign Language Recognition Based on Spatial-Temporal Graph Attention Network

Continuous sign language recognition(CSLR)is challenging due to the complexity of video background,hand gesture variability,and temporal modeling difficulties.This work proposes a CSLR method based on a spatialtemporal graph attention network to focus on essential features of video series.The method considers local details of sign language movements by taking the information on joints and bones as inputs and constructing a spatialtemporal graph to reflect inter-frame relevance and physical connections between nodes.The graph-based multihead attention mechanism is utilized with adjacent matrix calculation for better local-feature exploration,and short-term motion correlation modeling is completed via a temporal convolutional network.We adopted BLSTM to learn the long-termdependence and connectionist temporal classification to align the word-level sequences.The proposed method achieves competitive results regarding word error rates(1.59%)on the Chinese Sign Language dataset and the mean Jaccard Index(65.78%)on the ChaLearn LAP Continuous Gesture Dataset.

Time Series Forecasting Fusion Network Model Based on Prophet and Improved LSTM

Time series forecasting and analysis are widely used in many fields and application scenarios.Time series historical data reflects the change pattern and trend,which can serve the application and decision in each application scenario to a certain extent.In this paper,we select the time series prediction problem in the atmospheric environment scenario to start the application research.In terms of data support,we obtain the data of nearly 3500 vehicles in some cities in China fromRunwoda Research Institute,focusing on the major pollutant emission data of non-road mobile machinery and high emission vehicles in Beijing and Bozhou,Anhui Province to build the dataset and conduct the time series prediction analysis experiments on them.This paper proposes a P-gLSTNet model,and uses Autoregressive Integrated Moving Average model(ARIMA),long and short-term memory(LSTM),and Prophet to predict and compare the emissions in the future period.The experiments are validated on four public data sets and one self-collected data set,and the mean absolute error(MAE),root mean square error(RMSE),and mean absolute percentage error(MAPE)are selected as the evaluationmetrics.The experimental results show that the proposed P-gLSTNet fusion model predicts less error,outperforms the backbone method,and is more suitable for the prediction of time-series data in this scenario.

Short-term load forecasting based on fuzzy neural network

The fuzzy neural network is applied to the short-term load forecasting. The fuzzy rules and fuzzy membership functions of the network are obtained through fuzzy neural network learming. Three inference algorithms, i.e. themultiplicative inference, the maximum inference and the minimum inference, are used for comparison. The learningalgorithms corresponding to the inference methods are derived from back-propagation algorithm. To validate the fuzzyneural network model, the network is used to Predict short-term load by compaing the network output against the realload data from a local power system supplying electricity to a large steel manufacturer. The experimental results aresatisfactory.

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.