A Complex Fuzzy LSTM Network for Temporal-Related Forecasting Problems

Time-stamped data is fast and constantly growing and it contains significant information thanks to the quick development ofmanagement platforms and systems based on the Internet and cutting-edge information communication technologies.Mining the time series data including time series prediction has many practical applications.Many new techniques were developed for use with various types of time series data in the prediction problem.Among those,this work suggests a unique strategy to enhance predicting quality on time-series datasets that the timecycle matters by fusing deep learning methods with fuzzy theory.In order to increase forecasting accuracy on such type of time-series data,this study proposes integrating deep learning approaches with fuzzy logic.Particularly,it combines the long short-termmemory network with the complex fuzzy set theory to create an innovative complex fuzzy long short-term memory model(CFLSTM).The proposed model adds a meaningful representation of the time cycle element thanks to a complex fuzzy set to advance the deep learning long short-term memory(LSTM)technique to have greater power for processing time series data.Experiments on standard common data sets and real-world data sets published in the UCI Machine Learning Repository demonstrated the proposedmodel’s utility compared to other well-known forecasting models.The results of the comparisons supported the applicability of our proposed strategy for forecasting time series data.

Real-time UAV path planning based on LSTM network

To address the shortcomings of single-step decision making in the existing deep reinforcement learning based unmanned aerial vehicle(UAV)real-time path planning problem,a real-time UAV path planning algorithm based on long shortterm memory(RPP-LSTM)network is proposed,which combines the memory characteristics of recurrent neural network(RNN)and the deep reinforcement learning algorithm.LSTM networks are used in this algorithm as Q-value networks for the deep Q network(DQN)algorithm,which makes the decision of the Q-value network has some memory.Thanks to LSTM network,the Q-value network can use the previous environmental information and action information which effectively avoids the problem of single-step decision considering only the current environment.Besides,the algorithm proposes a hierarchical reward and punishment function for the specific problem of UAV real-time path planning,so that the UAV can more reasonably perform path planning.Simulation verification shows that compared with the traditional feed-forward neural network(FNN)based UAV autonomous path planning algorithm,the RPP-LSTM proposed in this paper can adapt to more complex environments and has significantly improved robustness and accuracy when performing UAV real-time path planning.

Feedback LSTM Network Based on Attention for Image Description Generator

Images are complex multimedia data which contain rich semantic information.Most of current image description generator algorithms only generate plain description,with the lack of distinction between primary and secondary object,leading to insufficient high-level semantic and accuracy under public evaluation criteria.The major issue is the lack of effective network on high-level semantic sentences generation,which contains detailed description for motion and state of the principal object.To address the issue,this paper proposes the Attention-based Feedback Long Short-Term Memory Network(AFLN).Based on existing codec framework,there are two independent sub tasks in our method:attention-based feedback LSTM network during decoding and the Convolutional Block Attention Module(CBAM)in the coding phase.First,we propose an attentionbased network to feedback the features corresponding to the generated word from the previous LSTM decoding unit.We implement feedback guidance through the related field mapping algorithm,which quantifies the correlation between previous word and latter word,so that the main object can be tracked with highlighted detailed description.Second,we exploit the attention idea and apply a lightweight and general module called CBAM after the last layer of VGG 16 pretraining network,which can enhance the expression of image coding features by combining channel and spatial dimension attention maps with negligible overheads.Extensive experiments on COCO dataset validate the superiority of our network over the state-of-the-art algorithms.Both scores and actual effects are proved.The BLEU 4 score increases from 0.291 to 0.301 while the CIDEr score rising from 0.912 to 0.952.

Multi-step ahead short-term predictions of storm surge level using CNN and LSTM network

Storm surges pose significant danger and havoc to the coastal residents’safety,property,and lives,particularly at offshore locations with shallow water levels.Predictions of storm surges with hours of warning time are important for evacuation measures in low-lying regions and coastal management plans.In addition to experienced predictions and numerical models,artificial intelligence(AI)techniques are also being used widely for short-term storm surge prediction owing to their merits in good level of prediction accuracy and rapid computations.Convolutional neural network(CNN)and long short-term memory(LSTM)are two of the most important models among AI techniques.However,they have been scarcely utilised for surge level(SL)forecasting,and combinations of the two models are even rarer.This study applied CNN and LSTM both individually and in combination towards multi-step ahead short-term storm surge level prediction using observed SL and wind information.The architectures of the CNN,LSTM,and two sequential techniques of combining the models(LSTM–CNN and CNN–LSTM)were constructed via a trial-and-error approach and knowledge obtained from previous studies.As a case study,11 a of hourly observed SL and wind data of the Xiuying Station,Hainan Province,China,were organised as inputs for training to verify the feasibility and superiority of the proposed models.The results show that CNN and LSTM had evident advantages over support vector regression(SVR)and multilayer perceptron(MLP),and the combined models outperformed the individual models(CNN and LSTM),mostly by 4%–6%.However,on comparing the model computed predictions during two severe typhoons that resulted in extreme storm surges,the accuracy was found to improve by over 10%at all forecasting steps.

Dynamic Hand Gesture Recognition Using 3D-CNN and LSTM Networks

Recognition of dynamic hand gestures in real-time is a difficult task because the system can never know when or from where the gesture starts and ends in a video stream.Many researchers have been working on visionbased gesture recognition due to its various applications.This paper proposes a deep learning architecture based on the combination of a 3D Convolutional Neural Network(3D-CNN)and a Long Short-Term Memory(LSTM)network.The proposed architecture extracts spatial-temporal information from video sequences input while avoiding extensive computation.The 3D-CNN is used for the extraction of spectral and spatial features which are then given to the LSTM network through which classification is carried out.The proposed model is a light-weight architecture with only 3.7 million training parameters.The model has been evaluated on 15 classes from the 20BN-jester dataset available publicly.The model was trained on 2000 video-clips per class which were separated into 80%training and 20%validation sets.An accuracy of 99%and 97%was achieved on training and testing data,respectively.We further show that the combination of 3D-CNN with LSTM gives superior results as compared to MobileNetv2+LSTM.

Behavior recognition based on the fusion of 3D-BN-VGG and LSTM network

In order to effectively solve the problems of low accuracy,large amount of computation and complex logic of deep learning algorithms in behavior recognition,a kind of behavior recognition based on the fusion of 3 dimensional batch normalization visual geometry group(3D-BN-VGG)and long short-term memory(LSTM)network is designed.In this network,3D convolutional layer is used to extract the spatial domain features and time domain features of video sequence at the same time,multiple small convolution kernels are stacked to replace large convolution kernels,thus the depth of neural network is deepened and the number of network parameters is reduced.In addition,the latest batch normalization algorithm is added to the 3-dimensional convolutional network to improve the training speed.Then the output of the full connection layer is sent to LSTM network as the feature vectors to extract the sequence information.This method,which directly uses the output of the whole base level without passing through the full connection layer,reduces the parameters of the whole fusion network to 15324485,nearly twice as much as those of 3D-BN-VGG.Finally,it reveals that the proposed network achieves 96.5%and 74.9%accuracy in the UCF-101 and HMDB-51 respectively,and the algorithm has a calculation speed of 1066 fps and an acceleration ratio of 1,which has a significant predominance in velocity.

Convolutional LSTM Network for Heart Disease Diagnosis on Electrocardiograms

Heart disease is a leading cause ofmortality worldwide.Electrocardiograms(ECG)play a crucial role in diagnosing heart disease.However,interpreting ECGsignals necessitates specialized knowledge and training.The development of automated methods for ECG analysis has the potential to enhance the accuracy and efficiency of heart disease diagnosis.This research paper proposes a 3D Convolutional Long Short-Term Memory(Conv-LSTM)model for detecting heart disease using ECG signals.The proposed model combines the advantages of both convolutional neural networks(CNN)and long short-term memory(LSTM)networks.By considering both the spatial and temporal dependencies of ECG,the 3D Conv-LSTM model enables the detection of subtle changes in the signal over time.The model is trained on a dataset of ECG recordings from patients with various heart conditions,including arrhythmia,myocardial infarction,and heart failure.Experimental results show that the proposed 3D Conv-LSTM model outperforms traditional 2D CNN models in detecting heart disease,achieving an accuracy of 88%in the classification of five classes.Furthermore,themodel outperforms the other state-of-the-art deep learning models for ECG-based heart disease detection.Moreover,the proposedConv-LSTMnetwork yields highly accurate outcomes in identifying abnormalities in specific ECG leads.The proposed 3D Conv-LSTM model holds promise as a valuable tool for automated heart disease detection and diagnosis.This study underscores the significance of incorporating spatial and temporal dependencies in ECG-based heart disease detection.It highlights the potential of deep-learning models in enhancing the accuracy and efficiency of diagnosis.

A multi-target stance detection based on Bi-LSTM network with position-weight

In the task of multi-target stance detection,there are problems the mutual influence of content describing different targets,resulting in reduction in accuracy.To solve this problem,a multi-target stance detection algorithm based on a bidirectional long short-term memory(Bi-LSTM)network with position-weight is proposed.First,the corresponding position of the target in the input text is calculated with the ultimate position-weight vector.Next,the position information and output from the Bi-LSTM layer are fused by the position-weight fusion layer.Finally,the stances of different targets are predicted using the LSTM network and softmax classification.The multi-target stance detection corpus of the American election in 2016 is used to validate the proposed method.The results demonstrate that the Bi-LSTM network with position-weight achieves an advantage of 1.4%in macro average F1 value in the comparison of recent algorithms.

Behavior recognition algorithm based on the improved R3D and LSTM network fusion

Because behavior recognition is based on video frame sequences,this paper proposes a behavior recognition algorithm that combines 3D residual convolutional neural network(R3D)and long short-term memory(LSTM).First,the residual module is extended to three dimensions,which can extract features in the time and space domain at the same time.Second,by changing the size of the pooling layer window the integrity of the time domain features is preserved,at the same time,in order to overcome the difficulty of network training and over-fitting problems,the batch normalization(BN)layer and the dropout layer are added.After that,because the global average pooling layer(GAP)is affected by the size of the feature map,the network cannot be further deepened,so the convolution layer and maxpool layer are added to the R3D network.Finally,because LSTM has the ability to memorize information and can extract more abstract timing features,the LSTM network is introduced into the R3D network.Experimental results show that the R3D+LSTM network achieves 91%recognition rate on the UCF-101 dataset.

Prediction of surface subsidence in Changchun City based on LSTM network

Monitoring and predicting of urban surface subsidence are important for urban disaster prevention and mitigation.In this paper,the Long Short-Term Memory(LSTM)network was used to predict the surface subsidence process of Changchun City from 2018 to 2020 based on PS-InSAR monitoring data.The results show that the prediction error of 57.89% of PS points in the LSTM network was less than 1mm with the average error of 1.8 mm and the standard deviation of 2.8 mm.The accuracy and reliability of the prediction were better than regression analysis,time series analysis and grey model.

Combination of WRF Model and LSTM Network for Solar Radiation Forecasting—Timor Leste Case Study

A study of a combination of Weather Research and Forecasting (WRF) model and Long Short Term Memory (LSTM) network for location in Dili Timor Leste is introduced in this paper. One calendar year’s results of solar radiation from January to December 2014 are used as input data to estimate future forecasting of solar radiation using the LSTM network for three months period. The WRF model version 3.9.1 is used to simulate one year’s solar radiation in horizontal resolution low scale for nesting domain 1 × 1 km. It is done by applying 6-hourly interval 1º × 1º NCEP FNL analysis data used as Global Forecast System (GFS). LSTM network is applied for forecasting in numerous learning problems for solar radiation forecasting. LSTM network uses two-layer LSTM architecture of 512 hidden neurons coupled with a dense output layer with linear as the model activation to predict with time steps are configured to 50 and the number of features is 1. The maximum epoch is set to 325 with batch size 300 and the validation split is 0.09. The results demonstrate that the combination of these two methods can successfully predict solar radiation where four error metrics of mean bias error (MBE), root mean square error (RMSE), normalized MBE (nMBE), and normalized RMSE (nRMSE) perform small error distribution and percentage in three months prediction where the error percentage is obtained below the 20% for nMBE and nRMSE. Meanwhile, the error distribution of RMSE is obtained below 200 W/m2 and maximum bias error is 0.07. Finally, the values of MBE, RMSE, nMBE, and nRMSE conclude that the good performance of the combination of two methods in this study can be applied to simulate any other weather variable for local necessary.

S-UNet:基于U-Net和LSTM的短临降水预报网络

随着深度学习技术的发展,多数研究工作将短临降水预报视为雷达回波序列的预测任务.由于降水复杂性的非线性时空变换,现有的短临预报存在准确性低、外推时效短、难以应对复杂的非线性时空变换等缺点.为解决以上问题,本文基于U-Net和LSTM提出了S-UNet短临降水预报网络.首先本文提出了S-UNet layer (SL)模块,以帮助网络更好地提取雷达序列特征,构建时空变化的整体趋势,从而提高网络效率,增加网络的外推时长.其次,为更好地应对雷达回波的变形、积累和消散的复杂性,增强网络对复杂的空间关系的捕获能力和运动轨迹的模拟能力,本文基于LSTM构建了雷达特征模块radar feature (RF).最后,将SL模块和RF模块与U-Net框架结合,提出了S-UNet短临降水预报网络,并在KNMI数据集上实现了先进的性能.实验结果表明,在KNMI的NL-50和NL-20数据集上,本文所提的方法与主流方法相比,海德克技能得分和关键成功指数分别提高了5.25%(6.57%)和2.17%(4.75%),达到了0.30 (0.29)和0.72 (0.58);准确率提高了2.10%(1.35%),达到了0.80 (0.80);假接受率降低了4.27%(1.80%),达到了0.24 (0.38).除此之外,本文通过消融实验证明了所提出模块及结合方法的有效性.

区域船舶交通流量预测ChebNet-LSTM模型

针对船舶交通流量预测模型对船舶流量数据空间特征考虑较少的问题,建立一种由切比雪夫网络(Chebyshev network,ChebNet)和长短期记忆网络(long short-term memory,LSTM)组成的深度学习预测模型ChebNet-LSTM。ChebNet的K阶卷积算子有利于提取船舶流量数据的空间特征,而LSTM用于学习船舶流量数据的时间特征。选取舟山水域中船舶流量不同的3个区域进行船舶流量预测实验。结果表明,所提出的ChebNet-LSTM模型可以有效地提取船舶流量数据的时空特征,在各项评价指标上的表现均优于对比模型,预测精度得到较大提高,可以为水上交通智能航行提供数据支撑。

基于注意力机制的ResNet-LSTM煤矿瓦斯浓度预测模型

对煤矿井下瓦斯浓度的预测一直以来是矿井安全进行早期预警和管理的关键问题。为了进一步提高煤矿瓦斯浓度预测的准确度,提出了一种基于深度学习的方法,称为AR-LSTM,它包括残差神经网络(ResNet)、长短时记忆(LSTM)网络和基于注意力的网络,用于煤矿井下瓦斯浓度的预测。AR-LSTM不仅使用瓦斯浓度这一变量,同时将采集的温度、风速和一氧化碳浓度作为输入。因此,在AR-LSTM模型中,ResNet-LSTM网络学习多变量时间序列数据的时序相关性和相互依赖性,注意力机制用于捕捉过去不同时间步的特征状态对未来瓦斯浓度的重要性程度。基于注意力的层可以自动加权过去的特征状态以提高预测准确性,使用煤矿地区的瓦斯浓度数据进行预测,并将其与3种基准方法进行比较。为了比较每种方法的整体性能,实验中使用了均方根误差E_(RMS)、平均绝对误差E_(MA)和决定系数R^(2)。实验结果表明,AR-LSTM模型能够以最高性能处理煤矿瓦斯浓度的预测问题,并且可以实现1步或多步提前预测。

LSTM Based Neural Network Model for Anomaly Event Detection in Care-Independent Smart Homes

This study introduces a long-short-term memory(LSTM)-based neural network model developed for detecting anomaly events in care-independent smart homes,focusing on the critical application of elderly fall detection.It balances the dataset using the Synthetic Minority Over-sampling Technique(SMOTE),effectively neutralizing bias to address the challenge of unbalanced datasets prevalent in time-series classification tasks.The proposed LSTM model is trained on the enriched dataset,capturing the temporal dependencies essential for anomaly recognition.The model demonstrated a significant improvement in anomaly detection,with an accuracy of 84%.The results,detailed in the comprehensive classification and confusion matrices,showed the model’s proficiency in distinguishing between normal activities and falls.This study contributes to the advancement of smart home safety,presenting a robust framework for real-time anomaly monitoring.

LSTM Network-Based Adaptation Approach for Dynamic Integration in Intelligent End-Edge-Cloud Systems

Edge computing, which migrates compute-intensive tasks to run on the storage resources of edge devices, efficiently reduces data transmission loss and protects data privacy. However, due to limited computing resources and storage capacity, edge devices fail to support real-time streaming data query and processing. To address this challenge, first, we propose a Long Short-Term Memory (LSTM) network-based adaptive approach in the intelligent end-edge-cloud system. Specifically, we maximize the Quality of Experience (QoE) of users by automatically adapting their resource requirements to the storage capacity of edge devices through an event mechanism. Second, to reduce the uncertainty and non-complete adaption of the edge device towards the user’s requirements, we use the LSTM network to analyze the storage capacity of the edge device in real time. Finally, the storage features of the edge devices are aggregated to the cloud to re-evaluate the comprehensive capability of the edge devices and ensure the fast response of the user devices during the dynamic adaptation matching process. A series of experimental results show that the proposed approach has superior performance compared with traditional centralized and matrix decomposition based approaches.

A multi-source information fusion layer counting method for penetration fuze based on TCN-LSTM

When employing penetration ammunition to strike multi-story buildings,the detection methods using acceleration sensors suffer from signal aliasing,while magnetic detection methods are susceptible to interference from ferromagnetic materials,thereby posing challenges in accurately determining the number of layers.To address this issue,this research proposes a layer counting method for penetration fuze that incorporates multi-source information fusion,utilizing both the temporal convolutional network(TCN)and the long short-term memory(LSTM)recurrent network.By leveraging the strengths of these two network structures,the method extracts temporal and high-dimensional features from the multi-source physical field during the penetration process,establishing a relationship between the multi-source physical field and the distance between the fuze and the target plate.A simulation model is developed to simulate the overload and magnetic field of a projectile penetrating multiple layers of target plates,capturing the multi-source physical field signals and their patterns during the penetration process.The analysis reveals that the proposed multi-source fusion layer counting method reduces errors by 60% and 50% compared to single overload layer counting and single magnetic anomaly signal layer counting,respectively.The model's predictive performance is evaluated under various operating conditions,including different ratios of added noise to random sample positions,penetration speeds,and spacing between target plates.The maximum errors in fuze penetration time predicted by the three modes are 0.08 ms,0.12 ms,and 0.16 ms,respectively,confirming the robustness of the proposed model.Moreover,the model's predictions indicate that the fitting degree for large interlayer spacings is superior to that for small interlayer spacings due to the influence of stress waves.

自适应注意力LSTM-ResNet下的滚动轴承故障诊断

滚动轴承信号具有复杂性和非线性的特点,对特征提取和故障分类带来挑战。为解决上述问题,文章提出一种自适应注意力LSTM-Resnet(long short-term memory residual network)下的滚动轴承故障诊断方法。首先设计双向LSTM组特征提取模型,获取复杂运行条件下的滚动轴承特征;然后,提出自适应注意力LSTM-ResNet完成特征学习,并自适应调整模型中关键特征的权重;最后,采用全局平均池化(global average pooling,GAP)方法结合Softmax模型缓解模型过拟合并完成故障分类。在数据集中完成滚动轴承故障分类,实验结果表明:文章方法的滚动轴承故障诊断准确率相对于SVD-ResNet方法和宽卷积模型更高,并且能在标记样本数量较少和噪声环境下均达到较高的检测准确率,具有更高的准确性和更强的鲁棒性。

Optimization of LSTM Ship Trajectory Prediction Based on Hybrid Genetic Algorithm

Accurate prediction of the movement trajectory of sea surface targets holds significant importance in achieving an advantageous position in the sea battle field.This prediction plays a crucial role in ensuring security defense and confrontation,and is essential for effective deployment of military strategy.Accurately predicting the trajectory of sea surface targets using AIS(Automatic Identification System)information is crucial for security defense and confrontation,and holds significant importance for military strategy deployment.In response to the problem of insufficient accuracy in ship trajectory prediction,this study proposes a hybrid genetic algorithm to optimize the Long Short-Term Memory(LSTM)algorithm.The HGA-LSTM algorithm is proposed for ship trajectory prediction.It can converge faster and obtain better parameter solutions,thereby improving the effectiveness of ship trajectory prediction.Compared to traditional LSTM and GA-LSTM algorithms,experimental results demonstrate that this algorithm outperforms them in both single-step and multi-step prediction.

Remaining useful life prediction for train bearing based on an ILSTM network with adaptive hyperparameter optimization

Remaining useful life(RUL)prediction for bearing is a significant part of the maintenance of urban rail transit trains.Bearing RUL is closely linked to the reliability and safety of train running,but the current prediction accuracy makes it difficult to meet the re-quirements of high reliability operation.Aiming at the problem,a prediction model based on an improved long short-term memory(ILSTM)network is proposed.Firstly,the variational mode decomposition is used to process the signal,the intrinsic mode function with stronger representation ability is determined according to energy entropy and the degradation feature data is constructed com-bined with the time domain characteristics.Then,to improve learning ability,a rectified linear unit(ReLU)is applied to activate a fully connected layer lying after the long short-term memory(LSTM)network,and the hidden state outputs of the layer are weighted by attention mechanism.The Harris Hawks optimization algorithm is introduced to adaptively set the hyperparameters to improve the performance of the LSTM.Finally,the ILSTM is applied to predict bearing RUL.Through experimental cases,the better perfor-mance in bearing RUL prediction and the effectiveness of each improving measures of the model are validated,and its superiority of hyperparameters setting is demonstrated.