Device Anomaly Detection Algorithm Based on Enhanced Long Short-Term Memory Network

The problems in equipment fault detection include data dimension explosion,computational complexity,low detection accuracy,etc.To solve these problems,a device anomaly detection algorithm based on enhanced long short-term memory(LSTM)is proposed.The algorithm first reduces the dimensionality of the device sensor data by principal component analysis(PCA),extracts the strongly correlated variable data among the multidimensional sensor data with the lowest possible information loss,and then uses the enhanced stacked LSTM to predict the extracted temporal data,thus improving the accuracy of anomaly detection.To improve the efficiency of the anomaly detection,a genetic algorithm(GA)is used to adjust the magnitude of the enhancements made by the LSTM model.The validation of the actual data from the pumps shows that the algorithm has significantly improved the recall rate and the detection speed of device anomaly detection,with the recall rate of 97.07%,which indicates that the algorithm is effective and efficient for device anomaly detection in the actual production environment.

利用长短期记忆网络LSTM对赤道太平洋海表面温度短期预报

海表面温度作为海洋中一个最重要的变量,对全球气候、海洋生态等有很大的影响,因此十分有必要对海表面温度(SST)进行预报。深度学习具备高效的数据处理能力,但目前利用深度学习对整个赤道太平洋的SST短期预报及预报技巧的研究仍较少。本文基于最优插值海表面温度(OISST)的日平均SST数据,利用长短期记忆(LSTM)网络构建了未来10天赤道太平洋(10°S~10°N,120°E~80°W)SST的逐日预报模型。LSTM预报模型利用1982~2010年的观测数据进行训练,2011~2020年的观测数据作为初值进行预报和检验评估。结果表明:赤道太平洋东部地区预报均方根误差(RMSE)大于中、西部,东部预报第1天RMSE为0.6℃左右,而中、西部均小于0.3℃。在不同的年际变化位相,预报RMSE在拉尼娜出现时期最大,正常年份次之,厄尔尼诺时期最小,RMSE在拉尼娜时期比在厄尔尼诺时期可达20%。预报偏差整体表现为东正、西负。相关预报技巧上,中部最好,可预报天数基本为10天以上,赤道冷舌附近可预报天数为4~7天,赤道西边部分地区可预报天数为3天。预报模型在赤道太平洋东部地区各月份预报技巧普遍低于西部地区,相比较而言各区域10、11月份预报技巧最低。总的来说,基于LSTM构建的SST预报模型能很好地捕捉到SST在时序上的演变特征,在不同案例中预报表现良好。同时该预报模型依靠数据驱动,能迅速且较好地预报未来10天以内的日平均SST的短期变化。

基于相关性检验的VMD-LSTM耦合模型月径流模拟研究

近年来,极端强降雨和干旱事件频发,流域水文过程的不确定性变化加剧,使得流域中长期径流预测的难度增加。为提升LSTM(长短期记忆神经网络)模型对径流时序变化的捕捉及拟合能力,以博阳河流域为研究区域,选取月降雨、蒸发及流量数据,利用VMD(变分模态分解)和相关性检验,排除无关频率分量对LSTM模型规律学习的干扰,以达到模型输入优选的目的;此外,还考虑了VMD与LSTM模型的不同耦合方式对模型精度和稳定性的影响,最终优选出二者兼具的VMD-LSTM月径流耦合模式。结果表明:VMD-LSTM耦合模型可显著提升模拟精度,但在模型稳定性方面有所欠缺;而基于相关性检验的VMD-LSTM耦合模型不仅能够进一步提高模型精度,并且在模型的稳定性方面也有所改进。在基于相关性检验的VMD-LSTM耦合模型的不同耦合方式对比中,对输入、输出均进行VMD分解且对输入变量进行优选的D_(1)耦合方案的模拟效果最好,其60次模拟计算的NSE均为0.98以上且稳定性极佳;另外,在分析方案D_(1)的可解释性时发现历史径流对于LSTM模型的影响要比降雨和蒸发大。该研究结论可为流域水资源管理提供精准可信的中长期径流模拟成果。

Navigation jamming signal recognition based on long short-term memory neural networks

This paper introduces the time-frequency analyzed long short-term memory(TF-LSTM) neural network method for jamming signal recognition over the Global Navigation Satellite System(GNSS) receiver. The method introduces the long shortterm memory(LSTM) neural network into the recognition algorithm and combines the time-frequency(TF) analysis for signal preprocessing. Five kinds of navigation jamming signals including white Gaussian noise(WGN), pulse jamming, sweep jamming, audio jamming, and spread spectrum jamming are used as input for training and recognition. Since the signal parameters and quantity are unknown in the actual scenario, this work builds a data set containing multiple kinds and parameters jamming to train the TF-LSTM. The performance of this method is evaluated by simulations and experiments. The method has higher recognition accuracy and better robustness than the existing methods, such as LSTM and the convolutional neural network(CNN).

Multi-head attention-based long short-term memory model for speech emotion recognition

To fully make use of information from different representation subspaces,a multi-head attention-based long short-term memory(LSTM)model is proposed in this study for speech emotion recognition(SER).The proposed model uses frame-level features and takes the temporal information of emotion speech as the input of the LSTM layer.Here,a multi-head time-dimension attention(MHTA)layer was employed to linearly project the output of the LSTM layer into different subspaces for the reduced-dimension context vectors.To provide relative vital information from other dimensions,the output of MHTA,the output of feature-dimension attention,and the last time-step output of LSTM were utilized to form multiple context vectors as the input of the fully connected layer.To improve the performance of multiple vectors,feature-dimension attention was employed for the all-time output of the first LSTM layer.The proposed model was evaluated on the eNTERFACE and GEMEP corpora,respectively.The results indicate that the proposed model outperforms LSTM by 14.6%and 10.5%for eNTERFACE and GEMEP,respectively,proving the effectiveness of the proposed model in SER tasks.

A multi-source information fusion layer counting method for penetration fuze based on TCN-LSTM

When employing penetration ammunition to strike multi-story buildings,the detection methods using acceleration sensors suffer from signal aliasing,while magnetic detection methods are susceptible to interference from ferromagnetic materials,thereby posing challenges in accurately determining the number of layers.To address this issue,this research proposes a layer counting method for penetration fuze that incorporates multi-source information fusion,utilizing both the temporal convolutional network(TCN)and the long short-term memory(LSTM)recurrent network.By leveraging the strengths of these two network structures,the method extracts temporal and high-dimensional features from the multi-source physical field during the penetration process,establishing a relationship between the multi-source physical field and the distance between the fuze and the target plate.A simulation model is developed to simulate the overload and magnetic field of a projectile penetrating multiple layers of target plates,capturing the multi-source physical field signals and their patterns during the penetration process.The analysis reveals that the proposed multi-source fusion layer counting method reduces errors by 60% and 50% compared to single overload layer counting and single magnetic anomaly signal layer counting,respectively.The model's predictive performance is evaluated under various operating conditions,including different ratios of added noise to random sample positions,penetration speeds,and spacing between target plates.The maximum errors in fuze penetration time predicted by the three modes are 0.08 ms,0.12 ms,and 0.16 ms,respectively,confirming the robustness of the proposed model.Moreover,the model's predictions indicate that the fitting degree for large interlayer spacings is superior to that for small interlayer spacings due to the influence of stress waves.

A Complex Fuzzy LSTM Network for Temporal-Related Forecasting Problems

Time-stamped data is fast and constantly growing and it contains significant information thanks to the quick development ofmanagement platforms and systems based on the Internet and cutting-edge information communication technologies.Mining the time series data including time series prediction has many practical applications.Many new techniques were developed for use with various types of time series data in the prediction problem.Among those,this work suggests a unique strategy to enhance predicting quality on time-series datasets that the timecycle matters by fusing deep learning methods with fuzzy theory.In order to increase forecasting accuracy on such type of time-series data,this study proposes integrating deep learning approaches with fuzzy logic.Particularly,it combines the long short-termmemory network with the complex fuzzy set theory to create an innovative complex fuzzy long short-term memory model(CFLSTM).The proposed model adds a meaningful representation of the time cycle element thanks to a complex fuzzy set to advance the deep learning long short-term memory(LSTM)technique to have greater power for processing time series data.Experiments on standard common data sets and real-world data sets published in the UCI Machine Learning Repository demonstrated the proposedmodel’s utility compared to other well-known forecasting models.The results of the comparisons supported the applicability of our proposed strategy for forecasting time series data.

Acute Complication Prediction and Diagnosis Model CLSTM-BPR:A Fusion Method of Time Series Deep Learning and Bayesian Personalized Ranking

Acute complication prediction model is of great importance for the overall reduction of premature death in chronic diseases.The CLSTM-BPR proposed in this paper aims to improve the accuracy,interpretability,and generalizability of the existing disease prediction models.Firstly,through its complex neural network structure,CLSTM-BPR considers both disease commonality and patient characteristics in the prediction process.Secondly,by splicing the time series prediction algorithm and classifier,the judgment basis is given along with the prediction results.Finally,this model introduces the pairwise algorithm Bayesian Personalized Ranking(BPR)into the medical field for the first time,and achieves a good result in the diagnosis of six acute complications.Experiments on the Medical Information Mart for Intensive Care IV(MIMIC-IV)dataset show that the average Mean Absolute Error(MAE)of biomarker value prediction of the CLSTM-BPR model is 0.26,and the average accuracy(ACC)of the CLSTM-BPR model for acute complication diagnosis is 92.5%.Comparison experiments and ablation experiments further demonstrate the reliability of CLSTM-BPR in the prediction of acute complication,which is an advancement of current disease prediction tools.

Real-time UAV path planning based on LSTM network

To address the shortcomings of single-step decision making in the existing deep reinforcement learning based unmanned aerial vehicle(UAV)real-time path planning problem,a real-time UAV path planning algorithm based on long shortterm memory(RPP-LSTM)network is proposed,which combines the memory characteristics of recurrent neural network(RNN)and the deep reinforcement learning algorithm.LSTM networks are used in this algorithm as Q-value networks for the deep Q network(DQN)algorithm,which makes the decision of the Q-value network has some memory.Thanks to LSTM network,the Q-value network can use the previous environmental information and action information which effectively avoids the problem of single-step decision considering only the current environment.Besides,the algorithm proposes a hierarchical reward and punishment function for the specific problem of UAV real-time path planning,so that the UAV can more reasonably perform path planning.Simulation verification shows that compared with the traditional feed-forward neural network(FNN)based UAV autonomous path planning algorithm,the RPP-LSTM proposed in this paper can adapt to more complex environments and has significantly improved robustness and accuracy when performing UAV real-time path planning.

Track correlation algorithm based on CNN-LSTM for swarm targets

The rapid development of unmanned aerial vehicle(UAV) swarm, a new type of aerial threat target, has brought great pressure to the air defense early warning system. At present, most of the track correlation algorithms only use part of the target location, speed, and other information for correlation.In this paper, the artificial neural network method is used to establish the corresponding intelligent track correlation model and method according to the characteristics of swarm targets.Precisely, a route correlation method based on convolutional neural networks (CNN) and long short-term memory (LSTM)Neural network is designed. In this model, the CNN is used to extract the formation characteristics of UAV swarm and the spatial position characteristics of single UAV track in the formation,while the LSTM is used to extract the time characteristics of UAV swarm. Experimental results show that compared with the traditional algorithms, the algorithm based on CNN-LSTM neural network can make full use of multiple feature information of the target, and has better robustness and accuracy for swarm targets.

基于LSTM-AEKF算法的锂离子电池SOC估计

针对扩展卡尔曼滤波(Extended Kalman filter,EKF)算法与长短期记忆网络(Long Short-Term Memory,LSTM)不能准确估计锂离子电池荷电状态(State of Charge,SOC)的问题,本文提出了一种基于二阶戴维宁(Thevenin)的等效电路模型,采用自适应扩展卡尔曼滤波(Adaptve Extended Kalman filter,AEKF)与LSTM相结合的SOC估计算法,即LSTM-AEKF算法。在二阶RC等效电路模型的基础上建立整数阶模型,并采用EKF算法辨识模型参数,采用LSTM-AEKF算法估计SOC,与AEKF算法、LSTM算法进行比较。根据马里兰大学公开数据集进行测试,结果表明,与传统方法相比,LSTM-AEKF算法估计SOC的平均绝对误差(Mean Absolute Error,MAE)与均方根误差(Root Mean Square Error,RMSE)分别下降了1.23%、1.5%,基于二阶RC模型的LSTM-AEKF算法可以有效估计SOC。

Deep Learning-Based Stock Price Prediction Using LSTM Model

The stock market is a vital component of the broader financial system,with its dynamics closely linked to economic growth.The challenges associated with analyzing and forecasting stock prices have persisted since the inception of financial markets.By examining historical transaction data,latent opportunities for profit can be uncovered,providing valuable insights for both institutional and individual investors to make more informed decisions.This study focuses on analyzing historical transaction data from four banks to predict closing price trends.Various models,including decision trees,random forests,and Long Short-Term Memory(LSTM)networks,are employed to forecast stock price movements.Historical stock transaction data serves as the input for training these models,which are then used to predict upward or downward stock price trends.The study’s empirical results indicate that these methods are effective to a degree in predicting stock price movements.The LSTM-based deep neural network model,in particular,demonstrates a commendable level of predictive accuracy.This conclusion is reached following a thorough evaluation of model performance,highlighting the potential of LSTM models in stock market forecasting.The findings offer significant implications for advancing financial forecasting approaches,thereby improving the decision-making capabilities of investors and financial institutions.

State of Health Estimation of Lithium-Ion Batteries Using Support Vector Regression and Long Short-Term Memory

Lithium-ion batteries are the most widely accepted type of battery in the electric vehicle industry because of some of their positive inherent characteristics. However, the safety problems associated with inaccurate estimation and prediction of the state of health of these batteries have attracted wide attention due to the adverse negative effect on vehicle safety. In this paper, both machine and deep learning models were used to estimate the state of health of lithium-ion batteries. The paper introduces the definition of battery health status and its importance in the electric vehicle industry. Based on the data preprocessing and visualization analysis, three features related to actual battery capacity degradation are extracted from the data. Two learning models, SVR and LSTM were employed for the state of health estimation and their respective results are compared in this paper. The mean square error and coefficient of determination were the two metrics for the performance evaluation of the models. The experimental results indicate that both models have high estimation results. However, the metrics indicated that the SVR was the overall best model.

Data-Driven Fault Detection of Multiple Open-Circuit Faults for MMC Systems Based on Long Short-Term Memory Networks

This paper presents a long short-term memory(LSTM)-based fault detection method to detect the multiple open-circuit switch faults of modular multilevel converter(MMC)systems with full-bridge sub-modules(FB-SMs).Eighteen sensor signals of grid voltages,grid currents and capacitance voltages of MMC for single and multi-switch faults are collected as sampling data.The output signal characteristics of four types of single switch faults of FB-SM,as well as double switch faults in the same and different phases of MMC,are analyzed under the conditions of load variations and control command changes.A multi-layer LSTM network is devised to deeply extract the fault characteristics of MMC under different faults and operation conditions,and a Softmax layer detects the fault types.Simulation results have confirmed that the proposed LSTM-based method has better detection performance compared with three other methods:K-nearest neighbor(KNN),naive bayes(NB)and recurrent neural network(RNN).In addition,it is highly robust to model uncertainties and Gaussian noise.The validity of the proposed method is further demonstrated by experiment studies conducted on a hardware-in-the-loop(HIL)testing platform.

LSTM-DPPO based deep reinforcement learning controller for path following optimization of unmanned surface vehicle

To solve the path following control problem for unmanned surface vehicles(USVs),a control method based on deep reinforcement learning(DRL)with long short-term memory(LSTM)networks is proposed.A distributed proximal policy opti-mization(DPPO)algorithm,which is a modified actor-critic-based type of reinforcement learning algorithm,is adapted to improve the controller performance in repeated trials.The LSTM network structure is introduced to solve the strong temporal cor-relation USV control problem.In addition,a specially designed path dataset,including straight and curved paths,is established to simulate various sailing scenarios so that the reinforcement learning controller can obtain as much handling experience as possible.Extensive numerical simulation results demonstrate that the proposed method has better control performance under missions involving complex maneuvers than trained with limited scenarios and can potentially be applied in practice.

光伏并网逆变器参数性故障的VMD-WPE和MPA-LSTM诊断方法研究

针对三相光伏并网逆变器参数性故障的特征量提取难、诊断准确率较低等问题,提出变分模态分解的小波包能量特征与海洋捕食者算法优化长短期记忆神经网络相结合的故障诊断方法。首先,以逆变器三相线电压为原始数据,以最小样本熵为准则优化变分模态分解的模态数;然后,利用小波包分解提取变分模态分解各模态分量的小波包能量作为故障特征量;最后,利用海洋捕食者算法优化长短期记忆网络超参数实现故障的参数性辨识。对比分析结果表明,所提方法用于光伏并网逆变器参数性故障诊断具有可行性和精确性。

Motion simulation of moorings using optimized LSTM neural network

Mooring arrays have been widely deployed in sustained ocean observation in high resolution to measure finer dynamic features of marine phenomena.However,the irregular posture changes and nonlinear response of moorings under the effect of ocean currents face huge challenges for the deployment of mooring arrays,which may cause the deviations of measurements and yield a vacuum of observation in the upper ocean.We developed a data-driven mooring simulation model based on LSTM(long short-term memory)neural network,coupling the ocean current with position data from moorings to predict the motion of moorings,including single-step output prediction and multi-step prediction.Based on the predictive information,the formation of the mooring array can be adjusted to improve the accuracy and integrity of measurements.Moreover,we proposed the cuckoo search(CS)optimization algorithm to tune the parameters of LSTM,which improves the robustness and generalization of the model.We utilize the datasets observed from moorings anchored in the Kuroshio Extension region to train and validate the simulation model.The experimental results demonstrate that the model can remarkably improve prediction accuracy and yield stable performance.Moreover,compared with other optimization algorithms,CS is more efficient and performs better in simulating the motion of moorings.

基于注意力机制的LSTM时变水声信道深度学习预测

This paper investigates the channel prediction algorithm of the time-varying channels in underwater acoustic(UWA)communication systems using the long short-term memory(LSTM)model with the attention mechanism.AttLstmPreNet is a deep learning model that combines an attention mechanism with LSTM-type models to capture temporal information with different scales from historical UWA channels.The attention mechanism is used to capture sparsity in the time-delay scales and coherence in the gep-time scale under the LSTM framework.The soft attention mechanism is introduced before the LSTM to support the model to focus on the features of input sequences and help improve the learning capacity of the proposed model.The performance of the proposed model is validated using different simulation time-varying UWA channels.Compared with the adaptive channel predictors and the plain LSTM model,the proposed model is better in terms of channel prediction accuracy.

Scenario-based Optimal Real-time Charging Strategy of Electric Vehicles with Bayesian Long Short-term Memory Networks

Joint operation optimization for electric vehicles(EVs)and on-site or adjacent photovoltaic generation(PVG)are pivotal to maintaining the security and economics of the operation of the power system concerned.Conventional offline optimization algorithms lack real-time applicability due to uncertainties involved in the charging service of an EV charging station(EVCS).Firstly,an optimization model for real-time EV charging strategy is proposed to address these challenges,which accounts for environmental uncertainties of an EVCS,encompassing EV arrivals,charging demands,PVG outputs,and the electricity price.Then,a scenario-based two-stage optimization approach is formulated.The scenarios of the underlying uncertain environmental factors are generated by the Bayesian long short-term memory(B-LSTM)network.Finally,numerical results substantiate the efficacy of the proposed optimization approach,and demonstrate superior profitability compared with prevalent approaches.

基于VMD和时空网络变分自编码器的负荷聚类

为了解决用户用电负荷曲线数据维度高、特征提取困难以及序列存在信号模态混叠的问题,提出了使用变分模态分解(variational modal decomposition,VMD)和改进基于时空网络的变分自编码器(variational auto-encoder,VAE)对电力负荷曲线进行特征提取。通过模态分解得到信号的固有模态,对模态重构得到时序特征较明显的序列信号。再通过长短期记忆网络(long short-term memory network,LSTM)和卷积网络(convolutional neural network,CNN)组成的时空变分自编码器进行潜在特征提取,并构建网络分类器来联合损失优化自编码器模型。最后使用Minibatchkmeans算法聚类并计算聚类中心。使用UCI数据集中葡萄牙居民用电量作为实验数据,通过实验结果表明经模态分解后通过降维再聚类的算法在戴维斯丁堡指数(Davies-Bouldin index,DBI)和轮廓系数(silhouette coefficient,SC)上表现出较好效果。