Predicting and Curing Depression Using Long Short Term Memory and Global Vector

In today’s world, there are many people suffering from mentalhealth problems such as depression and anxiety. If these conditions are notidentified and treated early, they can get worse quickly and have far-reachingnegative effects. Unfortunately, many people suffering from these conditions,especially depression and hypertension, are unaware of their existence until theconditions become chronic. Thus, this paper proposes a novel approach usingBi-directional Long Short-Term Memory (Bi-LSTM) algorithm and GlobalVector (GloVe) algorithm for the prediction and treatment of these conditions.Smartwatches and fitness bands can be equipped with these algorithms whichcan share data with a variety of IoT devices and smart systems to betterunderstand and analyze the user’s condition. We compared the accuracy andloss of the training dataset and the validation dataset of the two modelsnamely, Bi-LSTM without a global vector layer and with a global vector layer.It was observed that the model of Bi-LSTM without a global vector layer hadan accuracy of 83%,while Bi-LSTMwith a global vector layer had an accuracyof 86% with a precision of 86.4%, and an F1 score of 0.861. In addition toproviding basic therapies for the treatment of identified cases, our model alsohelps prevent the deterioration of associated conditions, making our methoda real-world solution.

一种基于long short-term memory的唇语识别方法

唇动视觉信息是说话内容的重要载体。受嘴唇外观、背景信息和说话习惯等影响,即使说话者说相同的内容,唇动视觉信息也会相差很大。为解决唇语视觉信息多样性的问题,提出一种基于long short-term memory(LSTM)的新的唇语识别方法。以往大多数的方法从嘴唇外表信息入手。本方法用嘴唇关键点坐标描述嘴唇形变信息作为唇语视频的特征,它具有类内一致性和类间区分性的特点。然后利用LSTM对特征进行时序编码,它能学习具有区分性和泛化性的空间-时序特征。在公开的唇语数据集GRID、MIRACL-VC和Oulu VS上对本方法做了针对分割的单词或短语的说话者独立的唇语识别评估。在GRID和MIRACL-VC上,本方法的准确率比传统方法至少高30%;在Oulu VS上,本方法的准确率接近于最优结果。以上实验结果表明,本文提出的基于LSTM的唇语识别方法有效地解决了唇语视觉信息多样性的问题。

Tool Health Condition Recognition Method for High Speed Milling of Titanium Alloy Based on Principal Component Analysis (PCA) and Long Short Term Memory (LSTM)

The healthy condition of the milling tool has a very high impact on the machining quality of the titanium components.Therefore,it is important to recognize the healthy condition of the tool and replace the damaged cutter at the right time.In order to recognize the health condition of the milling cutter,a method based on the long short term memory(LSTM)was proposed to recognize tool health state in this paper.The various signals collected in the tool wear experiments were analyzed by time-domain statistics,and then the extracted data were generated by principal component analysis(PCA)method.The preprocessed data extracted by PCA is transmitted to the LSTM model for recognition.Compared with back propagation neural network(BPNN)and support vector machine(SVM),the proposed method can effectively utilize the time-domain regulation in the data to achieve higher recognition speed and accuracy.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Conditional Random Field Tracking Model Based on a Visual Long Short Term Memory Network

In dense pedestrian tracking,frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories.In this study,a conditional random field tracking model is established by using a visual long short term memory network in the three-dimensional(3D)space and the motion estimations jointly performed on object trajectory segments.Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation.To address the uncertainty of the length and interval of trajectory segments,a multimode long short term memory network is proposed for the object motion estimation.The tracking performance is evaluated using the PETS2009 dataset.The experimental results show that the proposed method achieves better performance than the tracking methods based on the independent motion estimation.

Short-Term Relay Quality Prediction Algorithm Based on Long and Short-Term Memory

The fraction defective of semi-finished products is predicted to optimize the process of relay production lines, by which production quality and productivity are increased, and the costs are decreased. The process parameters of relay production lines are studied based on the long-and-short-term memory network. Then, the Keras deep learning framework is utilized to build up a short-term relay quality prediction algorithm for the semi-finished product. A simulation model is used to study prediction algorithm. The simulation results show that the average prediction absolute error of the fraction is less than 5%. This work displays great application potential in the relay production lines.

Analyses of fear memory in Arc/Arg3.1-deficient mice: intact short-term memory and impaired long-term and remote memory

Activity-regulated cytoskeleton-associated protein (Arc/Arg3.1) was originally identified in patients with seizures. It is densely distributed in the hip-pocampus and amygdala in particular. Because the expression of Arc/Arg3.1 is regulated by nerve in-puts, it is thought to be an immediate early gene. As shown both in vitro and in vivo, Arc/Arg3.1 is in-volved in synaptic consolidation and regulates some forms of learning and memory in rats and mice [1,2]. Furthermore, a recent study suggests that Arc/Arg3.1 may play a significant role in signal transmission via AMPA-type glutamate receptors [3-5]. Therefore, we conducted a detailed analysis of fear memory in Arc/Arg3.1-deficient mice. As previously reported, the knockout animals exhib-ited impaired fear memory in both contextual and cued test situations. Although Arc/Arg3.1-deficient mice showed almost the same performance as wild-type littermates 4 hr after a conditioning trial, their performance was impaired in the retention test after 24 hr or longer, either with or without reconsolidation. Immunohistochemical analyses showed an abnormal density of GluR1 in the hip-pocampus of Arc/Arg3.1-deficient mice;however, an application of AMPA potentiator did not improve memory performance in the mutant mice. Memory impairment in Arc/Arg3.1-deficient mice is so ro-bust that the mice provide a useful tool for devel-oping treatments for memory impairment.

State of Health Estimation of Lithium-Ion Batteries Using Support Vector Regression and Long Short-Term Memory

Lithium-ion batteries are the most widely accepted type of battery in the electric vehicle industry because of some of their positive inherent characteristics. However, the safety problems associated with inaccurate estimation and prediction of the state of health of these batteries have attracted wide attention due to the adverse negative effect on vehicle safety. In this paper, both machine and deep learning models were used to estimate the state of health of lithium-ion batteries. The paper introduces the definition of battery health status and its importance in the electric vehicle industry. Based on the data preprocessing and visualization analysis, three features related to actual battery capacity degradation are extracted from the data. Two learning models, SVR and LSTM were employed for the state of health estimation and their respective results are compared in this paper. The mean square error and coefficient of determination were the two metrics for the performance evaluation of the models. The experimental results indicate that both models have high estimation results. However, the metrics indicated that the SVR was the overall best model.

Short Term Traffic Flow Prediction Using Hybrid Deep Learning

Traffic flow prediction in urban areas is essential in the IntelligentTransportation System (ITS). Short Term Traffic Flow (STTF) predictionimpacts traffic flow series, where an estimation of the number of vehicleswill appear during the next instance of time per hour. Precise STTF iscritical in Intelligent Transportation System. Various extinct systems aim forshort-term traffic forecasts, ensuring a good precision outcome which was asignificant task over the past few years. The main objective of this paper is topropose a new model to predict STTF for every hour of a day. In this paper,we have proposed a novel hybrid algorithm utilizing Principal ComponentAnalysis (PCA), Stacked Auto-Encoder (SAE), Long Short Term Memory(LSTM), and K-Nearest Neighbors (KNN) named PALKNN. Firstly, PCAremoves unwanted information from the dataset and selects essential features.Secondly, SAE is used to reduce the dimension of input data using onehotencoding so the model can be trained with better speed. Thirdly, LSTMtakes the input from SAE, where the data is sorted in ascending orderbased on the important features and generates the derived value. Finally,KNN Regressor takes information from LSTM to predict traffic flow. Theforecasting performance of the PALKNN model is investigated with OpenRoad Traffic Statistics dataset, Great Britain, UK. This paper enhanced thetraffic flow prediction for every hour of a day with a minimal error value.An extensive experimental analysis was performed on the benchmark dataset.The evaluated results indicate the significant improvement of the proposedPALKNN model over the recent approaches such as KNN, SARIMA, LogisticRegression, RNN, and LSTM in terms of root mean square error (RMSE)of 2.07%, mean square error (MSE) of 4.1%, and mean absolute error (MAE)of 2.04%.

Short-TermWind Power Prediction Based on Combinatorial Neural Networks

Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.

LSTM-GAN:融合GAN和Bi-LSTM 的无监督时间序列异常检测

多元时间序列数据的异常检测旨在发现对应时序特征中不符合一般规律的特异性模式,识别特定时间步长中的异常状态.针对多元时序数据时间依赖性建模难以及数据维度不断增加导致难以有效进行异常检测等问题,本文以自编码器为基础,融合生成对抗网络(Generative Adversarial Network,GAN)和双向长短期记忆神经网络(Bi-directional Long Short-Term Memory,Bi-LSTM),提出了一种无监督异常检测模型LSTM-GAN,该模型在每一轮训练中,以迭代的方式重构正常数据,通过GAN来放大异常,Bi-LSTM来捕获时间特性,训练完成后的模型用于时序数据的异常检测.本文在4个公开数据集上和几种先进同类方法进行了对比实验,实验结果表明,LSTM-GAN的检测性能提升了4.4%~16.6%,在IT数据集SMD中的模型检测F1分数达到0.9672,实现了高效的时序数据异常检测.

基于Bi-LSTM神经网络的室内可见光定位方法

双向长短时记忆(Bi-LSTM)神经网络由于超参数众多,难以获得最优系统模型。同时,考虑到灰狼优化(GWO)算法可能过早收敛的情况,提出了一种采用GWO结合粒子群(GWO-PSO)算法优化Bi-LSTM神经网络的单灯定位方法。通过优化网络中的学习率、隐藏神经元个数等超参数,提高系统的稳定性和定位精度。最后,采用加权K邻近(WKNN)算法对误差较大的点进行优化,以获得更精确的定位位置。仿真结果表明,在3 m×3.6 m×3 m的室内环境中,所提定位方法的平均定位误差为3.57 cm,其中90%的定位误差在6 cm内。

基于BERT-Bi-LSTM-CRF模型的机场类中文航行通告要素实体识别

航行通告是民用航空情报领域的重要情报资料,针对中文航行通告专业名词较多、格式不统一及语义复杂等问题,提出了一种基于BERT-Bi-LSTM-CRF的实体识别模型,对航行通告E项内容中事件要素实体进行抽取。首先通过BERT(bidirectional encoder representations from transforms)模型对处理后的向量进行预训练,捕捉丰富的语义特征,然后传送至双向长短期记忆网络(bidirectional long short-term memory,Bi-LSTM)模型对上下文特征进行提取,最后利用条件随机场(conditional random field,CRF)模型对最佳实体标签预测并输出。收集并整理机场类航行通告相关的原始语料,经过文本标注与数据预处理,形成了可用于实体识别实验的训练集、验证集和评价集数据。基于此数据与不同的实体识别模型进行对比实验,BERT-Bi-LSTM-CRF模型的准确率为89.68%、召回率为81.77%、F_(1)为85.54%,其中F 1相比现有模型得到有效提升,结果验证了该模型在机场类航行通告中要素实体识别的有效性。

基于Bi-LSTM神经网络的短文本敏感词识别方法

为了准确识别与处理敏感词,针对分词时延较高、识别精度较低的问题,提出基于双向长短期记忆(Bi-LSTM)神经网络的短文本敏感词识别方法。分析敏感词库,将敏感词库划分为两大类、三个等级,预处理短文本干扰信息(特殊字符、繁体字与拆分汉字),引入Bi-LSTM神经网络构建短文本分词模型,二次训练确定最佳参数,反复计算词语的敏感性数值,通过敏感性对比函数,提取短文本敏感词,并匹配敏感词库,确定敏感词的类别与等级,实现短文本敏感词识别。实验结果表明:在不同实验组别下,应用本文方法获得的短文本分词时延均低于给定最大限值,短文本敏感词识别精度高于84.42%,应用性能较佳。

基于注意力机制与XBOA-Bi-LSTM的离心式压缩机故障预警方法

由于离心式压缩机存在着运行工况复杂、维修成本昂贵和长输管道工作环境恶劣的问题,为此,提出了一种基于注意力机制(AM)和蝴蝶算法优化双向长短期记忆神经网络(XBOA-Bi-LSTM)的离心式压缩机故障预警方法。首先,针对传统蝴蝶算法的收敛速度慢、转换概率单一和容易陷入局部最优等问题,通过引入无限折叠迭代混叠映射以丰富蝴蝶算法的初始种群;同时,提出了一种基于种群离散度与迭代次数的自适应惯性转换概率,以提高蝴蝶算法的寻优能力;然后,采用了灰色关联度分析法对测点数据进行了特征提取,结合注意力机制对输入序列进行了灰色关联度系数赋权;最后,建立了双向长短期记忆神经网络故障预警模型,采用仿真实验完成了对离心式压缩机的故障预警;以某天然气长输管道机组的离心式压缩机作为仿真对象,对该离心式压缩机故障预警方法的可行性进行了验证。研究结果表明:采用基于注意力机制与XBOA-Bi-LSTM的离心式压缩机故障预警方法时,在离心式压缩机故障发生前2 h~3 h内就发出预警信号,实现了对于离心式压缩机进气过滤器压差异常与支撑轴承工作异常的故障预警目的。

结合改进残差网络和 Bi-LSTM 的短期电力负荷预测

为充分挖掘电力负荷历史数据的潜在特征,提高短期负荷预测模型的预测精度,提出了一种由改进残差网络(ResNetPlus)、注意力机制(Attention mechanism,AM)和双向长短期记忆网络(Bi-directional long short-term memory,Bi-LSTM)结合而成的残差AM-Bi-LSTM预测模型。该模型将历史负荷、温度和所预测日期的特征作为输入,在Bi-LSTM模型基础上,引入多层改进残差网络提取输入数据的隐藏特征,有效克服了网络隐藏层数加深导致的网络退化问题,使模型的反向传播能力大幅提升;加入注意力机制,分析网络中输入信息与当前负荷的相关性并突出重要信息的影响,从而提高模型的速度与准确率;使用Snapshot策略集成收敛于不同局部极小值的多个模型,以提升模型的准确率和鲁棒性。最后,使用美国ISO-NE数据集进行模拟预测,测试结果表明:所提模型的平均预测精度达到98.27%;在连续的12个月中采用该模型的平均预测精度相比于LSTM模型提高了2.87%;在不同季节下采用该模型的平均预测精度相比于AM-Bi-LSTM和ResNetPlus模型分别提高了1.05%和1.16%。说明所提模型相较于对比模型具有较高的准确率、鲁棒性以及泛化能力。

基于改进Bi-LSTM-CRF的农业问答系统研究

针对农业领域问答系统面临的实体识别困难的问题,提出一种基于改进Bi-LSTM-CRF的实体识别方法。首先通过BERT预训练模型的预处理,生成基于上下文信息的词向量,然后将训练出的词向量输入Bi-LSTM-CRF做进一步的训练处理,最后,利用Python的Django框架设计农业领域的实体识别、实体查询、农知问答等子系统。经过试验对比,所提出的改进的Bi-LSTM-CRF在农业信息领域具有更好的实体识别能力,在农业信息语料库上的精确率、召回率和F1值分别为93.23%、91.08%和92.16%。实现农业领域实体识别和农业信息问答的知识图谱网站演示,对农业信息化的发展具有重要意义。

基于Bi-LSTM网络的管道异常数据检测方法

在管道运行过程中,受技术、计量仪器和自然环境等影响,导致管道数据经常出现异常值,影响调度人员无法进行正确的决策,不利于管道监控系统的安全稳定运行。传统的时序数据异常检测方法的准确率和检测速度得不到保证。针对该问题,提出一种基于卷积神经网络(Convolutional Neural Network,CNN)和双向长短期记忆(Bi-directional Long-Short Term Memory,Bi-LSTM)网络的管道异常数据检测方法。首先,研究管道异常数据的表征及异常数据的产生原因,对管道数据进行野点剔除、均值填充和归一化处理,后通过CNN对处理后的管道数据进行特征提取;其次,利用Bi-LSTM网络充分挖掘管道数据间的规律,训练得到预测模型;再次,确定动态阈值,通过计算预测值与真实值误差并与阈值进行比较,检测异常数据;最后,在真实应用场景测试,通过设计一系列对比实验验证了该方法在处理速度和检测准确率等方面具有明显优势,且检测异常点的准确率高于同类算法。

基于Bi-LSTM和图注意力网络的多标签文本分类算法

针对当前大多数分类算法忽略标签之间相关性的问题,提出一种基于双向长短时记忆(Bi-LSTM)网络和图注意力网络(Graph Attention Network,GAT)的多标签文本分类算法。使用词嵌入工具对文本序列和标签中的词向量进行预处理后的文本序列和标签分别输入到Bi-LSTM网络和GAT网络中;提取文本序列的上下文信息和全局特征,以及GAT网络捕获标签之间的相关性;将特征向量和标签相关性进行组合对标签文本分类任务进行预测。实验结果表明,所提算法通过有效关注标签之间的相关性使得文本分类任务的精度得以明显提高,在多个评估指标的测试结果优于其他对比方法。

基于Bi-LSTM循环神经网络的风储系统控制策略

“双碳”背景下风电的渗透率不断提高,将对电力系统的形态和运行机制产生深刻影响。本文提出了一种基于双向长短期记忆Bi-LSTM(bidirectional long short-term memory)循环神经网络的风储系统控制策略。采用双向长短时循环神经网络提取控制结果与风电场实际出力以及储能状态间的时序信息,通过构建基于双向长短时记忆循环神经网络的控制模型,使得风电场在多种运行工况下能够快速、准确地得到储能系统调节结果。基于实际风电场数据仿真结果表明,本文所提控制策略能够保证在一定经济效益的前提下,将风储系统控制误差保持在0.50%~1.37%。