Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Short-TermWind Power Prediction Based on Combinatorial Neural Networks

Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.

Conditional Random Field Tracking Model Based on a Visual Long Short Term Memory Network

In dense pedestrian tracking,frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories.In this study,a conditional random field tracking model is established by using a visual long short term memory network in the three-dimensional(3D)space and the motion estimations jointly performed on object trajectory segments.Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation.To address the uncertainty of the length and interval of trajectory segments,a multimode long short term memory network is proposed for the object motion estimation.The tracking performance is evaluated using the PETS2009 dataset.The experimental results show that the proposed method achieves better performance than the tracking methods based on the independent motion estimation.

Binaural Speech Separation Algorithm Based on Long and Short Time Memory Networks

Speaker separation in complex acoustic environment is one of challenging tasks in speech separation.In practice,speakers are very often unmoving or moving slowly in normal communication.In this case,the spatial features among the consecutive speech frames become highly correlated such that it is helpful for speaker separation by providing additional spatial information.To fully exploit this information,we design a separation system on Recurrent Neural Network(RNN)with long short-term memory(LSTM)which effectively learns the temporal dynamics of spatial features.In detail,a LSTM-based speaker separation algorithm is proposed to extract the spatial features in each time-frequency(TF)unit and form the corresponding feature vector.Then,we treat speaker separation as a supervised learning problem,where a modified ideal ratio mask(IRM)is defined as the training function during LSTM learning.Simulations show that the proposed system achieves attractive separation performance in noisy and reverberant environments.Specifically,during the untrained acoustic test with limited priors,e.g.,unmatched signal to noise ratio(SNR)and reverberation,the proposed LSTM based algorithm can still outperforms the existing DNN based method in the measures of PESQ and STOI.It indicates our method is more robust in untrained conditions.

Short-Term Relay Quality Prediction Algorithm Based on Long and Short-Term Memory

The fraction defective of semi-finished products is predicted to optimize the process of relay production lines, by which production quality and productivity are increased, and the costs are decreased. The process parameters of relay production lines are studied based on the long-and-short-term memory network. Then, the Keras deep learning framework is utilized to build up a short-term relay quality prediction algorithm for the semi-finished product. A simulation model is used to study prediction algorithm. The simulation results show that the average prediction absolute error of the fraction is less than 5%. This work displays great application potential in the relay production lines.

State of Health Estimation of Lithium-Ion Batteries Using Support Vector Regression and Long Short-Term Memory

Lithium-ion batteries are the most widely accepted type of battery in the electric vehicle industry because of some of their positive inherent characteristics. However, the safety problems associated with inaccurate estimation and prediction of the state of health of these batteries have attracted wide attention due to the adverse negative effect on vehicle safety. In this paper, both machine and deep learning models were used to estimate the state of health of lithium-ion batteries. The paper introduces the definition of battery health status and its importance in the electric vehicle industry. Based on the data preprocessing and visualization analysis, three features related to actual battery capacity degradation are extracted from the data. Two learning models, SVR and LSTM were employed for the state of health estimation and their respective results are compared in this paper. The mean square error and coefficient of determination were the two metrics for the performance evaluation of the models. The experimental results indicate that both models have high estimation results. However, the metrics indicated that the SVR was the overall best model.

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.

Wind Speed Short-Term Prediction Based on Empirical Wavelet Transform, Recurrent Neural Network and Error Correction

Predicting wind speed accurately is essential to ensure the stability of the wind power system and improve the utilization rate of wind energy.However,owing to the stochastic and intermittent of wind speed,predicting wind speed accurately is difficult.A new hybrid deep learning model based on empirical wavelet transform,recurrent neural network and error correction for short-term wind speed prediction is proposed in this paper.The empirical wavelet transformation is applied to decompose the original wind speed series.The long short term memory network and the Elman neural network are adopted to predict low-frequency and high-frequency wind speed sub-layers respectively to balance the calculation efficiency and prediction accuracy.The error correction strategy based on deep long short term memory network is developed to modify the prediction errors.Four actual wind speed series are utilized to verify the effectiveness of the proposed model.The empirical results indicate that the method proposed in this paper has satisfactory performance in wind speed prediction.

A Novel MegaBAT Optimized Intelligent Intrusion Detection System in Wireless Sensor Networks

Wireless Sensor Network(WSN),whichfinds as one of the major components of modern electronic and wireless systems.A WSN consists of numerous sensor nodes for the discovery of sensor networks to leverage features like data sensing,data processing,and communication.In thefield of medical health care,these network plays a very vital role in transmitting highly sensitive data from different geographic regions and collecting this information by the respective network.But the fear of different attacks on health care data typically increases day by day.In a very short period,these attacks may cause adversarial effects to the WSN nodes.Furthermore,the existing Intrusion Detection System(IDS)suffers from the drawbacks of limited resources,low detection rate,and high computational overhead and also increases the false alarm rates in detecting the different attacks.Given the above-mentioned problems,this paper proposes the novel MegaBAT optimized Long Short Term Memory(MBOLT)-IDS for WSNs for the effective detection of different attacks.In the proposed framework,hyperpara-meters of deep Long Short-Term Memory(LSTM)were optimized by the meta-heuristic megabat algorithm to obtain a low computational overhead and high performance.The experimentations have been carried out using(Wireless Sensor NetworkDetection System)WSN-DS datasets and performance metrics such as accuracy,recall,precision,specificity,and F1-score are calculated and compared with the other existing intelligent IDS.The proposed framework provides outstanding results in detecting the black hole,gray hole,scheduling,flooding attacks and significantly reduces the time complexity,which makes this system suitable for resource-constraint WSNs.

基于字词向量融合的民航智慧监管短文本分类

为解决民航监管事项所产生的检查记录仅依靠人工进行分类分析导致效率低的问题,提出一种基于数据增强与字词向量融合的双通道特征提取的短文本分类模型,探讨民航监管事项的分类,包括与人、设备设施环境、制度程序和机构职责等相关问题。为解决类别不平衡问题,采用数据增强算法在原始文本上进行变换,生成新的样本,使各个类别的样本数量更加均衡。将字向量和词向量按字融合拼接,得到具有词特征信息的字向量。将字词融合的向量分别送入到文本卷积神经网络(TextCNN)和双向长短期记忆(BiLSTM)模型中进行不同维度的特征提取,从局部的角度和全局的角度分别提取特征,并在民航监管事项检查记录数据集上进行试验。结果表明:该模型准确率为0.9837,F 1值为0.9836。与一些字嵌入模型和词嵌入模型相对比,准确率提升0.4%。和一些常用的单通道模型相比,准确率提升3%,验证了双通道模型提取的特征具有全面性和有效性。

基于改进麻雀搜索算法优化LSTM的滚动轴承故障诊断

为了对滚动轴承的工作状态及故障类别进行准确的诊断,本文采用长短时记忆(LSTM)神经网络作为分类器对滚动轴承数据集进行分类诊断。首先,从滚动轴承原始运行振动信号中提取时域和频域特征参数,组成具有高维特征参数的数据集;使用核主成分分析(KPCA)方法对高维特征集进行降维处理,选取重要性程度高的特征构成输入特征向量。然后,针对LSTM神经网络在滚动轴承故障诊断中存在的超参数难以确定的问题,提出一种基于自适应t分布策略的麻雀搜索算法优化LSTM神经网络的故障诊断方法(tSSA–LSTM)。最后,使用凯斯西储大学滚动轴承数据中心的数据进行故障诊断精度测试、泛化性能测试及噪声环境下故障诊断性能测试等多个仿真实验,并将本文提出的诊断模型与麻雀搜索算法优化长短时记忆神经网络(SSA–LSTM)、遗传算法优化长短时记忆神经网络(GA–LSTM)、粒子群算法优化长短时记忆神经网络(PSO–LSTM)及传统LSTM诊断模型进行对比。结果表明:tSSA可以更有效地对LSTM的隐含层神经元数量、周期次数、学习率等超参数进行合理优化;所提方法的平均诊断准确率达到98.86%,交叉验证平均诊断结果为98.57%;所提方法在噪声干扰下的故障诊断准确率也优于对比方法。因此,本文提出的tSSA–LSTM模型不仅可以更精准地诊断滚动轴承故障状态,而且具有更强的泛化能力及抗干扰能力,有效地提高了滚动轴承故障诊断的性能。

结合太阳辐射量计算与CNN-LSTM组合的光伏功率预测方法研究

为了提高模型预测性能,提出一种综合太阳辐射模型及深度学习的光伏功率预测模型。首先,利用太阳辐射机理建立太阳辐射模型(SRM),估算出水平面上总辐射值,再由斜面辐照度转换方法计算出光伏组件所接收的斜面辐射值。其次,通过皮尔逊相关分析法筛选出对光伏功率影响较大的主要因素,将斜面辐射计算值及主要影响因素作为输入,采用卷积神经网络(CNN)和长短期记忆网络(LSTM)建立光伏功率SRM-CNN-LSTM预测模型。分别利用春夏秋冬四季典型日的数据开展对比实验,结果表明:与几种其他方法相比,该文方法具有更好的预测效果。

基于BERT-BiLSTM-CRF模型的畜禽疫病文本分词研究

针对畜禽疫病文本语料匮乏、文本内包含大量疫病名称及短语等未登录词问题,提出了一种结合词典匹配的BERT-BiLSTM-CRF畜禽疫病文本分词模型。以羊疫病为研究对象,构建了常见疫病文本数据集,将其与通用语料PKU结合,利用BERT(Bidirectional encoder representation from transformers)预训练语言模型进行文本向量化表示;通过双向长短时记忆网络(Bidirectional long short-term memory network,BiLSTM)获取上下文语义特征;由条件随机场(Conditional random field,CRF)输出全局最优标签序列。基于此,在CRF层后加入畜禽疫病领域词典进行分词匹配修正,减少在分词过程中出现的疫病名称及短语等造成的歧义切分,进一步提高了分词准确率。实验结果表明,结合词典匹配的BERT-BiLSTM-CRF模型在羊常见疫病文本数据集上的F1值为96.38%,与jieba分词器、BiLSTM-Softmax模型、BiLSTM-CRF模型、未结合词典匹配的本文模型相比,分别提升11.01、10.62、8.3、0.72个百分点,验证了方法的有效性。与单一语料相比,通用语料PKU和羊常见疫病文本数据集结合的混合语料,能够同时对畜禽疫病专业术语及疫病文本中常用词进行准确切分,在通用语料及疫病文本数据集上F1值都达到95%以上,具有较好的模型泛化能力。该方法可用于畜禽疫病文本分词。

基于CEEMD-SE的CNN&LSTM-GRU短期风电功率预测

为进一步提升短期风电功率的预测精度,提出了一种基于互补集合经验模态分解-样本熵(complementary ensemble empirical mode decomposition-sample entropy,CEEMD-SE)的卷积神经网络(convolutional neural network,CNN)和长短期记忆-门控循环单元(longshorttermmemory-gatedrecurrentunit,LSTM-GRU)的短期风电功率预测模型。首先,利用互补集合经验模态分解将原始风电功率序列分解为若干本征模态函数(intrinsic mode function,IMF)分量和一个残差(residual,RES)分量,利用样本熵算法将相近的分量进行重构;其次,搭建卷积神经网络和长短期记忆网络的并行网络结构,提取数据的局部特征和时序特征,并将特征融合后输入门控循环单元网络中进行学习预测;最后,通过算例进行验证,结果表明采用该模型后预测精度得到了有效提升,其均方根误差降低了15.06%、平均绝对误差降低了15.22%、决定系数提高了1.91%。

面向喷染车间的挥发性有机物质量浓度预测方法及应用研究

以喷染车间挥发性有机物为研究对象,对喷染车间挥发性有机物(Volatile Organic Compounds, VOCs)质量浓度预测方法进行研究。首先,使用随机森林(Random Forest, RF)算法对影响喷染车间挥发性有机物质量浓度的特征变量进行权重分析。同时,构建基于长短期记忆神经网络(Long-Term and Short-Term Memory Neural Network, LSTM)的挥发性有机物质量浓度预测模型,并在此基础上引入麻雀搜索算法(Sparrow Search Algorithm, SSA)进行参数优化选择。最后,以浙江省杭州市某汽车喷染车间7月29日—10月28日的数据为样本,将温度、相对湿度、室内大气压、室外大气压作为模型输入变量,并与LSTM模型、随机森林-长短期记忆神经网络(Random Forest-Long Short-Term Memory neural network, RF-LSTM)模型、随机森林-反向传播神经网络(Random Forest-BP neural network, RF-BP)模型进行对比试验。结果显示,基于随机森林-麻雀搜索算法-长短期记忆神经网络(Random Forest-Sparrow Search Algorithm-Long Short-Term Memory neural network, RF-SSA-LSTM)模型的预测效果最佳,平均绝对误差、均方根误差和决定系数分别为2.812 2、3.457 4、0.988。同时,为验证RF-SSA-LSTM模型性能,通过不同时间步长实现对喷染车间VOCs质量浓度预测,结果显示预测误差较小,在可接受范围内。RF-SSA-LSTM预测模型提高了挥发性有机物质量浓度的预测精度,为减少挥发性有机物排放提供科学依据。

Dynamic Resource Allocation in LTE Radio Access Network Using Machine Learning Techniques

Current LTE networks are experiencing significant growth in the number of users worldwide. The use of data services for online browsing, e-learning, online meetings and initiatives such as smart cities means that subscribers stay connected for long periods, thereby saturating a number of signalling resources. One of such resources is the Radio Resource Connected (RRC) parameter, which is allocated to eNodeBs with the aim of limiting the number of connected simultaneously in the network. The fixed allocation of this parameter means that, depending on the traffic at different times of the day and the geographical position, some eNodeBs are saturated with RRC resources (overused) while others have unused RRC resources. However, as these resources are limited, there is the problem of their underutilization (non-optimal utilization of resources at the eNodeB level) due to static allocation (manual configuration of resources). The objective of this paper is to design an efficient machine learning model that will take as input some key performance indices (KPIs) like traffic data, RRC, simultaneous users, etc., for each eNodeB per hour and per day and accurately predict the number of needed RRC resources that will be dynamically allocated to them in order to avoid traffic and financial losses to the mobile network operator. To reach this target, three machine learning algorithms have been studied namely: linear regression, convolutional neural networks and long short-term memory (LSTM) to train three models and evaluate them. The model trained with the LSTM algorithm gave the best performance with 97% accuracy and was therefore implemented in the proposed solution for RRC resource allocation. An interconnection architecture is also proposed to embed the proposed solution into the Operation and maintenance network of a mobile network operator. In this way, the proposed solution can contribute to developing and expanding the concept of Self Organizing Network (SON) used in 4G and 5G networks.

基于LSTM的机场飞行区活动目标潜在冲突预测

针对机场飞行区冲突不断的问题,提出一种基于长短期记忆(LSTM)网络预测机场飞行区活动目标潜在冲突的方法。根据复杂网络理论,以航空器和车辆2类活动目标为研究对象,建立飞行区活动目标网络,设置网络动态演化模型,输入运行数据计算多个网络特征指标,对指标时间序列进行主成分分析,拟合成潜在冲突指数;利用Keras框架搭建LSTM网络模型,将指标时间序列输入LSTM网络进行训练和预测,并与其他预测方法对比;用西安咸阳机场实际运行数据进行实验,将预测值与真实值进行对比,各项指标预测均方误差分别为1.608%、13.126%、0.072%、0.004%、0.014%。结果表明:通过建立飞行区活动目标网络模型,可以用网络特征指标从不同角度刻画潜在冲突;LSTM网络可以有效预测飞行区活动目标网络的潜在冲突,提醒相关人员预防冲突发生,降低冲突概率。

基于注意力机制的CNN-BiLSTM的IGBT剩余使用寿命预测

针对绝缘栅双极型晶体管(IGBT)可靠性问题,提出了一种融合卷积神经网络(CNN)、双向长短期记忆(BiLSTM)网络和注意力机制的剩余使用寿命(RUL)预测模型,可用于IGBT的寿命预测。模型中使用CNN提取特征参数,BiLSTM提取时序信息,注意力机制加权处理特征参数。使用IGBT加速老化数据集对提出的模型进行验证。结果表明,对比自回归差分移动平均(ARIMA)、长短期记忆(LSTM)、多层LSTM(Multi-LSTM)、 BiLSTM预测模型,在均方根误差和决定系数等评价指标方面该模型的性能最优。验证了提出的寿命预测模型对IGBT失效预测是有效的。

基于VMD-SE的电力负荷分量的多特征短期预测

为提高电力负荷的预测精度,提出一种基于VMD-SE的电力负荷分量的多特征短期预测方法。首先采用变分模态分解(VMD)将原始负荷分解为一系列模态分量与残差,VMD的分解层数由样本熵值(sample entropy,SE)确定;然后对比原始负荷与模态分量的SE值,重构为平稳分量和波动分量,来降低运算规模;同时利用皮尔逊相关系数来筛选特征变量,删除特征冗余,建立灰狼算法优化后的支持向量回归模型(GWO-SVR)和长短期记忆神经网络(LSTM)分别对平稳分量和波动分量预测;最后以某地区2018—2020年用电负荷为例进行实验。实验证明:此模型精准度高达94.7%,平均绝对百分误差降低到2.98%,具有更好的精准性和适用性。

基于集群辨识和卷积神经网络-双向长短期记忆-时序模式注意力机制的区域级短期负荷预测

为了解决区域级短期电力负荷预测时输入特征过多和负荷时序性较强的问题,提出一种基于集群辨识和卷积神经网络(convolutional neural networks,CNN)-双向长短期记忆网络(bi-directional long short-term memory,BiLSTM)-时序模式注意力机制(temporal pattern attention,TPA)的预测方法。首先,将用电模式和天气作为影响因素,基于二阶聚类算法对区域内的负荷节点进行集群辨识,再从每个集群中挑选代表特征作为深度学习模型的输入,这样既能减少输入特征维度,降低计算复杂度,又能综合考虑预测区域的整体特征,提升预测精度。然后,针对区域电力负荷时序性的特点,用CNN-BiLSTM-TPA模型完成训练和预测,该模型能提取输入数据的双向信息生成隐状态矩阵,并对隐状态矩阵的重要特征加权,从多时间步上捕获双向时序信息用于预测。最后,在美国加利福尼亚州实例上分析验证了所提方法的有效性。