Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Short-TermWind Power Prediction Based on Combinatorial Neural Networks

Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.

Synthetic well logs generation via Recurrent Neural Networks

To supplement missing logging information without increasing economic cost, a machine learning method to generate synthetic well logs from the existing log data was presented, and the experimental verification and application effect analysis were carried out. Since the traditional Fully Connected Neural Network(FCNN) is incapable of preserving spatial dependency, the Long Short-Term Memory(LSTM) network, which is a kind of Recurrent Neural Network(RNN), was utilized to establish a method for log reconstruction. By this method, synthetic logs can be generated from series of input log data with consideration of variation trend and context information with depth. Besides, a cascaded LSTM was proposed by combining the standard LSTM with a cascade system. Testing through real well log data shows that: the results from the LSTM are of higher accuracy than the traditional FCNN; the cascaded LSTM is more suitable for the problem with multiple series data; the machine learning method proposed provides an accurate and cost effective way for synthetic well log generation.

Bitcoin Candlestick Prediction with Deep Neural Networks Based on Real Time Data

Currently,Bitcoin is the world’s most popular cryptocurrency.The price of Bitcoin is extremely volatile,which can be described as high-benefit and high-risk.To minimize the risk involved,a means of more accurately predicting the Bitcoin price is required.Most of the existing studies of Bitcoin prediction are based on historical(i.e.,benchmark)data,without considering the real-time(i.e.,live)data.To mitigate the issue of price volatility and achieve more precise outcomes,this study suggests using historical and real-time data to predict the Bitcoin candlestick—or open,high,low,and close(OHLC)—prices.Seeking a better prediction model,the present study proposes time series-based deep learning models.In particular,two deep learning algorithms were applied,namely,long short-term memory(LSTM)and gated recurrent unit(GRU).Using real-time data,the Bitcoin candlesticks were predicted for three intervals:the next 4 h,the next 12 h,and the next 24 h.The results showed that the best-performing model was the LSTM-based model with the 4-h interval.In particular,this model achieved a stellar performance with a mean absolute percentage error(MAPE)of 0.63,a root mean square error(RMSE)of 0.0009,a mean square error(MSE)of 9e-07,a mean absolute error(MAE)of 0.0005,and an R-squared coefficient(R2)of 0.994.With these results,the proposed prediction model has demonstrated its efficiency over the models proposed in previous studies.The findings of this study have considerable implications in the business field,as the proposed model can assist investors and traders in precisely identifying Bitcoin sales and buying opportunities.

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.

Wind Speed Short-Term Prediction Based on Empirical Wavelet Transform, Recurrent Neural Network and Error Correction

Predicting wind speed accurately is essential to ensure the stability of the wind power system and improve the utilization rate of wind energy.However,owing to the stochastic and intermittent of wind speed,predicting wind speed accurately is difficult.A new hybrid deep learning model based on empirical wavelet transform,recurrent neural network and error correction for short-term wind speed prediction is proposed in this paper.The empirical wavelet transformation is applied to decompose the original wind speed series.The long short term memory network and the Elman neural network are adopted to predict low-frequency and high-frequency wind speed sub-layers respectively to balance the calculation efficiency and prediction accuracy.The error correction strategy based on deep long short term memory network is developed to modify the prediction errors.Four actual wind speed series are utilized to verify the effectiveness of the proposed model.The empirical results indicate that the method proposed in this paper has satisfactory performance in wind speed prediction.

基于BP-DCKF-LSTM的锂离子电池SOC估计

电池荷电状态(SOC)的准确估计是电池管理系统(BMS)的核心功能之一。为了提高锂电池SOC估算精度,提出了一种将反向传播神经网络(BP)、双容积卡尔曼滤波(DCKF)和长短期记忆神经网络(LSTM)相结合的SOC估计方法。针对多温度条件下传统多项式拟合法在拟合开路电压(OCV)与SOC时效果较差的问题,提出了一种基于BP神经网络的拟合方法,通过验证表明该方法能有效提高拟合精度。针对单独使用模型法或数据驱动法估计SOC各自存在的优缺点,提出了一种将DCKF与LSTM相结合的估计方法,在提高估计精度的同时,可以减少参数调节时间和训练成本。实验验证表明,BP-DCKF-LSTM算法的均方根误差(RMSE)和平均绝对误差(MAE)分别小于0.5%和0.4%,具有较高的SOC估算精度和鲁棒性。

基于ASFF-AAKR和CNN-BILSTM滚动轴承寿命预测

针对滚动轴承寿命预测精度低,构建健康指标困难的问题。提出了一种基于自适应特征融合(adaptively spatial feature fusion,ASFF)和自联想核回归模型(auto associative kernel regression,AAKR)与卷积神经网络(convolutional neural networks,CNN)和双向长短期记忆网络(bi-directional long-short term memory,BILSTM)的轴承剩余寿命预测模型。首先,在时域、频域和时频域提取多维特征,利用单调性和趋势性筛选敏感特征;其次利用ASFF-AAKR对敏感特征进行特征融合构建健康指标;最后,将健康指标输入到CNN和BILSTM中,实现对滚动轴承的寿命预测。结果表明:所构建的寿命预测模型优于其他模型,该方法具有更低的误差、寿命预测精度更高。

一种基于LSTM-RNN的脉冲大倍率工况下锂离子电池仿真建模方法

锂离子电池是电力系统中不可或缺的重要储能元件,脉冲大倍率工况下运行的锂离子电池具有单次放电时间短、放电循环多、状态变化频繁、非线性极化现象明显等特点。该文以脉冲大倍率工况下锂离子电池模型为研究对象,针对电化学模型和等效电路模型对模型依赖度高、模型参数难以获取以及脉冲大倍率工况下非线性极化现象导致拟合精度不足等问题,提出基于长短期记忆循环神经网络(long short term memory recurrent neural network,LSTM-RNN)以实现准确的锂离子电池建模。该方法利用LSTM-RNN的动态逼近和长时记忆能力,以获取脉冲大倍率工况下锂离子电池性能参数和电池端电压、荷电状态、电流、温度之间的非线性关系。在6种脉冲大倍率放电工况下对磷酸铁锂电池进行建模,实验结果表明,所提出的基于长短期记忆循环神经网络的锂离子电池模型均能够准确表征磷酸铁锂电池工作特性。

融合BiLSTM与CNN的推特黑灰产分类模型

当前推特等国外社交平台,已成为从事网络黑灰产犯罪不可或缺的工具,对推特上黑灰产账号进行发现、检测和分类对于打击网络犯罪、维护社会稳定具有重大意义。现有的推文分类模型双向长短时记忆网络(bi-directional long short-term memory,BiLSTM)可以学习推文的上下文信息,却无法学习局部关键信息,卷积神经网络(convolution neural network,CNN)模型可以学习推文的局部关键信息,却无法学习推文的上下文信息。结合BiLSTM与CNN两种模型的优势,提出了BiLSTM-CNN推文分类模型,该模型将推文进行向量化后,输入BiLSTM模型学习推文的上下文信息,再在BiLSTM模型后引入CNN层,进行局部特征的提取,最后使用全连接层将经过池化的特征连接在一起,并应用softmax函数进行四分类。模型在自主构建的中文推特黑灰产推文数据集上进行实验,并使用TextCNN、TextRNN、TextRCNN三种分类模型作为对比实验,实验结果显示,所提的BiLSTM-CNN推文分类模型在对四类推文进行分类的宏准确率为98.32%,明显高于TextCNN、TextRNN和TextRCNN三种模型的准确率。

基于BLSTM-RNN的船舶轨迹修复方法

针对内河干线船舶AIS轨迹数缺失问题,提出一种基于双向长短时记忆网络(BLSTM-RNN)模型的船舶轨迹数据修复方法。通过利用船舶轨迹上下文信息及其他回传特征作为模型输入,构建两层的双向循环神经网络(RNN)模型。在模型输入上,采用相关性分析及序列自相关系数,确定船舶轨迹点相关变量及轨迹序列自相关滞后值;在模型结构上,以ACC率为指标对模型超参数值进行合理设置,以长江干线航道武汉段及重庆段船舶轨迹数据为样本,对模型进行实证验证。实验结果表明:与线性及其他机器学习方法相比BLSTM-RNN方法在精度上有一定提升;在武汉段顺直河段实验中,将修复误差控制在15 m量级内,远低于其他非线性方法的50 m量级;在重庆复杂河段内,可将修复误差控制在10 m量级;模型解决了传统方法在长距离丢失点上精度缺失的问题,在20个连续点丢失的情况上,将修复误差降低至50m量级。

基于EEMD与CNN-BiLSTM的噪声环境下滚动轴承故障诊断方法

针对滚动轴承在噪声环境中发生故障时,传统深度神经网络容易出现特征提取不充分,过拟合,泛化能力不足的问题,提出一种集成经验模态分解(EEMD)与卷积神经网络-双向长短时记忆网络(CNN-BiLSTM)的故障诊断方法。在信号预处理阶段使用EEMD将噪声环境下的振动信号分解为一系列固有模态函数,降低噪声的影响;在CNN部分的第1层使用大卷积核与多分支结构获得不同的感受野,在每一个分支中随机丢弃一些数据增强模型的抗干扰能力,从而提取到更具泛化能力的多样化特征信息,后续部分使用残差结构,以免网络较深时发生梯度消失的现象,解决深层次网络退化问题;在BiLSTM部分使用2个并行的分支结构,用于增强模型对时序信息的利用,从而提高模型在不同工况和噪声环境下的准确率。使用凯斯西储大学轴承数据集和西安交通大学轴承数据集对所提方法进行验证,并与其他深度学习方法和传统机器学习方法进行对比,结果表明本文方法在多种工况和噪声环境下均取得了优异的故障诊断性能。

基于SIP的FPGA驱动电压补偿测试研究

在基于SIP的现场可编程门阵列(FPGA)性能参数验证测试时,驱动电压测试会受到多种因素的影响,如PCB线阻、插座信号损耗以及测试温度等,这些因素导致ATE测试的实测值与真实值之间存在偏差。为了提高驱动电压的测试精度,提出一种基于卷积神经网络(CNN)与长短时记忆(LSTM)网络的误差补偿方法。将PCB线长、测试温度等参数作为特征输入到CNN-LSTM模型中,模型经过训练迭代后能够预测出驱动电压的误差值;再将预测的误差值应用于ATE测试机中,对实测值进行补偿和修正,从而使得测试结果更加接近真实值。实验结果表明,所提方法能够有效地减小测试误差,提高FPGA驱动电压测试的准确性。

基于长短期记忆神经网络的多级涡轮过渡态叶尖间隙预测

为了解决多级涡轮模型在高维度变量的复杂空间耦合效应下向高效、高精度过渡态叶尖间隙预测提出的挑战,本文搭建了基于贝叶斯优化和多任务学习算法的长短期记忆神经网络(BO-MTLLSTM)多级涡轮过渡态叶尖间隙智能预测模型,以实现过渡态叶尖间隙高效、高精度预测。在BOMTL-LSTM模型中,通过高效的长短期记忆神经网络(Long Short-Term Memory,LSTM)模型对基于有限元分析方法得到的高精度过渡态叶尖间隙时序信息进行学习,并在LSTM模型的基础上,引入多任务学习(Multi-Task Learning,MTL)用于多个叶尖间隙预测任务之间的信息共享,以缓解高维度变量复杂空间耦合作用的影响。同时,结合贝叶斯优化(Bayesian Optimization,BO)对神经网络模型超参数进行全局自动优化,提升预测精度与训练效率。结果表明,相比于传统计算模型,BO-MTL-LSTM模型在同等预测精度下,能够在秒量级时间内完成一个完整发动机历程的多级涡轮过渡态叶尖间隙的预测。此外,相比常规的BO-LSTM模型,BO-MTL-LSTM模型的均方根误差和平均绝对误差分别降低了84.39%和89.21%,模型训练时间缩短了30%,该模型可以实现多级叶尖间隙的高效、精准预测。

基于知识蒸馏压缩混合模型的航空发动机剩余寿命预测研究

航空发动机剩余使用寿命(RUL)预测对于提高飞机运行系统的安全性和降低维护成本至关重要。针对实际应用中边缘设备计算能力和内存有限的情况,保证模型RUL预测准确性的同时减少模型规模是亟待解决的问题。因此,提出了使用两次知识蒸馏压缩复杂的混合模型来预测航空发动机的RUL。首先,知识蒸馏用于异构网络之间进行知识转移,教师模型包含卷积神经网络(CNN)和双向长短期记忆网络(Bi-LSTM)等复杂网络,其中,CNN能够提取数据空间特征,而Bi-LSTM可以学习数据的双向长时间依赖性。学生模型则使用规模相对简单的多尺度卷积神经网络(MS-CNN),利用不同的卷积尺度各自提取数据信息,从而保证特征学习的完整性。其次,知识蒸馏用于同构网络之间进行知识迁移,学生网络与教师网络均使用多尺度卷积神经网络。最后,经过两次知识蒸馏获得的学生网络在C-MAPSS数据集上进行了RUL预测实验,结果显示蒸馏学生模型在4个数据集上预测准确性均有一定提升。蒸馏学生模型相比原始MS-CNN在FD001数据集获得的误差值降低了5.9%,同时也比Bi-LSTM误差值低6.7%,从而证明了所提方法在航空发动机RUL预测领域具有一定的竞争性。

GWO优化CNN-BiLSTM-Attenion的轴承剩余寿命预测方法

滚动轴承作为机械设备的重要部件,对其进行剩余使用寿命预测在企业的生产过程中变得越来越重要。目前,虽然主流的卷积神经网络(convolutional neural network, CNN)可以自动地从轴承的振动信号中提取特征,却不能给特征分配不同的权重来提高模型对重要特征的关注程度,对于长时间序列容易丢失重要信息。另外,神经网络中隐藏层神经元个数、学习率以及正则化参数等超参数还需要依靠人工经验设置。为了解决上述问题,提出基于灰狼优化(grey wolf optimizer, GWO)算法、优化集合CNN、双向长短期记忆(bidirectional long short term memory, BiLSTM)网络和注意力机制(Attention)轴承剩余使用寿命预测方法。首先,从原始振动信号中提取时域、频域以及时频域特征指标构建可选特征集;然后,通过构建考虑特征相关性、鲁棒性和单调性的综合评价指标筛选出高于设定阈值的轴承退化敏感特征集,作为预测模型的输入;最后,将预测值和真实值的均方误差作为GWO算法的适应度函数,优化预测模型获得最优隐藏层神经元个数、学习率和正则化参数,利用优化后模型进行剩余使用寿命预测,并在公开数据集上进行验证。结果表明,所提方法可在非经验指导下获得最优的超参数组合,优化后的预测模型与未进行优化模型相比,平均绝对误差与均方根误差分别降低了28.8%和24.3%。

动态图卷积联合记忆网络情绪脑电识别方法

针对无法有效利用脑电通道拓扑结构学习更有鉴别性的脑电特征问题,本文基于长短期记忆网络和图卷积神经网络,提出动态图卷积联合记忆网络(Dynamic Graph Convolutional Joint Long Short Term Memory Network,DGCJMN)方法。首先将脑电通道作为图的节点,微分熵作为节点特征,利用动态参数学习最优的脑电通道拓扑结构,构建特征图;之后,由图卷积神经网络提取图域特征,并结合长短期记忆网络和池化进一步提取特征;最后将图卷积网络、长短期记忆网络和池化提取的特征融合后进行情绪分类。所提方法在SEED数据集上针对积极、中性和消极3种情绪取得的平均准确率为95.93%,精确率、召回率和F1值分别为96.11%、95.93%和0.96,Kappa系数为0.939。混淆矩阵表明,模型对于3种情绪都达到了较好的分类效果。

基于麻雀搜索算法和长短期记忆神经网络的轨道交通站点客流预测

准确的短时客流预测可以为城市轨道交通的良好运营提供保障,但轨道交通的短时客流具有非线性和高随机性等特点,为了提高对短时客流的预测精度,提出将ISSA算法和LSTM模型进行组合,构建城市轨道交通短时客流预测模型.针对SSA算法收敛速度慢,容易陷入局部最优解的问题,引入黄金莱维飞行策略,通过动态调整探索者移动步长的方法,使得它在未知范围内搜索时,能够覆盖更大的范围,提高SSA算法全局搜索的能力.通过使用ISSA算法对LSTM模型的隐含层、学习率和迭代次数的神经元个数进行优化,构建ISSA-LSTM组合预测模型,用于城市轨道交通短时客流的预测.将该模型与BP、LSTM和SSA-LSTM等3种短时客流预测模型进行对比,结果表明:在针对工作日和非工作日客流的预测中,ISSA-LSTM模型预测误差最小,具有较好的预测效果.

基于循环神经网络的多模态数据层次化缓存系统设计

为提升对多模态数据的管理效果,提高数据访问速度并减轻数据库负载,设计一种基于循环神经网络的多模态数据层次化缓存系统。在DRAM/NVM混合内存模块中,利用DRAM完成主存NVM的缓存。当DRAM存在缓存缺失时,利用访问监控模块内置高速采集卡来采集NVM上频繁访问4 KB数据块的历史访问记录,再将历史访问记录编码为访问向量后构建训练集,作为长短期记忆(LSTM)网络的输入,用于预测访问频率。在缓存过滤模块中,将访问频率预测结果高于设定阈值部分的4 KB多模态数据读取到DRAM中进行缓存。实验结果显示:所设计系统可最大程度地降低系统带宽占用情况,TLB缺失率低,缓存执行效率较高,面对大页面具备显著缓存优势。

基于自主探索的移动机器人路径规划研究

移动机器人在路径规划过程中,当面对未知且动态变化的环境时,会存在与障碍物碰撞率高、易陷入局部最优等问题。针对这些问题,提出一种基于双延迟深度确定性策略梯度(TD3)算法的改进算法TD3pro,以提高移动机器人在未知动态环境下的路径规划性能。首先,引入长短期记忆(LSTM)神经网络并与TD3算法相结合,通过门结构筛选历史状态信息,并感知探测范围内障碍物的状态变化,帮助机器人更好地理解环境的动态变化和障碍物的移动模式,使移动机器人能够准确预测和响应动态障碍物的行为,从而降低与障碍物的碰撞率。其次,加入OU (Ornstein-Uhlenbeck)探索噪声,帮助移动机器人持续探索周围环境,增强移动机器人的探索能力和随机性。在此基础上,将单个经验池设置为成功、失败和临时3个经验池,以此提高有效经验样本的采样效率,进而减少训练时间。最后,在2个不同的动、静态障碍物混合场景中进行路径规划实验仿真。实验结果表明:场景1中该算法相较于深度确定性策略梯度(DDPG)算法以及TD3算法,模型收敛的回合数减少了100~200个,路径长度缩短了0.5~0.8,规划时间减少了1~4 s;场景2中该算法相较于TD3算法,模型收敛的回合数减少了100~300个,路径长度缩短了1~3,规划时间减少了4~8 s, DDPG算法失败,移动机器人无法成功抵达终点。由此可见,改进的算法具有更好的路径规划性能。