Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Short-TermWind Power Prediction Based on Combinatorial Neural Networks

Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.

Wind Speed Short-Term Prediction Based on Empirical Wavelet Transform, Recurrent Neural Network and Error Correction

Predicting wind speed accurately is essential to ensure the stability of the wind power system and improve the utilization rate of wind energy.However,owing to the stochastic and intermittent of wind speed,predicting wind speed accurately is difficult.A new hybrid deep learning model based on empirical wavelet transform,recurrent neural network and error correction for short-term wind speed prediction is proposed in this paper.The empirical wavelet transformation is applied to decompose the original wind speed series.The long short term memory network and the Elman neural network are adopted to predict low-frequency and high-frequency wind speed sub-layers respectively to balance the calculation efficiency and prediction accuracy.The error correction strategy based on deep long short term memory network is developed to modify the prediction errors.Four actual wind speed series are utilized to verify the effectiveness of the proposed model.The empirical results indicate that the method proposed in this paper has satisfactory performance in wind speed prediction.

基于LSTM-NeuralProphet模型的城市需水预测方法研究

城市水资源规划和管理是确保城市可持续发展和居民生活基本需求得到满足的关键环节,城市短期需水预测是城市水资源规划和管理的基础。由于气温、降水量和蒸发量等随季节变化明显,直接影响不同季节的用水峰值、高峰期,导致传统基于时间序列算法的固定时隙预测无法适应时隙的变化,从而不能保证预测精度。针对固定时隙预测精度低的问题,研究了基于四季24 h时间分辨率和夏季15 min时间分辨率的双时间尺度城市短期需水预测模型。该模型使用Anomaly-Transformer模型进行异常值检测,并通过分段曲线拟合对异常值校正,采用主成分分析法对城市短期需水影响因子进行分析提取主成分,在AutoML的标准模型分析中选取三个效果最好的模型作为Stacking模型的基学习器再结合长短期记忆网络(Long Short-Term Memory,LSTM)和Optune框架超参数优化后的NeuralProphet模型对双时间尺度的城市短期需水量进行预测,同时加入安全网机制,以保证LSTM-NeuralProphet模型的精确度。与其他模型(LSTM模型、NeuralProphet模型、BP神经网络模型)相比,LSTM-NeuralProphet模型的平均绝对误差在四季24 h时间分辨率的数据集上降低了0.18%~1.96%,在夏季15 min时间分辨率的数据集上降低了0.45%~11.90%。实验结果表明,LSTM-NeuralProphet模型具有更好的拟合效果和更高的预测精度,能较准确地预测双时间尺度下的城市需水量,可以较好地应用于城市短期需水预测研究中。

Synthetic well logs generation via Recurrent Neural Networks

To supplement missing logging information without increasing economic cost, a machine learning method to generate synthetic well logs from the existing log data was presented, and the experimental verification and application effect analysis were carried out. Since the traditional Fully Connected Neural Network(FCNN) is incapable of preserving spatial dependency, the Long Short-Term Memory(LSTM) network, which is a kind of Recurrent Neural Network(RNN), was utilized to establish a method for log reconstruction. By this method, synthetic logs can be generated from series of input log data with consideration of variation trend and context information with depth. Besides, a cascaded LSTM was proposed by combining the standard LSTM with a cascade system. Testing through real well log data shows that: the results from the LSTM are of higher accuracy than the traditional FCNN; the cascaded LSTM is more suitable for the problem with multiple series data; the machine learning method proposed provides an accurate and cost effective way for synthetic well log generation.

Bitcoin Candlestick Prediction with Deep Neural Networks Based on Real Time Data

Currently,Bitcoin is the world’s most popular cryptocurrency.The price of Bitcoin is extremely volatile,which can be described as high-benefit and high-risk.To minimize the risk involved,a means of more accurately predicting the Bitcoin price is required.Most of the existing studies of Bitcoin prediction are based on historical(i.e.,benchmark)data,without considering the real-time(i.e.,live)data.To mitigate the issue of price volatility and achieve more precise outcomes,this study suggests using historical and real-time data to predict the Bitcoin candlestick—or open,high,low,and close(OHLC)—prices.Seeking a better prediction model,the present study proposes time series-based deep learning models.In particular,two deep learning algorithms were applied,namely,long short-term memory(LSTM)and gated recurrent unit(GRU).Using real-time data,the Bitcoin candlesticks were predicted for three intervals:the next 4 h,the next 12 h,and the next 24 h.The results showed that the best-performing model was the LSTM-based model with the 4-h interval.In particular,this model achieved a stellar performance with a mean absolute percentage error(MAPE)of 0.63,a root mean square error(RMSE)of 0.0009,a mean square error(MSE)of 9e-07,a mean absolute error(MAE)of 0.0005,and an R-squared coefficient(R2)of 0.994.With these results,the proposed prediction model has demonstrated its efficiency over the models proposed in previous studies.The findings of this study have considerable implications in the business field,as the proposed model can assist investors and traders in precisely identifying Bitcoin sales and buying opportunities.

基于改进麻雀搜索算法优化LSTM的滚动轴承故障诊断

为了对滚动轴承的工作状态及故障类别进行准确的诊断,本文采用长短时记忆(LSTM)神经网络作为分类器对滚动轴承数据集进行分类诊断。首先,从滚动轴承原始运行振动信号中提取时域和频域特征参数,组成具有高维特征参数的数据集;使用核主成分分析(KPCA)方法对高维特征集进行降维处理,选取重要性程度高的特征构成输入特征向量。然后,针对LSTM神经网络在滚动轴承故障诊断中存在的超参数难以确定的问题,提出一种基于自适应t分布策略的麻雀搜索算法优化LSTM神经网络的故障诊断方法(tSSA–LSTM)。最后,使用凯斯西储大学滚动轴承数据中心的数据进行故障诊断精度测试、泛化性能测试及噪声环境下故障诊断性能测试等多个仿真实验,并将本文提出的诊断模型与麻雀搜索算法优化长短时记忆神经网络(SSA–LSTM)、遗传算法优化长短时记忆神经网络(GA–LSTM)、粒子群算法优化长短时记忆神经网络(PSO–LSTM)及传统LSTM诊断模型进行对比。结果表明:tSSA可以更有效地对LSTM的隐含层神经元数量、周期次数、学习率等超参数进行合理优化;所提方法的平均诊断准确率达到98.86%,交叉验证平均诊断结果为98.57%;所提方法在噪声干扰下的故障诊断准确率也优于对比方法。因此,本文提出的tSSA–LSTM模型不仅可以更精准地诊断滚动轴承故障状态,而且具有更强的泛化能力及抗干扰能力,有效地提高了滚动轴承故障诊断的性能。

基于字词向量融合的民航智慧监管短文本分类

为解决民航监管事项所产生的检查记录仅依靠人工进行分类分析导致效率低的问题,提出一种基于数据增强与字词向量融合的双通道特征提取的短文本分类模型,探讨民航监管事项的分类,包括与人、设备设施环境、制度程序和机构职责等相关问题。为解决类别不平衡问题,采用数据增强算法在原始文本上进行变换,生成新的样本,使各个类别的样本数量更加均衡。将字向量和词向量按字融合拼接,得到具有词特征信息的字向量。将字词融合的向量分别送入到文本卷积神经网络(TextCNN)和双向长短期记忆(BiLSTM)模型中进行不同维度的特征提取,从局部的角度和全局的角度分别提取特征,并在民航监管事项检查记录数据集上进行试验。结果表明:该模型准确率为0.9837,F 1值为0.9836。与一些字嵌入模型和词嵌入模型相对比,准确率提升0.4%。和一些常用的单通道模型相比,准确率提升3%,验证了双通道模型提取的特征具有全面性和有效性。

结合太阳辐射量计算与CNN-LSTM组合的光伏功率预测方法研究

为了提高模型预测性能,提出一种综合太阳辐射模型及深度学习的光伏功率预测模型。首先,利用太阳辐射机理建立太阳辐射模型(SRM),估算出水平面上总辐射值,再由斜面辐照度转换方法计算出光伏组件所接收的斜面辐射值。其次,通过皮尔逊相关分析法筛选出对光伏功率影响较大的主要因素,将斜面辐射计算值及主要影响因素作为输入,采用卷积神经网络(CNN)和长短期记忆网络(LSTM)建立光伏功率SRM-CNN-LSTM预测模型。分别利用春夏秋冬四季典型日的数据开展对比实验,结果表明:与几种其他方法相比,该文方法具有更好的预测效果。

基于CNN‑LSTM‑SE的心电图分类算法研究

心血管疾病是我国死亡率较高的疾病之一,通过观察心电图来判断心电信号是否出现异常能够对心血管疾病进行预防和筛查。由于心电图数据规模大且繁杂,临床医护人员在心电图筛查时,工作负担大且容易出现误诊或漏诊的情况。为了提高心电图的筛查效率、减少医护人员的压力,提出了一种基于卷积神经网络、长短期记忆神经网络和SE网络的心电图分类算法模型(CNN-LSTM-SE),该模型将心电图分成5种不同的类别。主要研究内容包括:选用MIT-BIH心律失常数据集作为心电信号的数据来源,使用巴特沃斯带通滤波器对心电信号进行去噪处理,通过Z-score方法对心电信号进行标准化处理,利用独热编码方法对心电信号标签进行编码,最后使用处理后的心电数据对所提算法模型进行训练和测试。实验结果表明:所提模型相较于其它模型,能够有效提高心电图分类的准确性,在实验数据集上的分类准确率达到99.1%。

面向喷染车间的挥发性有机物质量浓度预测方法及应用研究

以喷染车间挥发性有机物为研究对象,对喷染车间挥发性有机物(Volatile Organic Compounds, VOCs)质量浓度预测方法进行研究。首先,使用随机森林(Random Forest, RF)算法对影响喷染车间挥发性有机物质量浓度的特征变量进行权重分析。同时,构建基于长短期记忆神经网络(Long-Term and Short-Term Memory Neural Network, LSTM)的挥发性有机物质量浓度预测模型,并在此基础上引入麻雀搜索算法(Sparrow Search Algorithm, SSA)进行参数优化选择。最后,以浙江省杭州市某汽车喷染车间7月29日—10月28日的数据为样本,将温度、相对湿度、室内大气压、室外大气压作为模型输入变量,并与LSTM模型、随机森林-长短期记忆神经网络(Random Forest-Long Short-Term Memory neural network, RF-LSTM)模型、随机森林-反向传播神经网络(Random Forest-BP neural network, RF-BP)模型进行对比试验。结果显示,基于随机森林-麻雀搜索算法-长短期记忆神经网络(Random Forest-Sparrow Search Algorithm-Long Short-Term Memory neural network, RF-SSA-LSTM)模型的预测效果最佳,平均绝对误差、均方根误差和决定系数分别为2.812 2、3.457 4、0.988。同时,为验证RF-SSA-LSTM模型性能,通过不同时间步长实现对喷染车间VOCs质量浓度预测,结果显示预测误差较小,在可接受范围内。RF-SSA-LSTM预测模型提高了挥发性有机物质量浓度的预测精度,为减少挥发性有机物排放提供科学依据。

基于FMCW雷达的人体生命体征信号预测算法

将FMCW雷达检测到的人体生命体征信号,用于预测未来一段时间内人体生命体征信号是否异常,具有明显的应用价值。该方向当前研究主要针对如何进一步降低重构误差、提升生命体征信号的预测精度。为此,本文提出一种自适应变分模态分解-长短期记忆神经网络的生命体征信号预测方法。针对静止状态下的人体,通过雷达采集到的生命体征信号,采用粒子群算法优化变分模态分解VMD的模态分量个数K和惩罚系数α的值,实现自适应选取后用于VMD分解,再将分解后的模态分量进行叠加重构。采用粒子群算法优化长短期记忆网络模型中的网络层数、学习率、正则化系数等3个参数,自适应选取合适的参数组合,将重构后的信号通过优化后的LSTM网络进行预测。实验结果显示本文所提预测方法在10位志愿者的预测结果与原始数据的均方根误差平均值为0.017 188 9,平均绝对误差的平均值为0.007 158,相较于当前其他研究,预测精度上有明显提升。

机器学习算法在降水和气温多模式集成中的应用

选取CMIP6中5种全球气候模式,利用算术平均、权重平均、多元线性回归、BP神经网络、长短期记忆(LSTM)神经网络和随机森林(RF)等6种多模式集成方法,基于黄河流域水源涵养区历史降水量和气温数据,评估不同集成方法的模拟效果,并选取模拟效果最好的多模式集成方法预估未来SSP1-2.6、SSP2-4.5和SSP5-8.53种情景下黄河流域水源涵养区的降水和气温变化趋势。结果表明:多模式集成能很好地再现基准期降水和气温变化,3种机器学习算法表现相对较好,其中LSTM神经网络最好;在未来3种情景下,多年平均降水量均有所增加,四季降水量变化各有差异;SSP1-2.6情景下年降水量峰值出现在各时段初期,SSP2-4.5和SSP5-8.5情景下的年降水量呈增长趋势,远期下降趋势较明显;3种情景下气温都呈上升趋势,但变化差异较大,增温幅度和速率由小到大为SSP1-2.6、SSP2-4.5、SSP5-8.5,秋季气温增幅最大,冬季最小;多模式集成方法对未来降水量和气温的预估存在较大的不确定性,均表现为中远期大于近期,降水量预估的不确定性比气温大,其中降水量秋冬季不确定性明显大于春夏季。

基于注意力机制与LSTM-CCN的月降水量预测

针对现有月降水量预测方法预测准确性不高的问题,提出一种基于注意力机制与LSTM-CCN的月降水量预测方法。首先,利用长短时记忆神经网络(long short-term memory neural network,LSTM)提取气象数据在时间维度的特征分布,从时间相关性方面捕获相邻时间段或长距离气象数据段中的统计分布;其次,利用因果卷积神经网络(causal convolutional network,CCN)将气象数据映射到空间维度,深层次地从空间维度捕获气象数据在空间中的特征统计分布;再次,以并联的方式将时间和空间特征作为交叉注意力网络的输入,构造融合的时空特征;最后,以长短时记忆神经网络构造解码器,并将融合的时空特征作为解码器的输入,预测的月降水量作为输出。选取河南省新乡市2001~2017年数据集进行测试,结果表明:所提出方法的均方根误差仅为13.08 mm,相比主流方法具有更低的预测误差。研究成果可为提高气象预测的准确性和实用性提供参考。

基于神经网络模型的水平井破裂压力预测方法

破裂压力是井身结构设计的基础依据,也是水力压裂设备选型和方案设计的基础参数,通常采用测井解释获取破裂压力剖面,但其存在参数准确获取难、计算过程繁琐、普适性较差、计算精度低等问题,机器学习提供了一种解决这些问题的新方法。为此,以测井数据作为输入参数,采用4种不同的神经网络模型,建立水平井测井数据与破裂压力间的非线性关系,通过测试集预测结果的对比分析,优选出最佳的神经网络模型,并优化模型网络结构和超参数,实现水平井破裂压力的直接预测。研究结果表明:1)破裂压力与井斜角、横波时差和纵波时差表现为极强相关性,与井深、岩性密度和补偿中子表现为强相关性,与井径和自然伽马表现为弱相关性;2)不同组合的测井参数对模型预测结果具有显著影响,最优输入参数为井斜角、横波时差、纵波时差、井深、岩性密度和补偿中子;3)对比多层感知机、深度神经网络、循环神经网络和长短期记忆神经网络(LSTM)模型,发现LSTM模型的预测效果最佳;4)优化了LSTM模型的网络结构及超参数,优化后破裂压力预测的平均绝对百分比误差为0.106%、决定系数为0.996。LSTM模型能够有效构建水平井测井参数与破裂压力之间的非线性关系,可以实现水平井破裂压力的准确预测,对于准确预测破裂压力、简化破裂压力计算过程、推广机器学习在石油工程领域的应用具有重要的作用。

基于VMD-SE的电力负荷分量的多特征短期预测

为提高电力负荷的预测精度,提出一种基于VMD-SE的电力负荷分量的多特征短期预测方法。首先采用变分模态分解(VMD)将原始负荷分解为一系列模态分量与残差,VMD的分解层数由样本熵值(sample entropy,SE)确定;然后对比原始负荷与模态分量的SE值,重构为平稳分量和波动分量,来降低运算规模;同时利用皮尔逊相关系数来筛选特征变量,删除特征冗余,建立灰狼算法优化后的支持向量回归模型(GWO-SVR)和长短期记忆神经网络(LSTM)分别对平稳分量和波动分量预测;最后以某地区2018—2020年用电负荷为例进行实验。实验证明:此模型精准度高达94.7%,平均绝对百分误差降低到2.98%,具有更好的精准性和适用性。

基于注意力机制的CNN-BiLSTM的IGBT剩余使用寿命预测

针对绝缘栅双极型晶体管(IGBT)可靠性问题,提出了一种融合卷积神经网络(CNN)、双向长短期记忆(BiLSTM)网络和注意力机制的剩余使用寿命(RUL)预测模型,可用于IGBT的寿命预测。模型中使用CNN提取特征参数,BiLSTM提取时序信息,注意力机制加权处理特征参数。使用IGBT加速老化数据集对提出的模型进行验证。结果表明,对比自回归差分移动平均(ARIMA)、长短期记忆(LSTM)、多层LSTM(Multi-LSTM)、 BiLSTM预测模型,在均方根误差和决定系数等评价指标方面该模型的性能最优。验证了提出的寿命预测模型对IGBT失效预测是有效的。

一种采用记忆神经网络和曲线形状修正的负荷预测方法

针对分布式电源和新型负荷容量累积造成负荷影响因素多元化和不确定性特性增强的问题,文中提出一种采用记忆神经网络和曲线形状修正的负荷预测方法。在负荷峰值预测中,采用最大信息系数计算负荷峰值与影响因素的非线性相关性,实现对输入特征的筛选;综合考虑负荷峰值序列的长短期自相关性和输入特征与负荷峰值的不同程度相关性,结合Attention机制和双向长短时记忆(bidirectional long short-term memory,BiLSTM)神经网络建立负荷峰值预测模型。在负荷标幺曲线预测中,通过误差倒数法组合相似日和相邻日,建立负荷标幺曲线预测模型;针对预测偏差的非平稳特征,利用自适应噪声的完全集成经验模态分解和BiLSTM网络建立误差预测模型,对曲线形状进行修正。应用中国北方某城市的区域电网负荷数据为算例,验证了所提模型的有效性。