Short-TermWind Power Prediction Based on Combinatorial Neural Networks

Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Estimation of unloading relaxation depth of Baihetan Arch Dam foundation using long-short term memory network

The unloading relaxation caused by excavation for construction of high arch dams is an important factor influencing the foundation’s integrity and strength.To evaluate the degree of unloading relaxation,the long-short term memory(LSTM)network was used to estimate the depth of unloading relaxation zones on the left bank foundation of the Baihetan Arch Dam.Principal component analysis indicates that rock charac-teristics,the structural plane,the protection layer,lithology,and time are the main factors.The LSTM network results demonstrate the unloading relaxation characteristics of the left bank,and the relationships with the factors were also analyzed.The structural plane has the most significant influence on the distribution of unloading relaxation zones.Compared with massive basalt,the columnar jointed basalt experiences a more significant unloading relaxation phenomenon with a clear time effect,with the average unloading relaxation period being 50 d.The protection layer can effectively reduce the unloading relaxation depth by approximately 20%.

ART-2 neural network based on eternal term memory vector:Architecture and algorithm

Aimed at the problem that the traditional ART-2 neural network can not recognize a gradually changing course, an eternal term memory (ETM) vector is introduced into ART-2 to simulate the function of human brain, i.e. the deep remembrance for the initial impression.. The eternal term memory vector is determined only by the initial vector that establishes category neuron node and is used to keep the remembrance for this vector for ever. Two times of vigilance algorithm are put forward, and the posterior input vector must first pass the first vigilance of this eternal term memory vector, only succeeded has it the qualification to begin the second vigilance of long term memory vector. The long term memory vector can be revised only when both of the vigilances are passed. Results of recognition examples show that the improved ART-2 overcomes the defect of traditional ART-2 and can recognize a gradually changing course effectively.

Feature identification in complex fluid flows by convolutional neural networks

Recent advancements have established machine learning's utility in predicting nonlinear fluid dynamics,with predictive accuracy being a central motivation for employing neural networks.However,the pattern recognition central to the networks function is equally valuable for enhancing our dynamical insight into the complex fluid dynamics.In this paper,a single-layer convolutional neural network(CNN)was trained to recognize three qualitatively different subsonic buffet flows(periodic,quasi-periodic and chaotic)over a high-incidence airfoil,and a near-perfect accuracy was obtained with only a small training dataset.The convolutional kernels and corresponding feature maps,developed by the model with no temporal information provided,identified large-scale coherent structures in agreement with those known to be associated with buffet flows.Sensitivity to hyperparameters including network architecture and convolutional kernel size was also explored.The coherent structures identified by these models enhance our dynamical understanding of subsonic buffet over high-incidence airfoils over a wide range of Reynolds numbers.

Memory Analysis for Memristors and Memristive Recurrent Neural Networks

Traditional recurrent neural networks are composed of capacitors, inductors, resistors, and operational amplifiers.Memristive neural networks are constructed by replacing resistors with memristors. This paper focuses on the memory analysis,i.e. the initial value computation, of memristors. Firstly, we present the memory analysis for a single memristor based on memristors’ mathematical models with linear and nonlinear drift.Secondly, we present the memory analysis for two memristors in series and parallel. Thirdly, we point out the difference between traditional neural networks and those that are memristive. Based on the current and voltage relationship of memristors, we use mathematical analysis and SPICE simulations to demonstrate the validity of our methods.

Wind Speed Short-Term Prediction Based on Empirical Wavelet Transform, Recurrent Neural Network and Error Correction

Predicting wind speed accurately is essential to ensure the stability of the wind power system and improve the utilization rate of wind energy.However,owing to the stochastic and intermittent of wind speed,predicting wind speed accurately is difficult.A new hybrid deep learning model based on empirical wavelet transform,recurrent neural network and error correction for short-term wind speed prediction is proposed in this paper.The empirical wavelet transformation is applied to decompose the original wind speed series.The long short term memory network and the Elman neural network are adopted to predict low-frequency and high-frequency wind speed sub-layers respectively to balance the calculation efficiency and prediction accuracy.The error correction strategy based on deep long short term memory network is developed to modify the prediction errors.Four actual wind speed series are utilized to verify the effectiveness of the proposed model.The empirical results indicate that the method proposed in this paper has satisfactory performance in wind speed prediction.

Application of Neural Network in Fault Location of Optical Transport Network

Due to the increasing variety of information and services carried by optical networks, the survivability of network becomes an important problem in current research. The fault location of OTN is of great significance for studying the survivability of optical networks. Firstly, a three-channel network model is established and analyzing common alarm data, the fault monitoring points and common fault points are carried out. The artificial neural network is introduced into the fault location field of OTN and it is used to judge whether the possible fault point exists or not. But one of the obvious limitations of general neural networks is that they receive a fixedsize vector as input and produce a fixed-size vector as the output. Not only that, these models is even fixed for mapping operations (for example, the number of layers in the model). The difference between the recurrent neural network and general neural networks is that it can operate on the sequence. In spite of the fact that the gradient disappears and the gradient explodes still exist in the neural network, the method of gradient shearing or weight regularization is adopted to solve this problem, and choose the LSTM (long-short term memory networks) to locate the fault. The output uses the concept of membership degree of fuzzy theory to express the possible fault point with the probability from 0 to 1. Priority is given to the treatment of fault points with high probability. The concept of F-Measure is also introduced, and the positioning effect is measured by using location time, MSE and F-Measure. The experiment shows that both LSTM and BP neural network can locate the fault of optical transport network well, but the overall effect of LSTM is better. The localization time of LSTM is shorter than that of BP neural network, and the F1-score of LSTM can reach 0.961566888396156 after 45 iterations, which meets the accuracy and real-time requirements of fault location. Therefore, it has good application prospect and practical value to introduce neural network into the fault location field of optical transport network.

ST-Trader:A Spatial-Temporal Deep Neural Network for Modeling Stock Market Movement

Stocks that are fundamentally connected with each other tend to move together.Considering such common trends is believed to benefit stock movement forecasting tasks.However,such signals are not trivial to model because the connections among stocks are not physically presented and need to be estimated from volatile data.Motivated by this observation,we propose a framework that incorporates the inter-connection of firms to forecast stock prices.To effectively utilize a large set of fundamental features,we further design a novel pipeline.First,we use variational autoencoder(VAE)to reduce the dimension of stock fundamental information and then cluster stocks into a graph structure(fundamentally clustering).Second,a hybrid model of graph convolutional network and long-short term memory network(GCN-LSTM)with an adjacency graph matrix(learnt from VAE)is proposed for graph-structured stock market forecasting.Experiments on minute-level U.S.stock market data demonstrate that our model effectively captures both spatial and temporal signals and achieves superior improvement over baseline methods.The proposed model is promising for other applications in which there is a possible but hidden spatial dependency to improve time-series prediction.

Synthetic well logs generation via Recurrent Neural Networks

To supplement missing logging information without increasing economic cost, a machine learning method to generate synthetic well logs from the existing log data was presented, and the experimental verification and application effect analysis were carried out. Since the traditional Fully Connected Neural Network(FCNN) is incapable of preserving spatial dependency, the Long Short-Term Memory(LSTM) network, which is a kind of Recurrent Neural Network(RNN), was utilized to establish a method for log reconstruction. By this method, synthetic logs can be generated from series of input log data with consideration of variation trend and context information with depth. Besides, a cascaded LSTM was proposed by combining the standard LSTM with a cascade system. Testing through real well log data shows that: the results from the LSTM are of higher accuracy than the traditional FCNN; the cascaded LSTM is more suitable for the problem with multiple series data; the machine learning method proposed provides an accurate and cost effective way for synthetic well log generation.

Bitcoin Candlestick Prediction with Deep Neural Networks Based on Real Time Data

Currently,Bitcoin is the world’s most popular cryptocurrency.The price of Bitcoin is extremely volatile,which can be described as high-benefit and high-risk.To minimize the risk involved,a means of more accurately predicting the Bitcoin price is required.Most of the existing studies of Bitcoin prediction are based on historical(i.e.,benchmark)data,without considering the real-time(i.e.,live)data.To mitigate the issue of price volatility and achieve more precise outcomes,this study suggests using historical and real-time data to predict the Bitcoin candlestick—or open,high,low,and close(OHLC)—prices.Seeking a better prediction model,the present study proposes time series-based deep learning models.In particular,two deep learning algorithms were applied,namely,long short-term memory(LSTM)and gated recurrent unit(GRU).Using real-time data,the Bitcoin candlesticks were predicted for three intervals:the next 4 h,the next 12 h,and the next 24 h.The results showed that the best-performing model was the LSTM-based model with the 4-h interval.In particular,this model achieved a stellar performance with a mean absolute percentage error(MAPE)of 0.63,a root mean square error(RMSE)of 0.0009,a mean square error(MSE)of 9e-07,a mean absolute error(MAE)of 0.0005,and an R-squared coefficient(R2)of 0.994.With these results,the proposed prediction model has demonstrated its efficiency over the models proposed in previous studies.The findings of this study have considerable implications in the business field,as the proposed model can assist investors and traders in precisely identifying Bitcoin sales and buying opportunities.

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.

Online multi-target intelligent tracking using a deep long-short term memory network

Multi-target tracking is facing the difficulties of modeling uncertain motion and observation noise.Traditional tracking algorithms are limited by specific models and priors that may mismatch a real-world scenario.In this paper,considering the model-free purpose,we present an online Multi-Target Intelligent Tracking(MTIT)algorithm based on a Deep Long-Short Term Memory(DLSTM)network for complex tracking requirements,named the MTIT-DLSTM algorithm.Firstly,to distinguish trajectories and concatenate the tracking task in a time sequence,we define a target tuple set that is the labeled Random Finite Set(RFS).Then,prediction and update blocks based on the DLSTM network are constructed to predict and estimate the state of targets,respectively.Further,the prediction block can learn the movement trend from the historical state sequence,while the update block can capture the noise characteristic from the historical measurement sequence.Finally,a data association scheme based on Hungarian algorithm and the heuristic track management strategy are employed to assign measurements to targets and adapt births and deaths.Experimental results manifest that,compared with the existing tracking algorithms,our proposed MTIT-DLSTM algorithm can improve effectively the accuracy and robustness in estimating the state of targets appearing at random positions,and be applied to linear and nonlinear multi-target tracking scenarios.

Dynamic Resource Allocation in LTE Radio Access Network Using Machine Learning Techniques

Current LTE networks are experiencing significant growth in the number of users worldwide. The use of data services for online browsing, e-learning, online meetings and initiatives such as smart cities means that subscribers stay connected for long periods, thereby saturating a number of signalling resources. One of such resources is the Radio Resource Connected (RRC) parameter, which is allocated to eNodeBs with the aim of limiting the number of connected simultaneously in the network. The fixed allocation of this parameter means that, depending on the traffic at different times of the day and the geographical position, some eNodeBs are saturated with RRC resources (overused) while others have unused RRC resources. However, as these resources are limited, there is the problem of their underutilization (non-optimal utilization of resources at the eNodeB level) due to static allocation (manual configuration of resources). The objective of this paper is to design an efficient machine learning model that will take as input some key performance indices (KPIs) like traffic data, RRC, simultaneous users, etc., for each eNodeB per hour and per day and accurately predict the number of needed RRC resources that will be dynamically allocated to them in order to avoid traffic and financial losses to the mobile network operator. To reach this target, three machine learning algorithms have been studied namely: linear regression, convolutional neural networks and long short-term memory (LSTM) to train three models and evaluate them. The model trained with the LSTM algorithm gave the best performance with 97% accuracy and was therefore implemented in the proposed solution for RRC resource allocation. An interconnection architecture is also proposed to embed the proposed solution into the Operation and maintenance network of a mobile network operator. In this way, the proposed solution can contribute to developing and expanding the concept of Self Organizing Network (SON) used in 4G and 5G networks.

结合太阳辐射量计算与CNN-LSTM组合的光伏功率预测方法研究

为了提高模型预测性能,提出一种综合太阳辐射模型及深度学习的光伏功率预测模型。首先,利用太阳辐射机理建立太阳辐射模型(SRM),估算出水平面上总辐射值,再由斜面辐照度转换方法计算出光伏组件所接收的斜面辐射值。其次,通过皮尔逊相关分析法筛选出对光伏功率影响较大的主要因素,将斜面辐射计算值及主要影响因素作为输入,采用卷积神经网络(CNN)和长短期记忆网络(LSTM)建立光伏功率SRM-CNN-LSTM预测模型。分别利用春夏秋冬四季典型日的数据开展对比实验,结果表明:与几种其他方法相比,该文方法具有更好的预测效果。

基于ConvLSTM-CNN预测太平洋长鳍金枪鱼时空分布趋势

海洋渔场的变动由空间与环境因子共同驱动,渔场时空演变信息的精准预测是海洋捕捞的关键。本研究利用1995-2018年太平洋海域长鳍金枪鱼(Thunnus alalunga)的渔业生产统计数据,结合同期海洋环境数据包括海表面温度(Sea Surface Temperature,SST)、海表面盐度(Sea Surface Salinity,SSS)、初级生产力(Primary Productivity,PP)和溶解氧浓度(Dissolved Oxygen Concentration,DO),提出了一种融合卷积长短期记忆网络(Convolutional Long Short-Term Memory Networks,ConvLSTM)和卷积神经网络(Convolutional Neural Networks,CNN)的渔场时空分布预测模型。该模型引入特征提取模块,对时空因子进行编码,提取时空特征信息,同时采用CNN提取海洋环境变量的抽象特征,采用ConvLSTM提取渔业数据的高层时空关联信息,最后融合多种特征对渔场时空演变趋势进行预测。结果表明,模型的均方根误差为0.1036,较随机森林、BP神经网络和长短期记忆网络(Long Short Term Memory,LSTM)等传统渔场预报模型的预测误差降低15%~40%,预测的高产渔区与实际作业的高渔获量区匹配度为89%。该研究构建的渔场时空预测模型能够准确地预测出太平洋长鳍金枪鱼的时空分布,为太平洋长鳍金枪鱼的延绳钓渔业提供科学参考依据。

基于字词向量融合的民航智慧监管短文本分类

为解决民航监管事项所产生的检查记录仅依靠人工进行分类分析导致效率低的问题,提出一种基于数据增强与字词向量融合的双通道特征提取的短文本分类模型,探讨民航监管事项的分类,包括与人、设备设施环境、制度程序和机构职责等相关问题。为解决类别不平衡问题,采用数据增强算法在原始文本上进行变换,生成新的样本,使各个类别的样本数量更加均衡。将字向量和词向量按字融合拼接,得到具有词特征信息的字向量。将字词融合的向量分别送入到文本卷积神经网络(TextCNN)和双向长短期记忆(BiLSTM)模型中进行不同维度的特征提取,从局部的角度和全局的角度分别提取特征,并在民航监管事项检查记录数据集上进行试验。结果表明:该模型准确率为0.9837,F 1值为0.9836。与一些字嵌入模型和词嵌入模型相对比,准确率提升0.4%。和一些常用的单通道模型相比,准确率提升3%,验证了双通道模型提取的特征具有全面性和有效性。

机器学习算法在降水和气温多模式集成中的应用

选取CMIP6中5种全球气候模式,利用算术平均、权重平均、多元线性回归、BP神经网络、长短期记忆(LSTM)神经网络和随机森林(RF)等6种多模式集成方法,基于黄河流域水源涵养区历史降水量和气温数据,评估不同集成方法的模拟效果,并选取模拟效果最好的多模式集成方法预估未来SSP1-2.6、SSP2-4.5和SSP5-8.53种情景下黄河流域水源涵养区的降水和气温变化趋势。结果表明:多模式集成能很好地再现基准期降水和气温变化,3种机器学习算法表现相对较好,其中LSTM神经网络最好;在未来3种情景下,多年平均降水量均有所增加,四季降水量变化各有差异;SSP1-2.6情景下年降水量峰值出现在各时段初期,SSP2-4.5和SSP5-8.5情景下的年降水量呈增长趋势,远期下降趋势较明显;3种情景下气温都呈上升趋势,但变化差异较大,增温幅度和速率由小到大为SSP1-2.6、SSP2-4.5、SSP5-8.5,秋季气温增幅最大,冬季最小;多模式集成方法对未来降水量和气温的预估存在较大的不确定性,均表现为中远期大于近期,降水量预估的不确定性比气温大,其中降水量秋冬季不确定性明显大于春夏季。

基于改进麻雀搜索算法优化LSTM的滚动轴承故障诊断

为了对滚动轴承的工作状态及故障类别进行准确的诊断,本文采用长短时记忆(LSTM)神经网络作为分类器对滚动轴承数据集进行分类诊断。首先,从滚动轴承原始运行振动信号中提取时域和频域特征参数,组成具有高维特征参数的数据集;使用核主成分分析(KPCA)方法对高维特征集进行降维处理,选取重要性程度高的特征构成输入特征向量。然后,针对LSTM神经网络在滚动轴承故障诊断中存在的超参数难以确定的问题,提出一种基于自适应t分布策略的麻雀搜索算法优化LSTM神经网络的故障诊断方法(tSSA–LSTM)。最后,使用凯斯西储大学滚动轴承数据中心的数据进行故障诊断精度测试、泛化性能测试及噪声环境下故障诊断性能测试等多个仿真实验,并将本文提出的诊断模型与麻雀搜索算法优化长短时记忆神经网络(SSA–LSTM)、遗传算法优化长短时记忆神经网络(GA–LSTM)、粒子群算法优化长短时记忆神经网络(PSO–LSTM)及传统LSTM诊断模型进行对比。结果表明:tSSA可以更有效地对LSTM的隐含层神经元数量、周期次数、学习率等超参数进行合理优化;所提方法的平均诊断准确率达到98.86%,交叉验证平均诊断结果为98.57%;所提方法在噪声干扰下的故障诊断准确率也优于对比方法。因此,本文提出的tSSA–LSTM模型不仅可以更精准地诊断滚动轴承故障状态,而且具有更强的泛化能力及抗干扰能力,有效地提高了滚动轴承故障诊断的性能。

基于卷积和长短期记忆网络的地浸开采铀浓度预测研究

文章通过集成经验模态分解(Empirical Mode Decomposition,EMD)、卷积神经网络(Convolutional Neural Networks,CNN)、长短期记忆网络(Long Short-Term Memory,LSTM)、傅里叶变换,提出了一种新型地浸单元浸出液铀浓度预测方法。该方法将浸出液铀浓度监测值时间序列使用EMD进行分解,分解为趋势项、周期项和随机项。通过构建CNN+LSTM网络,并结合傅里叶变换和多项式拟合对铀浓度趋势项、周期项和随机项进行预测,3者预测之和作为铀浓度预测结果。实证结果表明:EMD能够有效分解铀浓度时间序列,模型拟合度比未进行EMD分解的模型提升超50%;基于EMD、CNN+LSTM和傅里叶变换的集成方法预测精度良好,预测的平均绝对百分比误差(Mean Absolute Percentage Error,MAPE)为0.348,与LSTM、反向传播(Back Propagation,BP)和门控循环网络(Gate Recurrent Unit,GRU)等模型相比最高提升超80%。文章提出的集成方法能够准确预测浸采单元铀浓度变化,解决了原有方法和模型无法对非线性、非平稳铀浓度序列进行准确预测的问题,从而为地浸矿山生产规划提供技术支持,并有助于提升中国铀矿山的数字化、信息化程度。