Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Conditional Random Field Tracking Model Based on a Visual Long Short Term Memory Network

In dense pedestrian tracking,frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories.In this study,a conditional random field tracking model is established by using a visual long short term memory network in the three-dimensional(3D)space and the motion estimations jointly performed on object trajectory segments.Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation.To address the uncertainty of the length and interval of trajectory segments,a multimode long short term memory network is proposed for the object motion estimation.The tracking performance is evaluated using the PETS2009 dataset.The experimental results show that the proposed method achieves better performance than the tracking methods based on the independent motion estimation.

A hybrid model for short-term rainstorm forecasting based on a back-propagation neural network and synoptic diagnosis

暴雨是我国最重要的自然灾害之一.大量的研究表明,暴雨的频率和强度在全球变暖的背景下正在逐年增强.但是如何成功的预测短期暴雨,特别是发生在复杂地形下的暴雨,仍然是一个巨大的挑战.本项研究采用BP神经网络和天气学诊断相结合的方法,探索了一种四川盆地西部复杂地形下的暴雨预报模型.该模型有效改善了喇叭口地形下,受低层偏东风影响的暴雨预报准确性.机器学习与天气学理论的结合,提升了模型的物理基础和预测成功率,同时该方法也为发展具有本地特征的暴雨预报客观工具,提供了一定的参考价值.

Short-TermWind Power Prediction Based on Combinatorial Neural Networks

Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.

Application of preoperative artificial neural network based on blood biomarkers and clinicopathological parameters for predicting longterm survival of patients with gastric cancer

BACKGROUND Because of the powerful abilities of self-learning and handling complex biological information,artificial neural network(ANN)models have been widely applied to disease diagnosis,imaging analysis,and prognosis prediction.However,there has been no trained preoperative ANN(preope-ANN)model to preoperatively predict the prognosis of patients with gastric cancer(GC).AIM To establish a neural network model that can predict long-term survival of GC patients before surgery to evaluate the tumor condition before the operation.METHODS The clinicopathological data of 1608 GC patients treated from January 2011 to April 2015 at the Department of Gastric Surgery,Fujian Medical University Union Hospital were analyzed retrospectively.The patients were randomly divided into a training set(70%)for establishing a preope-ANN model and a testing set(30%).The prognostic evaluation ability of the preope-ANN model was compared with that of the American Joint Commission on Cancer(8th edition)clinical TNM(cTNM)and pathological TNM(pTNM)staging through the receiver operating characteristic curve,Akaike information criterion index,Harrell's C index,and likelihood ratio chi-square.RESULTS We used the variables that were statistically significant factors for the 3-year overall survival as input-layer variables to develop a preope-ANN in the training set.The survival curves within each score of the preope-ANN had good discrimination(P<0.05).Comparing the preope-ANN model,cTNM,and pTNM in both the training and testing sets,the preope-ANN model was superior to cTNM in predictive discrimination(C index),predictive homogeneity(likelihood ratio chi-square),and prediction accuracy(area under the curve).The prediction efficiency of the preope-ANN model is similar to that of pTNM.CONCLUSION The preope-ANN model can accurately predict the long-term survival of GC patients,and its predictive efficiency is not inferior to that of pTNM stage.

A Short-Term Climate Prediction Model Based on a Modular Fuzzy Neural Network

In terms of the modular fuzzy neural network (MFNN) combining fuzzy c-mean (FCM) cluster and single-layer neural network, a short-term climate prediction model is developed. It is found from modeling results that the MFNN model for short-term climate prediction has advantages of simple structure, no hidden layer and stable network parameters because of the assembling of sound functions of the self-adaptive learning, association and fuzzy information processing of fuzzy mathematics and neural network methods. The case computational results of Guangxi flood season (JJA) rainfall show that the mean absolute error (MAE) and mean relative error (MRE) of the prediction during 1998-2002 are 68.8 mm and 9.78%, and in comparison with the regression method, under the conditions of the same predictors and period they are 97.8 mm and 12.28% respectively. Furthermore, it is also found from the stability analysis of the modular model that the change of the prediction results of independent samples with training times in the stably convergent interval of the model is less than 1.3 mm. The obvious oscillation phenomenon of prediction results with training times, such as in the common back-propagation neural network (BPNN) model, does not occur, indicating a better practical application potential of the MFNN model.

Short-Term Electricity Price Forecasting Using a Combination of Neural Networks and Fuzzy Inference

This paper presents an artificial neural network, ANN, based approach for estimating short-term wholesale electricity prices using past price and demand data. The objective is to utilize the piecewise continuous na-ture of electricity prices on the time domain by clustering the input data into time ranges where the variation trends are maintained. Due to the imprecise nature of cluster boundaries a fuzzy inference technique is em-ployed to handle data that lies at the intersections. As a necessary step in forecasting prices the anticipated electricity demand at the target time is estimated first using a separate ANN. The Australian New-South Wales electricity market data was used to test the system. The developed system shows considerable im-provement in performance compared with approaches that regard price data as a single continuous time se-ries, achieving MAPE of less than 2% for hours with steady prices and 8% for the clusters covering time pe-riods with price spikes.

Short-Term and Long-Term Price Forecasting Models for the Future Exchange of Mongolian Natural Sea Buckthorn Market

Sea buckthorn market floated uncertainly within a narrow range. The market situation provided upward pressure on prices, and producer and consumer interest were poor, coupled with weak prices in the regional markets. The objectives of the study are: 1) to estimate the relationship between wild Sea buckthorn (SB) price and Supply, Demand, while some other factors of crude oil price and exchange rate by using simultaneous Supply-Demand and Price system equation and Vector Error Correction Method (VECM);2) to forecast the short-term and long-term SB price;3) to compare and evaluate the price forecasting models. Firstly, the data was analyzed by Ferris and Engle-Granger’s procedure;secondly, both price forecasting methodologies were tested by Pindyck-Rubinfeld and Makridakis’s procedure. The result shows that the VECM model is more efficient using yearly data;a short-term price forecast decreases, and a long-term price forecast is predicted to increase the Mongolian Sea buckthorn market.

Decision Technique of Solar Radiation Prediction Applying Recurrent Neural Network for Short-Term Ahead Power Output of Photovoltaic System

In recent years, introduction of a renewable energy source such as solar energy is expected. However, solar radiation is not constant and power output of photovoltaic (PV) system is influenced by weather conditions. It is difficult for getting to know accurate power output of PV system. In order to forecast the power output of PV system as accurate as possible, this paper proposes a decision technique of forecasting model for short-term-ahead power output of PV system based on solar radiation prediction. Application of Recurrent Neural Network (RNN) is shown for solar radiation prediction in this paper. The proposed method in this paper does not require complicated calculation, but mathematical model with only useful weather data. The validity of the proposed RNN is confirmed by comparing simulation results of solar radiation forecasting with that obtained from other

Short-Term Relay Quality Prediction Algorithm Based on Long and Short-Term Memory

The fraction defective of semi-finished products is predicted to optimize the process of relay production lines, by which production quality and productivity are increased, and the costs are decreased. The process parameters of relay production lines are studied based on the long-and-short-term memory network. Then, the Keras deep learning framework is utilized to build up a short-term relay quality prediction algorithm for the semi-finished product. A simulation model is used to study prediction algorithm. The simulation results show that the average prediction absolute error of the fraction is less than 5%. This work displays great application potential in the relay production lines.

Classification of Arrhythmia Based on Convolutional Neural Networks and Encoder-Decoder Model

As a common and high-risk type of disease,heart disease seriously threatens people’s health.At the same time,in the era of the Internet of Thing(IoT),smart medical device has strong practical significance for medical workers and patients because of its ability to assist in the diagnosis of diseases.Therefore,the research of real-time diagnosis and classification algorithms for arrhythmia can help to improve the diagnostic efficiency of diseases.In this paper,we design an automatic arrhythmia classification algorithm model based on Convolutional Neural Network(CNN)and Encoder-Decoder model.The model uses Long Short-Term Memory(LSTM)to consider the influence of time series features on classification results.Simultaneously,it is trained and tested by the MIT-BIH arrhythmia database.Besides,Generative Adversarial Networks(GAN)is adopted as a method of data equalization for solving data imbalance problem.The simulation results show that for the inter-patient arrhythmia classification,the hybrid model combining CNN and Encoder-Decoder model has the best classification accuracy,of which the accuracy can reach 94.05%.Especially,it has a better advantage for the classification effect of supraventricular ectopic beats(class S)and fusion beats(class F).

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.

Forecasting of Short-term Load based on LMD and BBO-RBF Model

Short-term load forecasting is a basis of power system dispatching and operation. In order to improve the short term power load precision, a novel approach for short-term load forecasting is presented based on local mean decomposition (LMD) and the radial basis function neural network method (RBFNN). Firstly, the decomposition of LMD method based on characteristics of load data then the decomposed data are respectively predicted by using the RBF network model and predicted by using the BBO-RBF network model. The simulation results show that the RBF network model optimized by using BBO algorithm is optimized in error performance index, and the prediction accuracy is higher and more effective.

Wind Speed Short-Term Prediction Based on Empirical Wavelet Transform, Recurrent Neural Network and Error Correction

Predicting wind speed accurately is essential to ensure the stability of the wind power system and improve the utilization rate of wind energy.However,owing to the stochastic and intermittent of wind speed,predicting wind speed accurately is difficult.A new hybrid deep learning model based on empirical wavelet transform,recurrent neural network and error correction for short-term wind speed prediction is proposed in this paper.The empirical wavelet transformation is applied to decompose the original wind speed series.The long short term memory network and the Elman neural network are adopted to predict low-frequency and high-frequency wind speed sub-layers respectively to balance the calculation efficiency and prediction accuracy.The error correction strategy based on deep long short term memory network is developed to modify the prediction errors.Four actual wind speed series are utilized to verify the effectiveness of the proposed model.The empirical results indicate that the method proposed in this paper has satisfactory performance in wind speed prediction.

Short-term load forecasting based on fuzzy neural network

The fuzzy neural network is applied to the short-term load forecasting. The fuzzy rules and fuzzy membership functions of the network are obtained through fuzzy neural network learming. Three inference algorithms, i.e. themultiplicative inference, the maximum inference and the minimum inference, are used for comparison. The learningalgorithms corresponding to the inference methods are derived from back-propagation algorithm. To validate the fuzzyneural network model, the network is used to Predict short-term load by compaing the network output against the realload data from a local power system supplying electricity to a large steel manufacturer. The experimental results aresatisfactory.

Short-Term Load Forecasting Using Radial Basis Function Neural Network

An accurate short-term forecasting method for load of electric power system can help the electric power system’s operator to reduce the risk of unreliability of electricity supply. This paper proposed a radial basis function (RBF) neural network method to forecast the short-term load of electric power system. To demonstrate the effectiveness of the proposed method, the method is tested on the practical load data information of the Tai power system. The good agreements between the realistic values and forecasting values are obtained;the numerical results show that the proposed forecasting method is accurate and reliable.

Short Term Load Forecast Using Wavelet Neural Network

This paper presents a wavelet neural network (WNN) model combining wavelet transform and artificial neural networks for short term load forecast (STLF). Both historical load and temperature data having important impacts on load level were used in the proposed forecasting model. The model used the three-layer feed forward network trained by the error back-propagation algorithm. To enhance the forecast- ing accuracy by neural networks, wavelet multi-resolution analysis method was introduced to pre-process these data and reconstruct the predicted output. The proposed model has been evaluated with actual data of electricity load and temperature of Hunan Province. The simulation results show that the model is capable of providing a reasonable forecasting accuracy in STLF.

State of Health Estimation of Lithium-Ion Batteries Using Support Vector Regression and Long Short-Term Memory

Lithium-ion batteries are the most widely accepted type of battery in the electric vehicle industry because of some of their positive inherent characteristics. However, the safety problems associated with inaccurate estimation and prediction of the state of health of these batteries have attracted wide attention due to the adverse negative effect on vehicle safety. In this paper, both machine and deep learning models were used to estimate the state of health of lithium-ion batteries. The paper introduces the definition of battery health status and its importance in the electric vehicle industry. Based on the data preprocessing and visualization analysis, three features related to actual battery capacity degradation are extracted from the data. Two learning models, SVR and LSTM were employed for the state of health estimation and their respective results are compared in this paper. The mean square error and coefficient of determination were the two metrics for the performance evaluation of the models. The experimental results indicate that both models have high estimation results. However, the metrics indicated that the SVR was the overall best model.