Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Classification of Arrhythmia Based on Convolutional Neural Networks and Encoder-Decoder Model

As a common and high-risk type of disease,heart disease seriously threatens people’s health.At the same time,in the era of the Internet of Thing(IoT),smart medical device has strong practical significance for medical workers and patients because of its ability to assist in the diagnosis of diseases.Therefore,the research of real-time diagnosis and classification algorithms for arrhythmia can help to improve the diagnostic efficiency of diseases.In this paper,we design an automatic arrhythmia classification algorithm model based on Convolutional Neural Network(CNN)and Encoder-Decoder model.The model uses Long Short-Term Memory(LSTM)to consider the influence of time series features on classification results.Simultaneously,it is trained and tested by the MIT-BIH arrhythmia database.Besides,Generative Adversarial Networks(GAN)is adopted as a method of data equalization for solving data imbalance problem.The simulation results show that for the inter-patient arrhythmia classification,the hybrid model combining CNN and Encoder-Decoder model has the best classification accuracy,of which the accuracy can reach 94.05%.Especially,it has a better advantage for the classification effect of supraventricular ectopic beats(class S)and fusion beats(class F).

A phenomenological memristor model for synaptic memory and learning behaviors

Properties that are similar to the memory and learning functions in biological systems have been observed and reported in the experimental studies of memristors fabricated by different materials. These properties include the forgetting effect, the transition from short-term memory（STM） to long-term memory（LTM）, learning-experience behavior, etc. The mathematical model of this kind of memristor would be very important for its theoretical analysis and application design.In our analysis of the existing memristor model with these properties, we find that some behaviors of the model are inconsistent with the reported experimental observations. A phenomenological memristor model is proposed for this kind of memristor. The model design is based on the forgetting effect and STM-to-LTM transition since these behaviors are two typical properties of these memristors. Further analyses of this model show that this model can also be used directly or modified to describe other experimentally observed behaviors. Simulations show that the proposed model can give a better description of the reported memory and learning behaviors of this kind of memristor than the existing model.

Prediction of discharge in a tidal river using the LSTM-based sequence-to-sequence models

The complexity of river-tide interaction poses a significant challenge in predicting discharge in tidal rivers.Long short-term memory(LSTM)networks excel in processing and predicting crucial events with extended intervals and time delays in time series data.Additionally,the sequence-to-sequence(Seq2Seq)model,known for handling temporal relationships,adapting to variable-length sequences,effectively capturing historical information,and accommodating various influencing factors,emerges as a robust and flexible tool in discharge forecasting.In this study,we introduce the application of LSTM-based Seq2Seq models for the first time in forecasting the discharge of a tidal reach of the Changjiang River(Yangtze River)Estuary.This study focuses on discharge forecasting using three key input characteristics:flow velocity,water level,and discharge,which means the structure of multiple input and single output is adopted.The experiment used the discharge data of the whole year of 2020,of which the first 80%is used as the training set,and the last 20%is used as the test set.This means that the data covers different tidal cycles,which helps to test the forecasting effect of different models in different tidal cycles and different runoff.The experimental results indicate that the proposed models demonstrate advantages in long-term,mid-term,and short-term discharge forecasting.The Seq2Seq models improved by 6%-60%and 5%-20%of the relative standard deviation compared to the harmonic analysis models and improved back propagation neural network models in discharge prediction,respectively.In addition,the relative accuracy of the Seq2Seq model is 1%to 3%higher than that of the LSTM model.Analytical assessment of the prediction errors shows that the Seq2Seq models are insensitive to the forecast lead time and they can capture characteristic values such as maximum flood tide flow and maximum ebb tide flow in the tidal cycle well.This indicates the significance of the Seq2Seq models.

Deep Learning-Based Stock Price Prediction Using LSTM Model

The stock market is a vital component of the broader financial system,with its dynamics closely linked to economic growth.The challenges associated with analyzing and forecasting stock prices have persisted since the inception of financial markets.By examining historical transaction data,latent opportunities for profit can be uncovered,providing valuable insights for both institutional and individual investors to make more informed decisions.This study focuses on analyzing historical transaction data from four banks to predict closing price trends.Various models,including decision trees,random forests,and Long Short-Term Memory(LSTM)networks,are employed to forecast stock price movements.Historical stock transaction data serves as the input for training these models,which are then used to predict upward or downward stock price trends.The study’s empirical results indicate that these methods are effective to a degree in predicting stock price movements.The LSTM-based deep neural network model,in particular,demonstrates a commendable level of predictive accuracy.This conclusion is reached following a thorough evaluation of model performance,highlighting the potential of LSTM models in stock market forecasting.The findings offer significant implications for advancing financial forecasting approaches,thereby improving the decision-making capabilities of investors and financial institutions.

Recent Progresses in Deep Learning Based Acoustic Models

In this paper,we summarize recent progresses made in deep learning based acoustic models and the motivation and insights behind the surveyed techniques.We first discuss models such as recurrent neural networks(RNNs) and convolutional neural networks(CNNs) that can effectively exploit variablelength contextual information,and their various combination with other models.We then describe models that are optimized end-to-end and emphasize on feature representations learned jointly with the rest of the system,the connectionist temporal classification(CTC) criterion,and the attention-based sequenceto-sequence translation model.We further illustrate robustness issues in speech recognition systems,and discuss acoustic model adaptation,speech enhancement and separation,and robust training strategies.We also cover modeling techniques that lead to more efficient decoding and discuss possible future directions in acoustic model research.

Physics-informed neural network approach for heat generation rate estimation of lithium-ion battery under various driving conditions

Accurate insight into the heat generation rate(HGR) of lithium-ion batteries(LIBs) is one of key issues for battery management systems to formulate thermal safety warning strategies in advance.For this reason,this paper proposes a novel physics-informed neural network(PINN) approach for HGR estimation of LIBs under various driving conditions.Specifically,a single particle model with thermodynamics(SPMT) is first constructed for extracting the critical physical knowledge related with battery HGR.Subsequently,the surface concentrations of positive and negative electrodes in battery SPMT model are integrated into the bidirectional long short-term memory(BiLSTM) networks as physical information.And combined with other feature variables,a novel PINN approach to achieve HGR estimation of LIBs with higher accuracy is constituted.Additionally,some critical hyperparameters of BiLSTM used in PINN approach are determined through Bayesian optimization algorithm(BOA) and the results of BOA-based BiLSTM are compared with other traditional BiLSTM/LSTM networks.Eventually,combined with the HGR data generated from the validated virtual battery,it is proved that the proposed approach can well predict the battery HGR under the dynamic stress test(DST) and worldwide light vehicles test procedure(WLTP),the mean absolute error under DST is 0.542 kW/m^(3),and the root mean square error under WLTP is1.428 kW/m^(3)at 25℃.Lastly,the investigation results of this paper also show a new perspective in the application of the PINN approach in battery HGR estimation.

An Aircraft Trajectory Anomaly Detection Method Based on Deep Mixture Density Network

The timely and accurately detection of abnormal aircraft trajectory is critical to improving flight safety.However,the existing anomaly detection methods based on machine learning cannot well characterize the features of aircraft trajectories.Low anomaly detection accuracy still exists due to the high-dimensionality,heterogeneity and temporality of flight trajectory data.To this end,this paper proposes an abnormal trajectory detection method based on the deep mixture density network(DMDN)to detect flights with unusual data patterns and evaluate flight trajectory safety.The technique consists of two components:Utilization of the deep long short-term memory(LSTM)network to encode features of flight trajectories effectively,and parameterization of the statistical properties of flight trajectory using the Gaussian mixture model(GMM).Experiment results on Guangzhou Baiyun International Airport terminal airspace show that the proposed method can effectively capture the statistical patterns of aircraft trajectories.The model can detect abnormal flights with elevated risks and its performance is superior to two mainstream methods.The proposed model can be used as an assistant decision-making tool for air traffic controllers.

Comparative analysis of time series neural network methods for three-way catalyst modeling

Relative Oxygen Level of the Three-Way Catalyst is an important parameter that affects the conversion efficiency of pollutants. ROL is a time-varying hidden state variable that is difficult to directly observe in practice. Therefore, it is common to use a method of clearing oxygen storage to simplify control in vehicles. However, this method negates the positive effects of ROL on pollutant treatment. ROL can be indirectly observed through modeling methods. Chemical modeling methods involve extensive computational requirements that cannot meet the demands of practical control. In contrast, time-series neural networks offer computational speed advantages when dealing with similar problems. Therefore, the ROL observation models using both NARX and LSTM neural networks are developed and compared in this study. The results indicate that the NARX neural network exhibits higher precision with a smaller number of neurons and time steps. The LSTM neural network demonstrates greater stability when dealing with data error fluctuations. In practical applications, the ROL model can monitor the TWC operating status and assist in the development of intelligent pollutant aftertreatment control strategies.

Outlier Reconstruction Based Distribution System State Estimation Using Equivalent Model of Long Short-term Memory and Metropolis-Hastings Sampling

The accuracy of distribution system state estimation(DDSE)is reduced when phasor measurement unit(PMU)measurements contain outliers because of cyber attacks or global positioning system spoofing attacks.Therefore,to enhance the robustness of DDSE to measurement outliers,approximate the target distribution of Metropolis-Hastings(MH)sampling,and judge the prediction of the long short-term memory(LSTM)network,this paper proposes an outlier reconstruction based state estimation method using the equivalent model of the LSTM network and MH sampling(E-LM model),motivated by the characteristics of the chronological correlations of PMU measurements.First,the target distribution of outlier reconstruction is derived using a kernel density estimation function.Subsequently,the reasons and advantages of the E-LM model are explained and analyzed from a mathematical point of view.The proposed LSTM-based MH sampling can approximate the target distribution of MH sampling to decrease the number of the futile iterations.Moreover,the proposed MH-based forecasting of the LSTM can judge each LSTM prediction,which is independent of its true value.Finally,simulations are conducted to evaluate the performance of the E-LM model by integrating the LSTM network and the MH sampling into the outlier reconstruction based DDSE.

A hybrid Wavelet-CNN-LSTM deep learning model for short-term urban water demand forecasting

Short-term water demand forecasting provides guidance on real-time water allocation in the water supply network, which help water utilities reduce energy cost and avoid potential accidents. Although a variety of methods have been proposed to improve forecast accuracy, it is still difficult for statistical models to learn the periodic patterns due to the chaotic nature of the water demand data with high temporal resolution. To overcome this issue from the perspective of improving data predictability, we proposed a hybrid Wavelet-CNN-LSTM model, that combines time-frequency decomposition characteristics of Wavelet Multi-Resolution Analysis (MRA) and implement it into an advanced deep learning model, CNN-LSTM. Four models - ANN, Conv1D, LSTM, GRUN - are used to compare with Wavelet-CNN-LSTM, and the results show that Wavelet-CNN-LSTM outperforms the other models both in single-step and multi-steps prediction. Besides, further mechanistic analysis revealed that MRA produce significant effect on improving model accuracy.

Deep Learning-Based Symbol Detection for Time-Varying Nonstationary Channels

The highly dynamic channel(HDC)in an extremely dynamic environment mainly has fast timevarying nonstationary characteristics.In this article,we focus on the most difficult HDC case,where the channel coherence time is less than the symbol period.To this end,we propose a symbol detector based on a long short-term memory(LSTM)neural network.Taking the sampling sequence of each received symbol as the LSTM unit's input data has the advantage of making full use of received information to obtain better performance.In addition,using the basic expansion model(BEM)as the preprocessing unit significantly reduces the number of neural network parameters.Finally,the simulation part uses the highly dynamic plasma sheath channel(HDPSC)data measured from shock tube experiments.The results show that the proposed BEM-LSTM-based detector has better performance and does not require channel estimation or channel model information.

Measuring moisture content of dead fine fuels based on the fusion of spectrum meteorological data

Dead fine fuel moisture content(DFFMC)is a key factor affecting the spread of forest fires,which plays an important role in evaluation of forest fire risk.In order to achieve high-precision real-time measurement of DFFMC,this study established a long short-term memory(LSTM)network based on particle swarm optimization(PSO)algorithm as a measurement model.A multi-point surface monitoring scheme combining near-infrared measurement method and meteorological measurement method is proposed.The near-infrared spectral information of dead fine fuels and the meteorological factors in the region are processed by data fusion technology to construct a spectral-meteorological data set.The surface fine dead fuel of Mongolian oak(Quercus mongolica Fisch.ex Ledeb.),white birch(Betula platyphylla Suk.),larch(Larix gmelinii(Rupr.)Kuzen.),and Manchurian walnut(Juglans mandshurica Maxim.)in the maoershan experimental forest farm of the Northeast Forestry University were investigated.We used the PSO-LSTM model for moisture content to compare the near-infrared spectroscopy,meteorological,and spectral meteorological fusion methods.The results show that the mean absolute error of the DFFMC of the four stands by spectral meteorological fusion method were 1.1%for Mongolian oak,1.3%for white birch,1.4%for larch,and 1.8%for Manchurian walnut,and these values were lower than those of the near-infrared method and the meteorological method.The spectral meteorological fusion method provides a new way for high-precision measurement of moisture content of fine dead fuel.

Iterative Model Predictive Control for Automatic Carrier Landing of Carrier-Based Aircrafts Under Complex Surroundings and Constraints

This paper considers the automatic carrier landing problem of carrier-based aircrafts subjected to constraints,deck motion,measurement noises,and unknown disturbances.The iterative model predictive control(MPC)strategy with constraints is proposed for automatic landing control of the aircraft.First,the long short-term memory(LSTM)neural network is used to calculate the adaptive reference trajectories of the aircraft.Then the Sage-Husa adaptive Kalman filter and the disturbance observer are introduced to design the composite compensator.Second,an iterative optimization algorithm is presented to fast solve the receding horizon optimal control problem of MPC based on the Lagrange’s theory.Moreover,some sufficient conditions are derived to guarantee the stability of the landing system in a closed loop with the MPC.Finally,the simulation results of F/A-18A aircraft show that compared with the conventional MPC,the presented MPC strategy improves the computational efficiency by nearly 56%and satisfies the control performance requirements of carrier landing.

Velocity forecasts using a combined deep learning model in hybrid electric vehicles with V2V and V2I communication

Vehicle velocity forecast is an important clue in improving the performance of energy management in hybrid electric vehicles(HEV). This paper presents a new combined model for predicting vehicle’s velocity time series. The main features of the model are to combine the feature extraction capability of deep restricted Boltzmann machines(DBM) and sequence pattern predicting capability of bidirectional long short-term memory(BLSTM). Hence, the model is named as DBMBLSTM. In addition, the DRMBLSTM model utilizes the vehicle driving information and roadside infrastructure information provided respectively through vehicle-to-vehicle(V2V) and vehicle-to-infrastructure(V2I) communication channels to predict vehicle velocity at various length of prediction horizon. Furthermore, the predictions results of this study are compared with the state of the art of vehicle velocity forecasts. The root mean square error(RMSE) is used as an evaluation criteria of predictions accuracy. Finally,these compared prediction model are applied in model predictive control(MPC) energy management strategy for the verifications of fuel economy improvement of a HEV. Simulation results confirm that the proposed combined deep learning model performs better than other five prediction methods. Therefore, it is a means of arriving at a reliable forecast model for HEV.

Li-ion battery temperature estimation based on recurrent neural networks

The monitoring of Li-ion battery temperatures is essential to ensure high efficiency and safety.In this work,two types of recurrent neural networks (RNNs),which are long short-term memory-RNN (LSTM-RNN) and gated recurrent unit-RNN(GRU-RNN),are proposed to estimate the surface temperature of 18650 Li-ion batteries during the discharging processes under different ambient temperatures.The datasets acquired from the Prognostics Center of Excellence (PCo E) of NASA are used to train,validate and test the networks.In previous work,temperature has been set as the output of the networks;however,here the temperature difference along the time axis is adopted as the output.The net heat generated results in net temperature change,which is more closely aligned with electrochemical and thermodynamic laws.Extensive simulation results show that the two RNNs can achieve accurate real-time battery temperature estimation.The maximum absolute error in temperature estimation is approximately 0.75°C and the correlation coefficient between the estimated and measured temperature curves is greater than 0.95.The influences of three crucial parameters,which are the number of hidden neurons,initial learning rate and maximum number of iterations,are also assessed in terms of training time,root mean square error and mean absolute error.

An attention-based deep learning model for citywide traffic flow forecasting

Prompt and accurate traffic flow forecasting is a key foundation of urban traffic management.However,the flows in different areas and feature channels(inflow/outflow)may correspond to different degrees of importance in forecasting flows.Many forecasting models inadequately consider this heterogeneity,resulting in decreased predictive accuracy.To overcome this problem,an attention-based hybrid spatiotemporal residual model assisted by spatial and channel information is proposed in this study.By assigning different weights(attention levels)to different regions,the spatial attention module selects relatively important locations from all inputs in the modeling process.Similarly,the channel attention module selects relatively important channels from the multichannel feature map in the modeling process by assigning different weights.The proposed model provides effective selection and attention results for key areas and channels,respectively,during the forecasting process,thereby decreasing the computational overhead and increasing the accuracy.In the case involving Beijing,the proposed model exhibits a 3.7%lower prediction error,and its runtime is 60.9%less the model without attention,indicating that the spatial and channel attention modules are instrumental in increasing the forecasting efficiency.Moreover,in the case involving Shanghai,the proposed model outperforms other models in terms of generalizability and practicality.

Forecast electricity demand in commercial building with machine learning models to enable demand response programs

Electricity load forecasting is an important part of power system dispatching.Accurately forecasting electricity load have great impact on a number of departments in power systems.Compared to electricity load simulation(white-box model),electricity load forecasting(black-box model)does not require expertise in building construction.The development cycle of the electricity load forecasting model is much shorter than the design cycle of the electricity load simulation.Recent developments in machine learning have lead to the creation of models with strong fitting and accuracy to deal with nonlinear characteristics.Based on the real load dataset,this paper evaluates and compares the two mainstream short-term load forecasting techniques.Before the experiment,this paper first enumerates the common methods of short-term load forecasting and explains the principles of Long Short-term Memory Networks(LSTMs)and Support Vector Machines(SVM)used in this paper.Secondly,based on the characteristics of the electricity load dataset,data pre-processing and feature selection takes place.This paper describes the results of a controlled experiment to study the importance of feature selection.The LSTMs model and SVM model are applied to one-hour ahead load forecasting and one-day ahead peak and valley load forecasting.The predictive accuracy of these models are calculated based on the error between the actual and predicted loads,and the runtime of the model is recorded.The results show that the LSTMs model have a higher prediction accuracy when the load data is sufficient.However,the overall performance of the SVM model is better when the load data used to train the model is insufficient and the time cost is prioritized.

Surveillance-image-based outdoor air quality monitoring

Air pollution threatens human health,necessitating effective and convenient air quality monitoring.Recently,there has been a growing interest in using camera images for air quality estimation.However,a major challenge has been nighttime detection due to the limited visibility of nighttime images.Here we present a hybrid deep learning model,capitalizing on the temporal continuity of air quality changes for estimating outdoor air quality from surveillance images.Our model,which integrates a convolutional neural network(CNN)and long short-term memory(LSTM),adeptly captures spatial-temporal image features,enabling air quality estimation at any time of day,including PM_(2.5) and PM10 concentrations,as well as the air quality index(AQI).Compared to independent CNN networks that solely extract spatial features,our model demonstrates superior accuracy on self-constructed datasets with R^(2)?0.94 and RMSE=5.11 mg m^(-3) for PM_(2.5),R^(2)=0.92 and RMSE=7.30 mg m^(-3) for PM10,and R^(2)=0.94 and RMSE?5.38 for AQI.Furthermore,our model excels in daytime air quality estimation and enhances nighttime predictions,elevating overall accuracy.Validation across diverse image datasets and comparative analyses underscore the applicability and superiority of our model,reaffirming its applicability and superiority for air quality monitoring.

Spatiotemporal emotion recognition based on 3D time-frequency domain feature matrix

The research of emotion recognition based on electroencephalogram(EEG)signals often ignores the related information between the brain electrode channels and the contextual emotional information existing in EEG signals,which may contain important characteristics related to emotional states.Aiming at the above defects,a spatiotemporal emotion recognition method based on a 3-dimensional(3 D)time-frequency domain feature matrix was proposed.Specifically,the extracted time-frequency domain EEG features are first expressed as a 3 D matrix format according to the actual position of the cerebral cortex.Then,the input 3 D matrix is processed successively by multivariate convolutional neural network(MVCNN)and long short-term memory(LSTM)to classify the emotional state.Spatiotemporal emotion recognition method is evaluated on the DEAP data set,and achieved accuracy of 87.58%and 88.50%on arousal and valence dimensions respectively in binary classification tasks,as well as obtained accuracy of 84.58%in four class classification tasks.The experimental results show that 3 D matrix representation can represent emotional information more reasonably than two-dimensional(2 D).In addition,MVCNN and LSTM can utilize the spatial information of the electrode channels and the temporal context information of the EEG signal respectively.