Practical Options for Adopting Recurrent Neural Network and Its Variants on Remaining Useful Life Prediction

The remaining useful life(RUL)of a system is generally predicted by utilising the data collected from the sensors that continuously monitor different indicators.Recently,different deep learning(DL)techniques have been used for RUL prediction and achieved great success.Because the data is often time-sequential,recurrent neural network(RNN)has attracted significant interests due to its efficiency in dealing with such data.This paper systematically reviews RNN and its variants for RUL prediction,with a specific focus on understanding how different components(e.g.,types of optimisers and activation functions)or parameters(e.g.,sequence length,neuron quantities)affect their performance.After that,a case study using the well-studied NASA’s C-MAPSS dataset is presented to quantitatively evaluate the influence of various state-of-the-art RNN structures on the RUL prediction performance.The result suggests that the variant methods usually perform better than the original RNN,and among which,Bi-directional Long Short-Term Memory generally has the best performance in terms of stability,precision and accuracy.Certain model structures may fail to produce valid RUL prediction result due to the gradient vanishing or gradient exploring problem if the parameters are not chosen appropriately.It is concluded that parameter tuning is a crucial step to achieve optimal prediction performance.

基于循环神经网络的电信行业容量数据预测方法

在电信运维的容量预测过程中,存在容量指标和部署业务种类繁多的问题。现有研究未考虑指标数据类型的差异,对所有类型的数据使用同种预测方法,使得预测效果参差不齐。为了提升指标预测效率,提出一种指标数据类型分类方法,利用该方法将数据类型分为趋势型、周期型和不规则型。针对其中的周期型数据预测,提出基于双向循环神经网络(BiRNN)的周期型容量指标预测模型,记作BiRNN-BiLSTM-BI。首先,为分析容量数据的周期特征,提出一种忙闲分布分析算法;其次,搭建循环神经网络(RNN)模型,该模型包含一层BiRNN和一层双向长短时记忆网络(BiLSTM);最后,充分利用系统忙闲分布信息,对BiRNN输出的结果进行优化。与传统的三次指数平滑、差分自回归移动平均(ARIMA)模型和反向传播(BP)神经网络模型进行比较的实验结果表明,在统一日志数据集和分布式缓存数据集上,提出的BiRNN-BiLSTM-BI模型的均方误差(MSE)分别比对比模型中表现最优的模型降低了15.16%和45.67%,可见预测准确率得到了很大程度的提升。

A NOVEL MULTI-VALUED BAM MODEL WITH IMPROVED ERROR-CORRECTING CAPABILITY

A Hyperbolic Tangent multi-valued Bi-directional Associative Memory (HTBAM) model is proposed in this letter. Two general energy functions are defined to prove the stability of one class of multi-valued Bi-directional Associative Mernorys(BAMs), with HTBAM being the special case. Simulation results show that HTBAM has a competitive storage capacity and much more error-correcting capability than other multi-valued BAMs.

Towards Sensor-free Academic Emotion Prediction in Programming Environment

he transition from traditional learning to practice-oriented programming learning will bring learners discomfort.The discomfort quickly breeds negative emotions when encountering programming difficulties,which leads the learner to lose interest in programming or even give up.Emotion plays a crucial role in learning.Educational psychology research shows that positive emotion can promote learning performance,increase learning interest and cultivate creative thinking.Accurate recognition and interpretation of programming learners’emotions can give them feedback in time,and adjust teaching strategies accurately and individually,which is of considerable significance to improve effects of programming learning and education.The existing methods of sensor-free emotion prediction include emotion prediction based on keyboard dynamic,mouse interaction data and interaction logs,respectively.However,none of the three studies considered the temporal characteristics of emotion,resulting in low recognition accuracy.For the first time,this paper proposes an emotion prediction model based on time series and context information.Then,we establish a Bi-recurrent neural network,obtain the time sequence characteristics of data automatically,and explore the application of deep learning in the field of Academic Emotion prediction.The results show that the classification ability of this model is much better than that of the original LSTM(Long-Short Term Memory),GRU(Gate Recurrent Unit)and RNN(Re-current Neural Network),and this model has better generalization ability.

基于深度学习模型的摘要结构功能识别方法研究

【目的/意义】学术文献的摘要由目的、方法、结果等结构组成,这些结构具有特定的功能。目前,针对摘要功能结构识别的研究不多,且存在识别效率不高的问题,本文引入双向循环神经网络(Bidirectional Recurrent Neural Network, BiRNN)、双向长短时记忆网络(Bidirectional Long Short Term Memory, BiLSTM)、BiLSTM-CRF、BERT等深度学习模型,对1232篇情报类期刊论文进行摘要结构功能识别研究。【方法/过程】引入5折交叉验证法进行多次实验,以避免一次实验的偶然性;实验结果用"均值±标准差"形式表示,同时考虑模型的平均性能和稳定性;实验结果用F1值进行评价。【结果/结论】与BiRNN、BiLSTM、BiLSTM-CRF等模型相比,BERT模型具有最高的均值和最低的标准差,这表明该模型不仅具有最优的结构功能识别能力,而且性能稳定,该模型特别适用于摘要结构功能识别任务。【局限/创新】本文采用的实验语料规模较小且为人工标注,这限制了识别效率的提升。