基于改进激活函数和学习率的双向LSTM研究

Research on Bidirectional LSTM Based on Improved Activation Function and Learning Rate

【目的】目前双向长短期记忆网络(Long short term memory,LSTM)在语音识别、图像识别和情感分类等方面的应用越来越广泛,基于此研究如何提高双向LSTM的准确率。【方法】提出一种改进的双向LSTM,通过对LSTM中输入门与输出门激活函数的改进,并结合改进的学习率,能够大大地提高神经网络的收敛速度与准确率。【结果】在不同领域的数据集MNIST和IMDB上分别进行实验,改进的双向LSTM在MNIST数据集上的准确率达到了99.61%,比基准双向LSTM的准确率提高了1.172%,在IMDB数据集上的准确率则达到了88.08%,比基准双向LSTM的准确率提高了1.6%,而且在两个数据集上在迭代次数较小的情况下,相比基准双向LSTM,改进的算法都达到了较高的准确率和较低的损失率。【结论】由此证明改进的双向LSTM优于基准双向LSTM。

[Purposes]With the wide application of bidirectional LSTM recurrent neural networks in speech recognition,image recognition and sentiment classification,it has become especially important to improve the accuracy of bidirectional LSTM recurrent neural networks.[Methods]An improved bidirectional LSTM recurrent neural network is proposed,which greatly improves the convergence speed and accuracy of neural network by improving the input gate and output gate activation functions in LSTM and combining with the improved learning rate.[Findings]Experiments were conducted on different domain data sets MNIST and IMDB,and the accuracy reached 99.61%on the MNIST data set,which is 1.172% higher than that of the benchmark bidirectional LSTM,and 88.08% on the IMDB data set,which is 1.6% higher than that of the benchmark bidirectional LSTM,and on both data sets,higher accuracy and lower loss rate are achieved with smaller number of iterations compared to the benchmark bidirectional LSTM.[Conclusions]This proves that the improved bidirectional LSTM recurrent neural network outperforms the benchmark bidirectional LSTM recurrent neural network.