递归神经网络研究综述

Research Review of Recurrent Neural Networks

递归神经网络(RNN)因具存储特性,可以处理前后输入有关系的序列数据,故广泛应用于文本音频、视频等领域.当输入间隙较大时,RNN存在短期记忆问题,无法处理很长的输入序列,而长短期记忆(LSTM)能很好地处理长期依赖性问题.自LSTM提出以来,几乎所有基于RNNs的令人兴奋的结果都是由LSTM实现的,因此LSTM成为深度学习的焦点.综述首先简述了RNN以及LSTM及其几种变体的基本工作原理及特点,然后介绍了RNN和LSTM在某些领域中的应用,最后分析并总结了RNN未来的发展方向.

Recurrent neural network(RNN)is a kind of neural network with feedback connection in each layer.Because of its storage characteristics,it can process the sequence data which is related before and after input,and can be widely used in the field of text audio,video and so on.But when the input gap is large,RNN has a short-term memory problem,which can not process long input sequences,while long short-term memory(LSTM)can deal with the long-term dependence problem well.Almost all the exciting results based on RNNs have been realized by LSTM since LSTM was proposed,so LSTM has become the focus of deep learning.This review firstly introduces the basic working principle and characteristics of RNN,and then it introduces the principle and characteristics of LSTM and its variants,as well as the application of RNN and LSTM in various fields.Finally,the future research direction of RNN is proposed.