回复式神经网络及其应用研究综述

Review of Research on Recurrent Neural Network and Its Applications

回复式神经网络是深度学习领域中一类非常强大的神经网络模型,它主要用于处理和预测序列数据.得益于回复式神经网络的"记忆"功能,它和它的变体网络已经被成功应用于多种任务,比如:机器翻译、语音识别、视频行为识别等.本文首先介绍回复式神经网络的发展历程,之后阐述其网络基本原理、计算过程和网络训练方法.针对传统回复式神经网络存在的短期记忆和梯度消失问题,长短时记忆和门控回复式单元网络结构被详细地分析和对比.然后介绍回复式神经网络的热门应用领域及其相关工作.最后结合近年来工业界和学术界关于回复式神经网络的研究进展,本文总结了回复式神经网络的研究发展趋势.

Recurrent Neural Netw ork(RNN)is a kind of very pow erful neural netw ork model in the field of deep learning,w hich is mainly used to process and predict sequence data.Benefiting from the"M emory"function,RNN and its variant netw orks have been successfully applied to a variety of tasks,such as machine translation,speech recognition,video action recognition,and so on.This paper first introduces the development of RNN,and then elaborates on the basic principles,computational process,and how to train the RNN.In terms of the shortcomings of RNN:short-term memory and gradient vanishing,Long Short-Term M emory(LSTM)and Gated Recurrent Unit(GRU)netw orks are analyzed and compared in detail.Then,the popular research areas and the related research w orks are summarized.Finally,based on the research progress on RNN in industry and academics,this paper discusses the development trend of RNN in the future.