局部感知递归神经网络在语言模型中的应用

Character-Level language modeling with local-aware recurrent neural network

数据序列预测被许多人认为是机器学习和人工智能领域中的关键问题,在一系列的单词或字符数据处理任务的语言模型中,递归神经网络展示出了当前最优秀的序列预测能力。文中通过将三层RNN按一定层次组合在一起,由低层到高层使每一层负责不同层次的信息处理,从而使新模型具有更强的信息综合能力,从而使得模型更容易处理较长的数据序列。在Penn Treebank Data数据集做字符级(Character-Level)语言模型中的测试结果显示,新模型获得了与CNN-LSTM等当前最好模型相匹敌的成绩。

Data sequence predicting is considered by Scientist as a key problem in machine learning and artificial intelligent. Meanwhile,the recurrent neural network has shown it ＇s state-to-art sequence prediction ability in a series of word or character data sequence processing. In this paper,three layers of recurrent neural network were stacked together as a special hierarchy,from low level to high level,each processing specific level of data sequence, to make a new model that is more powerful on data comprehensive and long sequence processing. On Penn Treebank Data dataset test,the language model achieves the result that is competitive to models as CNN-LSTM＇s.