基于深度学习的藏文分词方法

Tibetan word segmentation based on deep learning

重点研究将深度学习技术应用于藏文分词任务,采用多种深度神经网络模型,包括循环神经网络(RNN)、双向循环神经网络(Bi RNN)、层叠循环神经网络(Stacked RNN)、长短期记忆模型(LSTM)和编码器-标注器长短期记忆模型(Encoder-Labeler LSTM)。多种模型在以法律文本、政府公文、新闻为主的分词语料中进行实验,实验数据表明,编码器-标注器长短期记忆模型得到的分词结果最好,分词准确率可以达到92.96%,召回率为93.30%,F值为93.13%。

The application of deep learning on Tibetan word segmentation was studied.Several models of deep neural network were implemented,including recurrent neural network,bi-directional recurrent neural network,stacked recurrent neural network,long short-term memory network and encoder-labeler long short-term memory network.These models were performed on written style corpus,including legal text,government documents and news.Experimental results show that the encoder-labeler long shortterm memory network achieves the best results,the precision,recall and F value reach 92.96%,93.30% and 93.13% respectively.