基于神经网络的藏文正字检错法

Tibetan Character Error Detection Based on Neural Network

在缺乏标注数据的条件下,该文将藏文正字检错任务视为一个分类问题:首先从语言学知识中构建音节混淆子集并给每个原句加噪,然后建立深层双向表征的BERT作为分类模型,最后为了证明该方法的有效性,构建两个基线模型和三种不同领域的测试集,实验结果表明,该方法的结果优于两个基线模型。该文方法在相同领域测试集上句子分类的正确率达到93.74%,不同领域测试集上也能达到83.6%。对错误音节的识别率为74.53%,同时对无错误音节的误判率只有2.30%。

This paper puts the Tibetan character error detection task as a classification problem.First of all,the syllable confusion subset is built according to the language knowledge and each Tibetan sentence is add with noise.Then a deep bi-direction representation based BERT is applied in the classification model.Two baseline model and test sets of different domains are then constructed.The experimental results show that this method is superior to the two baseline models.The accuracy of sentence classification in the same method can reach 93.74%,and achieve 83.6%in test from different fields.In the syllable level,the performance of true negative s is 74.53%,and false negative is 2.30%.