基于组块分割的无监督藏文句法分析方法研究

Research on Unsupervised Tibetan Syntax Analysis with Chunk Segmentation

由于藏文句法树库的构建在藏语自然语言处理中是一项专业性强及费时费力的工作,目前还没有公开的藏文句法树库,这一现状导致有监督方法在藏文句法分析任务中的运用存在较大困难。鉴于此,提出了一种基于组块分割的无监督藏文句法分析方法,首先对藏文组块进行了定义和分割,然后在此基础上提出无监督藏文句法分析方法并设计了藏文句法优化算法。提出的方法在不同句子长度和不同领域分布的各类测试集上都取得了显著效果,优于基准方法的结果,证实了提出的方法在缺少标记数据的设定下的有效性。

The construction of Tibetan syntax treebank is a laborious task that requires dense expertise.Since there is no public well-labeled dataset of Tibetan syntax treebank,supervised proposal in Tibetan syntax analysis is barely a promising option.To this end,in this work,we proposed an unsupervised Tibetan syntax analysis based on chunk segmentation techniques.We firstly introduced the definition of syntactic chunk for Tibetan and apply chunk-segmentation operations,then proposed an unsupervised Tibetan syntax analysis techniques and algorithm for optimizing Tibetan syntax.The proposed method was evaluated on test datasets with different sentence length and domain distributions and produces promising results,over-passing all other baselines.The results indicate the effectiveness of unsupervised syntax analysis method based on chunk segmentation in the task of Tibetan syntax analysis under settings where labeled data is not available.