基于层叠条件随机场的哈语树库构建技术研究

RESEARCH ON THE TECHNOLOGY OF BUILDING KAZAKH TREEBANK BASED ON CASCADED CONDITIONAL RANDOM FIELD

针对如何提高基于统计的哈萨克语句法分析算法的处理性能问题,提出一种通过人机交互来构建哈萨克语树库的方法。在自动句法标注阶段,采用层叠条件随机场模型实现,并在其低层与高层模型之间加入改进的基于转换的错误驱动学习算法来进行简单句的自动句法标注及自动校正。最后对特殊的整体标记错误进行人工校对,形成基于短语结构的哈萨克语树库。实验结果表明,该方法在很大程度上减少了人力及物力的投入,提高了分析精度及整体处理效率,并为后期基于哈萨克语的句法机器翻译及文本挖掘奠定了一定的基础。

On the issue of how to improve the processing performance of statistical analysis-based Kazakh syntax parsing algorithm,this paper proposes a method of constructing the Kazakh treebank by human-computer interaction. In automatic syntax annotation stage,it achieves by using the cascade conditional random field model. And between its low-level and high-level models it adds the improved and transformation-based error-driven learning algorithm to carry out automatic syntax annotation and automatic correction of the simple sentences.Finally for special entire marking errors the artificial proofreading will be conducted,thus the method forms the phrase structure-based Kazakh treebank. Experimental results show that this method reduces to a large extent the investment on human power and material resources,improves the parsing accuracy and overall processing efficiency. Moreover,it lays the certain foundation for the Kazakh-based syntactic machine translation and text mining afterwards.