基于规则与统计相结合的中文文本自动查错模型与算法

A Hybrid Model of Combining Rule-based and Statistics-based Approaches for Automatic Detecting Errors in Chinese Text

中文文本自动校对是自然语言处理领域具有挑战性的研究课题。本文提出了一种规则与统计相结合的中文文本自动查错模型与算法。根据正确文本分词后单字词的出现规律以及“非多字词错误”的概念,提出一组错误发现规则,并与针对分词后单字散串建立的字二元、三元统计模型和词性二元、三元统计模型相结合,建立了文本自动查错模型与实现算法。通过对30篇含有578个错误测试点的文本进行实验,所提算法的查错召回率为86.85%、准确率为69.43%,误报率为30.57%。

Chinese text automatic proofreading is an important research subjeci in NLP. A hybrid model based on the combination of rules and statistics are proposed in this article. According to the distribution of Chinese single-character after word segmentation in Chinese text and the conception of ＂non-multi-character word error＂, we proposed a group of rules to find errors in texts, to construct the automatic error-detection model and to implement its algorithm by com- bining the scattered single-character Bigram models, part-of-speech Bigram and Trigram models. Our experiment for the 30 texts that contain 578 error test points shows that the recall rate is 86. 85% and accuracy rate is 69. 43%, distorting rate is 30. 57%.