面向新媒体领域的错别字自动校对

Automatic Proofreading of Wrong Characters for New Media Field

新媒体平台每天原创新闻发布量巨大,采用人工审核内容中的错别字已经不切实际。本文提出了一种基于n-gram模型与规则相结合的方法,采集上亿篇新闻文章作为训练语料,对分词后的语料进行统计分析形成三元n-gram模型库,基于上下文语境构建错别字混淆集,通过最优化方法计算混淆词在目标场景中的支持度,有效实现错别字的自动检查与纠错。实验结果显示,文章查错召回率为78.9%,准确率为85.1%,具有重要的实际意义和广泛的应用领域。

Every day, a huge amount of original news is released in new media platform, so it is unrealistic to manually check the wrong characters in the audited content. In this paper, a method based on N-gram model and rules is proposed to collect hundreds of millions of news articles as training corpus. The corpus after word segmentation is statistically analyzed to form a ternary N-gram model library. The confusion set is constructed based on context. The support of confusion words in target scene is calculated by optimization method. Automatically checking and correcting errors. The experimental results show that the recall rate of error detection is 78.9%, and the accuracy rate is 85.1%. It has important practical significance and wide application fields.