基于MEM和HMM的中文词性标注方法

Chinese Part-of-Speech Tagging Method Based on Maximum Entropy Model and Hidden Markov Model

为了进一步提高中文语料库中语料的词性标注效率,在分析最大熵模型(MEM)和隐马尔科夫模型(HMM)所涉及理论、算法及其在中文词性标注技术中的应用的基础上,进行了基于MEM和HMM的中文词性标注实验.实验结果显示,基于MEM和HMM的中文词性标注算法都获得了一致性很好且覆盖率较高的标注效果,中文词性标注的准确率、召回率和F1这3个指标均达到92%以上;MEM的标注效果总体上比HMM的稍佳.

In order to further improve the efficiency of part-of-speech tagging in Chinese corpora,experiments of Chinese part-of-speech tagging methods based on the maximum entropy model(MEM)and the hidden Markov model(HMM)are designed according to the theoretical basis,algorithms,and application technology.The experimental results show that the Chinese part-of-speech tagging algorithms based on MEM and HMM have obtained a very consistent and high-coverage tagging result and the three indicators of tagging accuracy,recall rate and F1 value have reached above 92%,with the effect of MEM better than that of HMM.