摘要
针对高棉语分词及词性标注问题,提出一种基于层叠条件随机场模型的自动分词及词性标注方法。该方法由三层条件随机场模型构成:第一层是分词模型,该模型以字符簇为粒度,结合上下文信息与高棉语的构词特点构建特征模板,实现对高棉语句子的自动分词;第二层是分词结果修正模型,该模型以词语为粒度,结合上下文信息与高棉语中命名实体的构成特点构建特征模板,实现对第一层分词结果的修正;第三层是词性标注模型,该模型以词语为粒度,结合上下文信息与高棉语丰富的词缀信息构建特征模板,实现对高棉语句子中的词语进行自动标注词性。基于该模型进行开放测试实验,最终准确率为95.44%,结果表明该方法能有效解决高棉语的分词和词性标注问题。
This paper presents a Khmer automatic word segmentation and POS tagging method based on Cascaded Conditional Random Fields(CCRFs)model.The approach consists of three layers of Conditional Random Fields(CRFs)models:the first layer is the word segmentation model in Khmer character cluster(KCC)granularity,integrating the word formation characteristics of Khmer into the feature template;the second layer is the word segmentation correction model in word granularity,integrating the characteristic of Khmer named entities into the feature template;the third layer is the POS tagging model,integrating the rich affixes information into the feature template,and achieved the Khmer POS tagging.We experimented on an open corpus and obtained a final accuracy rate of 95.44%,indicating that the proposed method can effectively solve the Khmer word segmentation and POS tagging problems.
作者
潘华山
严馨
周枫
余正涛
郭剑毅
PAN Huashan YAN Xin ZHOU Feng YU Zhengtao GUO Jianyi(School of Information Engineering and Automation, Kunming University of Science and Technology and Key Lab of Computer Technologies Application of Yunnan Province, Kunming, Yunnan 650500,Chin)
出处
《中文信息学报》
CSCD
北大核心
2016年第4期110-116,共7页
Journal of Chinese Information Processing
基金
国家自然科学基金(61462055)
关键词
高棉语
层叠条件随机场
分词
词性标注
Khmer
cascaded conditional random fields
word segmentation
POS tagging