摘要
针对条件随机场分词不具有良好的领域自适应性,提出一种条件随机场与领域词典相结合的方法提高领域自适应性,并根据构词规则提出了固定词串消解,动词消解,词概率消解三种方法消除歧义。实验结果表明,该分词流程和方法,提高了分词的准确率和自适应性,在计算机领域和医学领域的分词结果 F值分别提升了7.6%和8.7%。
According to the Conditional Random Field for Chinese word segmentation, the field is hard to adaptive. Acombination of CRF and domain dictionary is proposed to improve the field adaptability, and for eliminating ambiguity,this paper uses fixed word collocation, verb dictionary and word probability by the rule of word formation. The experientalresults show that this approach improves the accuracy and adaptability of the word segmentation. F value of the segmentationresults in computer and medical fields is increased by 7.6% and 8.7%.
作者
朱艳辉
刘璟
徐叶强
田海龙
马进
ZHU Yanhui;LIU Jing;XU Yeqiang;TIAN Hailong;MA Jin(School of Computer and Communication, Hunan University of Technology, Zhuzhou, Hunan 412007, China)
出处
《计算机工程与应用》
CSCD
北大核心
2016年第15期97-100,共4页
Computer Engineering and Applications
基金
国家自然科学基金(No.61170102)
国家社科基金资助项目(No.12BYY045)
湖南省教育厅重点项目(No.15A049)
关键词
中文分词
条件随机场
领域自适应
歧义消解
领域分词
逆向最大匹配算法
Chinese word segmentation
Conditional Random Field(CRF)
domain adaption
ambiguity resolution
domain segmentation
reverse directional maximum match method