一种改进的中文分词在主题搜索中的应用

Application of an Improved Chinese Word Segmentation Technology in Topic Search

主题搜索的核心内容是以中文分词为基础的内容匹配,而中文分词的准确性以及对未登记词的识别率问题仍是目前主题搜索的瓶颈.提出了一种改进最大匹配中文分词算法IMMM,通过词库预处理、未登录词处理和歧义消除等策略,并将主题分类和分词词典的存储相结合,构造了一个主题搜索系统.实验证明,改进后的算法较传统的搜索算法在搜索准确率方面有了较大的改进,系统整体搜索效率有明显提高.

The topic search＇ s core is the contents of the match which is based on the Chinese word segmentation,but the Chinese word segmentation＇ s accuracy and unregistered word＇ s recognition is still the bottleneck of the topic search.This paper proposed an improved maximum matching of word segmentation algorithm IMMM.In improved algorithm designed thesaurus pretreatment,unknown word processing and disambiguation strategy,combined subject categories and sub-word dictionary storage,finally construct a topic search system.The algorithm results show that the improved algorithm is better than traditional algorithms,and the search accuracy rate has been greatly improved.The system＇s efficiency is improved.