摘要
在自然语言处理过程中,词库的大小直接影响着快速输入、信息检索和文本相似度对比等一系列工作的准确性和效率。目前,藏文词库建设主要靠人工扩充方法,未见藏文词库学习算法的研究或文献。文章设计了一种通过遍历藏文文本音节字并识别和统计每个词,将达到阈值的词添加到藏文词库中的算法。经测试,该算法在词库扩充方面具有良好的效果。
In the process of the natural language,the size of thesaurus directly affects the accuracy and efficien?cy of a series of work,such as fast input,information retrieval and text similarity comparison.At present,the Ti?betan lexicon mainly depends on manual construction and expansion,and researches or literatures on Tibetan lexicon learning algorithm were not seen.In this paper,an algorithm is designed to add the words that reach the threshold to the Tibetan lexicon by traversing the syllables of the Tibetan text,recognizing and counting each word.The test results showed that the algorithm has a good effect in the expansion of the vocabulary.
作者
严李强
周建朋
冯郅皓
YAN Li-qiang;ZHOU Jian-peng;FENG Zhi-hao(School of Engineering,Tibet University,Lhasa 850000,China;School of information science and technology,Tibet university,Lhasa 850000,China;Tibet Kumquat intelligent Technology Co.,Ltd.,Lhasa 850000,China)
出处
《高原科学研究》
CSCD
2019年第3期106-112,共7页
Plateau Science Research
基金
国家自然科学基金项目(61561045)
西藏大学青年科研培育基金项目(ZDPJZK1705)
关键词
藏文
词库扩充
自学习
算法
Tibetan
Lexicon Expansion
Self-Learning
Algorithm