摘要
为了实现维吾尔语文本的相似性检测,提出一种基于N-gram和语义分析的相似性检测方法。根据维吾尔语单词特征,采用了N-gram统计模型来获得词语,并根据词语在文本中的出现频率来构建词语-文本关系矩阵,并作为文本模型。采用了潜在语义分析(LSA)来获得词语及其文本之间的隐藏关联,以此解决维吾尔语词义模糊的问题,并获得准确的相似度。在包含重组和同义词替换的剽窃文本集上进行实验,结果表明该方法能够准确有效地检测出相似性。
In order to detect similarities between Uighur texts, this paper proposed a similarity detection method based on N-gram and semantic analysis. Firstly, it used N-gram statistical model to obtain the words based on Uyghur word features, and constructed the word-text relation matrix according to the appearance frequency of the words in the text. Then, it adopted an LSA to obtain the hidden association between the words and their texts, so as to solve the problem of vague semantic meaning in Uyghur language and obtain exact similarity. Experiments on plagiarized text sets containing reorganization and synonym replacement show that this method can detect the similarity accurately and effectively.
作者
张莹
亚森·艾则孜
吴顺祥
Zhang Ying;Yasen Aizezi;Wu Shunxiang(Dept. of Information Security Engineering,Xinjiang Police College,Urumchi 830011,China;Dept. of Automation,Xiamen University,Xiamen Fujian 361005,China)
出处
《计算机应用研究》
CSCD
北大核心
2019年第9期2722-2725,2729,共5页
Application Research of Computers
基金
国家自然科学基金资助项目(61762086)
新疆维吾尔自治区高校科研计划立项项目(XJEDU2016S090)