语义分析在汉语相似性文献检测中的应用研究

Research on the Application of Semantic Analysis in Chinese Document Similarity Detection

相似性文献检测技术在数字图书馆、知识产权保护等应用中有着较强的实际意义和应用价值.汉语中存在着许多歧义现象,导致汉语相似性文献检测的难度远大于英文文献.结合汉语的特点,提出了一种改进的基于词频统计的汉语相似性文献检测方法.首先,利用关键词和统计信息进行切词与消歧.其次,在语义分析的基础上抽取特征词并计算权重.最后,通过构造加权词频向量空间完成相似性检测.对本科生毕业论文的小规模实验表明:该方法能有效消除切分歧义,降低运算规模,提高检测精度与速度.

The technology of Chinese documents similarity detection has a great practical significance and application value in the fields of digital libraries, intellectual property protection and other applications. There are many ambiguities in Chinese. This leads to the fact that Chinese documents similarity detection is more difficult than that of English. With Chinese characteristics, an improved method is provided to detect the similarity of Chinese documents based on word frequency statistics.