网页结构相似性的度量是Web信息处理中的一项重要任务,在数据抽取和搜索引擎等研究领域有着潜在的重要研究价值.好的相似性度量方法可以提高数据抽取的准确率和速度,还可以提高搜索引擎的速度,提高返回数据的质量,减少大量冗余数据占据...网页结构相似性的度量是Web信息处理中的一项重要任务,在数据抽取和搜索引擎等研究领域有着潜在的重要研究价值.好的相似性度量方法可以提高数据抽取的准确率和速度,还可以提高搜索引擎的速度,提高返回数据的质量,减少大量冗余数据占据的存储空间.实现该任务的已有算法往往存在着计算复杂度过高的问题,针对这一问题研究HTML文档标签特点,减少传统算法中使用的算子,进而采用简单树匹配算法来计算Web文档之间的结构相似度.简单树匹配不允许结点的替换和跨层匹配,从而大大提高了算法的运行效率.实验结果表明,所提出的方法不论在速度还是精度上都优于著名的Bag of XPaths方法.展开更多
To quickly find documents with high similarity in existing documentation sets, fingerprint group merging retrieval algorithm is proposed to address both sides of the problem:a given similarity threshold could not be t...To quickly find documents with high similarity in existing documentation sets, fingerprint group merging retrieval algorithm is proposed to address both sides of the problem:a given similarity threshold could not be too low and fewer fingerprints could lead to low accuracy. It can be proved that the efficiency of similarity retrieval is improved by fingerprint group merging retrieval algorithm with lower similarity threshold. Experiments with the lower similarity threshold r=0.7 and high fingerprint bits k=400 demonstrate that the CPU time-consuming cost decreases from 1 921 s to 273 s. Theoretical analysis and experimental results verify the effectiveness of this method.展开更多
文摘网页结构相似性的度量是Web信息处理中的一项重要任务,在数据抽取和搜索引擎等研究领域有着潜在的重要研究价值.好的相似性度量方法可以提高数据抽取的准确率和速度,还可以提高搜索引擎的速度,提高返回数据的质量,减少大量冗余数据占据的存储空间.实现该任务的已有算法往往存在着计算复杂度过高的问题,针对这一问题研究HTML文档标签特点,减少传统算法中使用的算子,进而采用简单树匹配算法来计算Web文档之间的结构相似度.简单树匹配不允许结点的替换和跨层匹配,从而大大提高了算法的运行效率.实验结果表明,所提出的方法不论在速度还是精度上都优于著名的Bag of XPaths方法.
基金Project(60873081) supported by the National Natural Science Foundation of ChinaProject(NCET-10-0787) supported by the Program for New Century Excellent Talents in University, ChinaProject(11JJ1012) supported by the Natural Science Foundation of Hunan Province, China
文摘To quickly find documents with high similarity in existing documentation sets, fingerprint group merging retrieval algorithm is proposed to address both sides of the problem:a given similarity threshold could not be too low and fewer fingerprints could lead to low accuracy. It can be proved that the efficiency of similarity retrieval is improved by fingerprint group merging retrieval algorithm with lower similarity threshold. Experiments with the lower similarity threshold r=0.7 and high fingerprint bits k=400 demonstrate that the CPU time-consuming cost decreases from 1 921 s to 273 s. Theoretical analysis and experimental results verify the effectiveness of this method.
