基于相似性摘要算法的应用与研究

Application and Research on Similarity Digest Algorithm

算法可以将任意大小的数据转换成一段极短的摘要信息,并通过摘要信息计算两组数据的相似程度,在电子犯罪取证调查、恶意软件检测、垃圾邮件过滤等安全领域极具应用价值。将相似性摘要算法分为基于内容分割的分片哈希算法、特征提取算法以及局部敏感哈希算法等三个类别,分别讨论其特点和适用范围,以及在恶意软件以及文本文件等应用场景中测试相似性检测效果,同时指出现有算法的不足以及研究方向。

The similarity digest algorithm can convert any size data into a very small digest, and calculate the similarity between the two data by comparing their digest. It has great application value in the security field such as electronic crime forensic investigation, malware detection and spare filtering. The similarity digest algorithm is divided into three categories： context triggered piecewise hashing algorithm, feature extraction algorithm and local sensitive hash algorithm. Discusses the characteristics of similarity digest algorithms and the use of malware detection and text comparison, points out the disadvantages of the existing algorithms and the direction for future research.