基于比对矩阵模型的信息相似性检测系统设计

Design of information similarity detection system based on comparison matrix model

为解决因数据过量重复而带来的信息网络运行过慢问题,设计基于比对矩阵模型的信息相似性检测系统,进行相似检测相关关键技术与模型算法研究。在总体功能架构中,规范化连接相似性信息捕获模块与检测分发模块,实现信息相似性检测系统硬件运行环境搭建。在此基础上,根据相似性信息数据识别结果,建立全新的比对矩阵模型,并以此为条件,对检测数据库进行定容处理,实现信息相似性检测系统软件运行环境搭建,结合相关硬件执行结构,完成基于比对矩阵模型的信息相似性检测系统设计。对比实验结果表明应用信息相似性检测系统后,单一节点内重复数据总量不超过3.8×10^11T,数据重复频率始终低于40%。

In order to solve the problem that the information network runs too slowly due to excessive data duplication,the information similarity detection system based on the comparison matrix model is designed to study the key techniques and model algorithms related to similar detection. In the overall functional architecture,the normalized connection similarity information capture module and the detection distribution module are implemented,and the hardware operating environment of the information similarity detection system is constructed. On this basis,based on the similarity information data identification results,a new comparison matrix model is established,on this basis,the detection database is processed with constant volume,and the software running environment of the information similarity detection system is built.Combining with the related hardware execution structure,the information similarity detection system based on the comparison matrix model is designed. The comparison experiment results show that after applying the information similarity detection system,the total amount of repeated data in a single node does not exceed 3.8×10^11 T,and the data repetition frequency is always lower than 40%.