基于NER和TF/IDF算法的涉密文件的脱密鉴别方法研究

Research on depassword authentication method of confidential files based on NER and TF/IDF algorithm

随着网络以及各类应用的发展,涉密材料往往在网络渠道传播、流传过程中被泄露,而且往往是涉密材料已经传播一定的范围之后,才能引起相关部门的注意而采用相应的措施。在公开网络产生泄密事件处理过程中,最重要的因素是在泄密材料出现于公开网络并广泛传播之前,发现并确定传播材料是否涉密、可能的来源以及相应的密级。文章提出一种基于NER和TF/IDF算法的涉密文件的脱密鉴别方法,利用NER和TF/IDF算法识别涉密文件中的命名实体,构成矩阵,经过HASH脱密,发送统一中心平台存储。鉴别时,取出文件中各命名实体对应的TF/IDF值乘以出现的次数并求和,其值跟预先设定的阈值比较,以此来确定是否为涉密文件,从而进一步判断涉密文件的来源。

With the development of network and various applications, classified materials are often leaked in the process of dissemination and circulation through network channels, and it is often only after the classified materials have spread to a certain extent that the relevant departments can pay attention to them and adopt corresponding measures. The most important factor in the process of dealing with leaks generated by open networks is to discover and determine whether the disseminated materials are classified, their possible sources, and the corresponding classification levels before the leaked materials appear on open networks and are widely disseminated. In this paper, we propose a declassification identification method based on NER and TF/IDF algorithms to identify the named entities in classified documents using NER and TF/IDF algorithms, constitute a matrix, declassify them by HASH, and send them to the unified central platform for storage. The TF/IDF value corresponding to each named entity in the file is multiplied by the number of occurrences and summed up, and the value is compared with the pre-set threshold value to determine whether the file is classified or not, so as to further determine the origin of the classified file.