基于决策树的敏感词变形体识别算法研究及应用

Research and application of change form of sensitive words recognition algorithm based on decision tree

针对网络中敏感词变形体识别效率不高的问题,提出了基于决策树的敏感词变形体识别算法。首先,通过分析汉字的结构和读音等特征,研究敏感词及变形体;其次,基于敏感词库构建敏感词决策树;最后,通过多因子改进模型,对微博等新媒体的文本敏感程度进行计算。实验结果表明,该算法在识别中文敏感词及变形体时,查全率和查准率最高分别可达95%和94%,与基于确定有穷自动机的改进算法相比,查全率和查准率分别提高了19.8%和21.1%;与敏感信息决策树信息过滤算法相比,查全率和查准率分别提高17.9%和18.1%。通过分析,该算法对敏感词变形体的识别和自动过滤是有效的。

In order to solve the problem that the recognition efficiency of sensitive word deformed bodies of the network text is not high,this paper proposed a sensitive word deformed bodies recognition algorithm based on decision tree.Firstly,it studied sensitive words and deformed bodies by analyzing the characteristics of Chinese characters and pronunciation and so on.Secondly,it constructed a sensitive word decision tree based on sensitive word library.Finally,it calculated the text sensitivity of new media such as Weibo by multi-factor improved model.The experimental results show that the proposed algorithm can achieve the highest recall rate and precision rate of 95%and 94%respectively when identifying Chinese sensitive words and deformed bodies.Compared with the improved algorithm based on the finite automaton,the recall rate and the precision rate are increased by 19.8%and 21.1%respectively.Compared with the sensitive information decision tree information filtering algorithm,the recall rate and the precision rate are increased by 17.9%and 18.1%respectively.The analysis show that the algorithm is effective in the recognition and automatic filtering of sensitive word deformed bodies.