基于短语的贝叶斯中文垃圾邮件过滤方法

Bayesian Chinese Spam Filtering Method Based on Phrases

朴素贝叶斯算法在垃圾邮件过滤领域得到了广泛应用,该算法中,特征提取是一个必不可少的环节。过去针对中文的垃圾邮件过滤方法都以词作为文本的特征项单位进行提取,面对大规模的邮件训练样本,这种算法的时间效率会成为邮件过滤技术中的一个瓶颈。对此,提出一种基于短语的贝叶斯中文垃圾邮件过滤方法,在特征项提取阶段结合文本分类领域提出的新的短语分析方法,按照基本名词短语、基本动词短语、基本语义分析规则,以短语为单位进行提取。通过分别以词和短语为单位进行垃圾邮件过滤的对比测试实验证实了所提出方法的有效性。

Naive Bayesian has been widely used in the field of spam filtering, in which the feature extraction is one of the essential links in the algorithm. In the past, only words were used as text features for the extraction in the method of Chinese spare filtering. In face of large-scale email training samples, time efficiency of this algorithm will become a bot- tleneck of spare filtering technology. A Bayesian spare filtering algorithm based on phrases was proposed here which combines a new phrase analysis method put forward in text classification field. Phrases are extracted as the unit accor- ding to the rules of basic noun phrases, verb phrases and semantic analysis. Through comparison test experiment of spare filtering based on words and phrases as unit, the effectiveness of the proposed method was confirmed.