基于套索(Lasso)的中文垃圾邮件过滤

Spam Filtering of Chinese Text Email Via Lasso

使用向量空间模型表示的文本邮件数据高维而稀疏,不利于邮件过滤分类模型的建立,通常需在分类器训练前进行维数约减。Lasso回归是一种基于l1正则化的多元线性模型,其在模型参数估计的同时实现了变量选择。提出使用Lasso回归进行垃圾邮件过滤,建立Lasso回归邮件分类模型、Lasso回归词条选择结合逻辑回归的分类模型,结合中文文本垃圾邮件数据集TREC06C进行垃圾邮件过滤实验。实验结果表明Lasso回归词条选择结合逻辑回归的邮件分类模型性能更佳。

Text email data depicted with vector space model are of high dimensionality and sparsity, which are not suitable for establishing email filtering classification model. Generally, such data should be reduced before classifier training. Lasso regression is a multivariate linear model based on l1 regularization, which can estimate model parameters while selecting the variables simultaneously. In this paper, the approaches to email classification based on Lasso are proposed. Also, the Lasso classification model and the logistical model with the selected term are established. Besides, simulation experiments with TREC06 C are carried out, and the results show that logistic regression model plus the term selected with Lasso achieves better performances.