邮件网络协同过滤机制研究

Spam Collaborative Filtering in Enron E-mail Network

基于Enron邮件集合探索真实邮件网络,揭示出邮件网络的无标度特性和有限小世界特性.在此基础上,依据用户间交互强度设计出垃圾邮件协同过滤机制,通过调整参数λ,用户可以决定主要是依靠自己还是其他用户协同进行垃圾信息过滤.算法即使在没有对用户个人阅读习惯充分训练的情况下,也可以通过基于交互强度的网络协同方式实现良好过滤.同时为了解决Enron数据集缺乏标注的情况,基于训练样本集W和测试样本集T独立同分布的假设,利用改进的EM(Expectation maximization)算法最小化W∪T集合上风险函数,给出了未知样本的一个良好标注.真实数据上的实验表明,同单机过滤和集成过滤方法相比,协同过滤能够提高平均过滤精度且方法简单易行.

A Social network analysis in Enron corpus found that the real e-mail network was a scale-tree ana small woma in some degree. Then a spam collaborative filtering method was designed based on users＇interaction. By adjusting the parameter λ, users can decide filtering spam by themselves or others or trade-off between them. Even in the absence of reading habits of users, the collaborative filtering method could achieve good performance. Because the Enron corpus was unlabeled, by adding i.i.d, assumption constraint to training data set W and test data set T, we labeled Enron corpus using improved EM （Expectation maximization） algorithm in a sense of minimum statistical risk in W U T. Experiment results showed that the collaborative filtering method is simple and effective which can steadily increase average accuracy compared with single machine and ensemble filterings.