基于KNN的不良文本过滤方法

Reactionary Text Filtering Method Based on K-Nearest Neighbor

不良文本过滤是当前的一个研究热点。通过对χ2统计量的具体分析,证明χ2统计量在2类文本特征项提取过程中特有的优势。提出正面文本阈值δ,并从理论上推断出该值的大小。在此基础上改进KNN算法,消除了KNN算法中N的不确定性,彻底实现了无参性,大幅减少了分类所用的时间。实验证明,该算法符合Web实时在线分类的要求。

Reactionary text filtering is a hot research now.This paper proves that statistics χ2 has the unique advantages in the features extraction of the two types of texts based on statistics χ2 analysis.It proposes the threshold δ of the positive texts and infers the value of it in theory,and the K-Nearest Neighbor（KNN） algorithm is improved.This algorithm eliminates the uncertainty of KNN algorithm,realizes no reference,and reduces the time used in the text categorization.Experimental results show that the algorithm meets the real-time online text categorization.