基于改进的χ~2检验的热点词突发性度量研究

Bursty Measurement of Hot Term Based on Improvement χ~2 Test Combined with TF

采用原始χ2检验公式进行突发性度量时存在低频词偏袒问题,论文提出了结合TF的改进的χ2检验方法能有效克服该问题。该方法将词频累加和作为文档统计篇数的影响因子β引入原始χ2检验公式从而解决了低频词偏袒问题,提高了度量热点词突发性的精确度。动态突发性热点词库依据改进后的χ2检验公式得到的突发性度量值来建立,并将该词库运用在动态突发性向量空间模型中来发现与追踪网络突发性热点话题。实例验证表明,利用该文的方法进行话题发现与追踪,可以获得有更高的准确率、召回率以及F度量。

Original x2 test formula favors low frequency words when it measures bursty of hot term. To overcome this problem, the im- provedx2 test formula combined with TF is proposed. In this approach, the term frequency summary, an impact factor 13 to the document statistics, is introduced into the original x2 test formula. The experimental results show the dynamic bursty vector space model achieved high- er precision, recall and F-measure in online bursty topic detection and tracking, when dynamic bursty lexicon is constructed according to the bursty measurement using the improved x2 test.