利用prefix-hash-tree实现从中文文本到事务数据的转换

Transforming Chinese Document into Transaction Data Using Prefix-Hash-Tree

电子文档的飞速增长为自动文本分类提供了巨大的机遇和挑战。在现有的众多方法中,关联分类以其较高的准确率和较快的训练时间而成为一种重要的自动文本分类方法。为实现基于关联的文本分类,首先需要将无结构的文本转换为结构化的事务数据,本文提出的prefix-hash-tree是针对汉语的特殊性而设计的一种数据结构,利用它可以方便地将中文文本转化为事务数据,实验证明利用该数据结构相应的查找、插入和重构算法都具有较好的效率。

The rapid growth in the amount of electronic documents brings both great opportunities and real challenges for automatic text classification. Among many existed approaches, association rule based document classification has aroused great attention as to its high accuracy and fast training time. In this paper, a special data structure called pre- fix-hash-tree is designed to efficiently transform unstructured Chinese text into structured transaction data. Experi- ments confirm that its relevant algorithms show high efficiency.