利用数据库技术实现的可扩展的分类算法

A Scalable Classification Algorithm Exploring Database Technology

重点研究将数据挖掘中的分类技术与数据库技术紧密结合的高效的可扩展的分类算法.提出一种基于分组记数技术构造分类器的方法,利用数据库系统的结构化查询语言来实现主要计算任务.为了提高算法的执行效率,还提出了优化策略和冗余规则的剪裁策略,并将分类规则的发现过程与相关属性的选择方法有机地结合在一起.使用这些方法和策略,分类算法能够从大规模数据集中快速地发现一组简洁的规则.除了具有与现有分类算法相当的准确度和较高的执行效率以外,该分类算法还具有良好的基于训练集元组个数和属性个数两方面的可扩展性和易于实现的特点.

This paper focuses on the study of efficient and scalable classification algorithm that tightly integrates classification technology with relational database system technology. In this paper, an approach based on grouping and counting is proposed to build classifier, which uses SQL (structured query language) provided by relational database to implement the major computation tasks. In order to improve the performance, several optimization strategies and a redundant rules?pruning strategy together with a feature selection method integrating with the process of finding classification rules are also proposed. With all these methods and strategies, the classification algorithm can find a compact set of classification rules quickly from a large volume of data. In addition to the same classification accuracy with current popular classification algorithms and high training speed, the unique features of the classification algorithm also include its linear scalability with respect to the number of training samples and the number of attributes, and the simplicity in implementation.