摘要
Working as an ensemble method that establishes a committee of classifiers first and then aggregates their out- comes through majority voting, bagging has attracted consid- erable research interest and been applied in various applica- tion domains. It has demonstrated several advantages, but in its present form, bagging has been found to be less accurate than some other ensemble methods. To unlock its power and expand its user base, we propose an approach that improves bagging through the use of multi-algorithm ensembles. In a multi-algorithm ensemble, multiple classification algorithms are employed. Starting from a study of the nature of diver- sity, we show that compared to using different training sets alone, using heterogeneous algorithms together with different training sets increases diversity in ensembles, and hence we provide a fundamental explanation for research utilizing het- erogeneous algorithms. In addition, we partially address the problem of the relationship between diversity and accuracy by providing a non-linear function that describes the relation- ship between diversity and correlation. Furthermore, after re- alizing that the bootstrap procedure is the exclusive source of diversity in bagging, we use heterogeneity as another source of diversity and propose an approach utilizing heterogeneous algorithms in bagging. For evaluation, we consider several benchmark data sets from various application domains. The results indicate that, in terms of Fl-measure, our approach outperforms most of the other state-of-the-art ensemble meth- ods considered in experiments and, in terms of mean margin, our approach is superior to all the others considered in experiments.
Working as an ensemble method that establishes a committee of classifiers first and then aggregates their out- comes through majority voting, bagging has attracted consid- erable research interest and been applied in various applica- tion domains. It has demonstrated several advantages, but in its present form, bagging has been found to be less accurate than some other ensemble methods. To unlock its power and expand its user base, we propose an approach that improves bagging through the use of multi-algorithm ensembles. In a multi-algorithm ensemble, multiple classification algorithms are employed. Starting from a study of the nature of diver- sity, we show that compared to using different training sets alone, using heterogeneous algorithms together with different training sets increases diversity in ensembles, and hence we provide a fundamental explanation for research utilizing het- erogeneous algorithms. In addition, we partially address the problem of the relationship between diversity and accuracy by providing a non-linear function that describes the relation- ship between diversity and correlation. Furthermore, after re- alizing that the bootstrap procedure is the exclusive source of diversity in bagging, we use heterogeneity as another source of diversity and propose an approach utilizing heterogeneous algorithms in bagging. For evaluation, we consider several benchmark data sets from various application domains. The results indicate that, in terms of Fl-measure, our approach outperforms most of the other state-of-the-art ensemble meth- ods considered in experiments and, in terms of mean margin, our approach is superior to all the others considered in experiments.
