Evolving Effective Ensembles for Image Classification Using Multi-objective Multi-tree Genetic Programming

Abstract

The high variations across images make image classification a challenging task, where the limited number of training instances further increases the difficulty of achieving good generalization performance. Applying ensemble learning to classification often yields better generalization results on unseen data than using a single classifier. However, for an ensemble to generalize properly, its base learners should be accurate and diverse. Genetic programming (GP) has achieved promising results in image classification. However, existing methods typically employ single-tree representation (i.e., an individual contains a single tree) and are not easy to evolve multiple base learners especially when only limited training data is available. This paper proposes a new ensemble construction method for image classification using multi-objective multi-tree GP (i.e., on individual contains multiple trees). In the new method, a GP individual forms an ensemble, and its multiple trees are base learners that can learn informative features from a relatively small number of training instances. To find effective GP individuals/ensembles, i.e., to make its multiple trees accurate and diverse, the proposed method formulates the ensemble learning problem as a multi-objective task explicitly. Thus, the new objective functions are developed to maximize the diversity and minimize the classification error simultaneously. The proposed method achieves significantly better generalization performance than many competitive methods on four datasets of varying difficulty. Further analysis demonstrates the effectiveness and potentially high interpretability of the constructed ensembles.