Measuring Structural Complexity of GP Models for Feature Engineering over the Generations
Created by W.Langdon from
gp-bibliography.bib Revision:1.8010
- @InProceedings{batista:2024:CEC2,
-
author = "Joao Eduardo Batista and Adam Kotaro Pindur and
Hitoshi Iba and Sara Silva",
-
title = "Measuring Structural Complexity of {GP} Models for
Feature Engineering over the Generations",
-
booktitle = "2024 IEEE Congress on Evolutionary Computation (CEC)",
-
year = "2024",
-
editor = "Bing Xue",
-
address = "Yokohama, Japan",
-
month = "30 " # jun # " - 5 " # jul,
-
publisher = "IEEE",
-
keywords = "genetic algorithms, genetic programming, Measurement,
Analytical models, Computational modeling, Pipelines,
Predictive models, Prediction algorithms, Complexity
theory, Model Complexity, Feature Engineering, Model
Interpretability, Classification",
-
isbn13 = "979-8-3503-0837-2",
-
DOI = "doi:10.1109/CEC60901.2024.10611989",
-
abstract = "Feature engineering is a necessary step in the machine
learning pipeline. Together with other preprocessing
methods, it allows the conversion of raw data into a
dataset containing only the necessary features to solve
the task at hand, reducing the computational complexity
of inducing models and creating models that are
potentially simpler, more robust, and more
interpretable. We use M3GP, a wrapper-based feature
engineering algorithm, to induce a set of features that
are adapted in number and in shape to several
classifiers with different levels of predictive power,
from decision trees with depth 3 to random forests with
100 estimators and no depth limit. Intuition tells us
that classifiers that are restricted in the number of
features should compensate for this restriction by
using features with a high degree of correlation with
the target objective. By opposition, the principle
behind the boosting algorithm tells us that we can
create a strong classifier using a large set of weak
features. This indicates that classifiers with no
restrictions should prefer many but weaker features.
Our results confirm this hypothesis while also
revealing that M3GP induces unnecessarily complex
features. We measure complexity using several
structural complexity metrics found in the literature
and show that, although our pipeline consistently
obtains good results, the structural complexity of the
induced models varies drastically across runs.
Additionally, while the test performance peaks in the
early stages of the evolution, the complexity of the
feature engineering models continues to grow, with
little to no return in test performance. This work
promotes using several complexity metrics to measure
model interpretability and identifies issues related to
model complexity in M3GP, proposing solutions to
improve the computational cost of inducing models and
the complexity of the final models.",
-
notes = "also known as \cite{10611989}
WCCI 2024",
- }
Genetic Programming entries for
Joao Eduardo Batista
Adam Kotaro Pindur
Hitoshi Iba
Sara Silva
Citations