Transformers as Surrogate Models for Genetic Programming in AutoML Tasks

Created by W.Langdon from gp-bibliography.bib Revision:1.8464

@InProceedings{teixeira:2025:GECCO,

• author = "Matheus Candido Teixeira and Gisele Lobo Pappa",
• title = "Transformers as Surrogate Models for Genetic Programming in {AutoML} Tasks",
• booktitle = "Proceedings of the 2025 Genetic and Evolutionary Computation Conference",
• year = "2025",
• editor = "Ryan Urbanowicz and Will N. Browne",
• pages = "472--480",
• address = "Malaga, Spain",
• series = "GECCO '25",
• month = "14-18 " # jul,
• organisation = "SIGEVO",
• publisher = "Association for Computing Machinery",
• publisher_address = "New York, NY, USA",
• keywords = "genetic algorithms, genetic programming, Evolutionary Machine Learning",
• isbn13 = "979-8-4007-1465-8",
• URL = "https://doi.org/10.1145/3712256.3726396",
• DOI = "doi:10.1145/3712256.3726396",
• size = "9 pages",
• abstract = "In applications where the fitness function has a high computational cost, one of the main drawbacks of Evolutionary Algorithms when compared to other search methods is a prohibitive computational cost. The use of surrogates as proxies for fitness function calculation to alleviate this problem is not new, but addressing the problem as a binary relation learning, i.e., evaluating if one individual is better or worse than another without estimating the actual value of the fitness, is a recent trend.This paper proposes a transformer-encoder as a surrogate to evaluate pairs of solutions and determine their relationship, i.e., which one is better/worse than the other. We experimented the model in the context of AutoML, which seeks to find the best combination of algorithms for a classification problem. To optimize the pipeline, we can use a genetic programming, but the cost of evaluating each individual is generally expensive.We trained the encoder with several parameters and compared its performance against traditional GP - evaluating fitness at each generation. Results confirm using the encoder as a surrogate does not degrade the fitness values of the evolved population of ML pipelines and can even improve it in some cases (up to 285 times faster).",
• notes = "GECCO-2025 EML A Recombination of the 34th International Conference on Genetic Algorithms (ICGA) and the 30th Annual Genetic Programming Conference (GP)",

}

Genetic Programming entries for Matheus Candido Teixeira Gisele L Pappa

Citations