Abstract
This work is inspired by the idea of seeding Genetic Programming (GP) populations with trained models from a pool of different Machine Learning (ML) methods, instead of using randomly generated individuals. If one considers standard GP, tackling this problem is very challenging, because each ML method uses its own representation, typically very different from the others. However, the task becomes easier if we use Geometric Semantic GP (GSGP). In fact, GSGP allows us to abstract from the representation, focusing purely on semantics. Following this idea, we introduce EGSGP, a novel method that can be seen either as a new initialization technique for GSGP, or as an ensemble method, that uses GSGP to combine different Base Learners (BLs). To counteract overfitting, we focused on the study of elitism and Soft Target (ST) regularization, studying several variants of EGSGP. In particular, systems that use or do not use elitism, and that use (with different parameters) or do not use ST were investigated. After an intensive study of the new parameters that characterize EGSGP, those variants were compared with the used BLs and with GSGP on three real-life regression problems. The presented results indicate that EGSGP outperforms the BLs and GSGP on all the studied test problems. While the difference between EGSGP and GSGP is statistically significant on two of the three test problems, EGSGP outperforms all the BLs in a statistically significant way only on one of them.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Zhou, Z.H.: Ensemble Methods: Foundations and Algorithms. Chapman and Hall/CRC, Boca Raton (2019)
Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45014-9_1
Thrun, S., Pratt, L.: Learning to learn: introduction and overview. In: Learning to Learn, pp. 3–17. Springer, Heidelberg (1998). https://doi.org/10.1007/978-1-4615-5529-2_1
Vanschoren, J.: Meta-learning. In: Hutter, F., Kotthoff, L., Vanschoren, J. (eds.) Automated Machine Learning. TSSCML, pp. 35–61. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-05318-5_2
Rokach, L.: Pattern classification using ensemble methods, vol. 75. World Scientific (2010)
Moraglio, A., Krawiec, K., Johnson, C.G.: Geometric semantic genetic programming. In: Coello, C.A.C., Cutello, V., Deb, K., Forrest, S., Nicosia, G., Pavone, M. (eds.) PPSN 2012. LNCS, vol. 7491, pp. 21–31. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32937-1_3
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection, vol. 1. MIT press, Cambridge (1992)
Bakurov, I., Vanneschi, L., Castelli, M., Fontanella, F.: EDDA-V2 – an improvement of the evolutionary demes despeciation algorithm. In: Auger, A., Fonseca, C.M., Lourenço, N., Machado, P., Paquete, L., Whitley, D. (eds.) PPSN 2018. LNCS, vol. 11101, pp. 185–196. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99253-2_15
Vanneschi, L., Castelli, M., Silva, S.: A survey of semantic methods in genetic programming. Genet. Program. Evol. Mach. 15(2), 195–214 (2014)
Vanneschi, L.: An introduction to geometric semantic genetic programming. In: Schütze, O., Trujillo, L., Legrand, P., Maldonado, Y. (eds.) NEO 2015. SCI, vol. 663, pp. 3–42. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-44003-3_1
Pawlak, T.P., Krawiec, K.: Semantic geometric initialization. In: Heywood, M.I., McDermott, J., Castelli, M., Costa, E., Sim, K. (eds.) EuroGP 2016. LNCS, vol. 9594, pp. 261–277. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30668-1_17
Bakurov, I.: An initialization technique for geometric semantic genetic programming based on demes evolution and despeciation: machine learning for rare diseases: a case study. PhD thesis, NOVA IMS (2018)
Fan, D.W., Chan, P.K., Stolfo, S.J.: A comparative evaluation of combiner and stacked generalization. In: Proceedings of AAAI-96 Workshop on Integrating Multiple Learned Models, pp. 40–46 (1996)
Aghajanyan, A.: Soft target regularization: an effective technique to reduce over-fitting in neural networks. In: 2017 3rd IEEE International Conference on Cybernetics (CYBCONF), pp. 1–5. IEEE (2017)
Vanneschi, L., Castelli, M.: Soft target and functional complexity reduction: a hybrid regularization method for genetic programming. Expert Syst. Appl. 177, 114929 (2021)
Castelli, M., Vanneschi, L., Silva, S.: Prediction of high performance concrete strength using genetic programming with geometric semantic genetic operators. Expert Syst. Appl. 40(17), 6856–6862 (2013)
Vanneschi, L.: Improving genetic programming for the prediction of pharmacokinetic parameters. Memetic Comput. 6(4), 255–262 (2014)
Castelli, M., Vanneschi, L., Silva, S.: Prediction of the unified parkinson’s disease rating scale assessment using a genetic programming system with geometric semantic genetic operators. Expert Syst. Appl. 41(10), 4608–4616 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rosenfeld, L., Vanneschi, L. (2023). EGSGP: An Ensemble System Based on Geometric Semantic Genetic Programming. In: De Stefano, C., Fontanella, F., Vanneschi, L. (eds) Artificial Life and Evolutionary Computation. WIVACE 2022. Communications in Computer and Information Science, vol 1780. Springer, Cham. https://doi.org/10.1007/978-3-031-31183-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-31183-3_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31182-6
Online ISBN: 978-3-031-31183-3
eBook Packages: Computer ScienceComputer Science (R0)