ABSTRACT
Explainable artificial intelligence (XAI) is an important and rapidly expanding research topic. The goal of XAI is to gain trust in a machine learning (ML) model through clear insights into how the model arrives at its predictions. Genetic programming (GP) is often cited as being uniquely well-suited to contribute to XAI because of its capacity to learn (small) symbolic models that have the potential to be interpreted. Nevertheless, like many ML algorithms, GP typically results in a single best model. However, in practice, the best model in terms of training error may well not be the most suitable one as judged by a domain expert for various reasons, including overfitting, multiple different models existing that have similar accuracy and unwanted errors on particular data points due to typical accuracy measures like mean squared error. Hence, to increase chances that domain experts deem a resulting model plausible, it becomes important to be able to explicitly search for multiple, diverse, high-quality models that trade-off different meanings of accuracy. In this paper, we achieve exactly this with a novel multi-modal multi-tree multi-objective GP approach that extends a modern model-based GP algorithm known as GP-GOMEA that is already effective at searching for small expressions.
Supplemental Material
Available for Download
Supplemental material.
- Anton Bouter, Tanja Alderliesten, Cees Witteveen, and Peter A. N. Bosman. 2017. Exploiting Linkage Information in Real-Valued Optimization with the Real-Valued Gene-Pool Optimal Mixing Evolutionary Algorithm. In Proceedings of the Genetic and Evolutionary Computation Conference (Berlin, Germany) (GECCO '17). Association for Computing Machinery, New York, NY, USA, 705--712. Google ScholarDigital Library
- Armand R. Burks and William F. Punch. 2015. An Efficient Structural Diversity Technique for Genetic Programming. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation (Madrid, Spain) (GECCO '15). Association for Computing Machinery, New York, NY, USA, 991--998. Google ScholarDigital Library
- R.B. Chapman. 1972. Hydrodynamic drag of semisubmerged ships. Journal of Basic Engineering 94 (1972), 879--884. Issue 4. Google ScholarCross Ref
- European Commission, Content Directorate-General for Communications Networks, and Technology. 2019. Ethics guidelines for trustworthy AI. Publications Office. Google ScholarCross Ref
- Edwin D. de Jong, Richard A. Watson, and Jordan B. Pollack. 2001. Reducing Bloat and Promoting Diversity Using Multi-Objective Methods. In Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation (San Francisco, California) (GECCO'01). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 11--18.Google Scholar
- K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-IL Transactions on Evolutionary Computation 6, 2 (April 2002), 182--197. Google ScholarDigital Library
- Arkadiy Dushatskiy, Marco Virgolin, Anton Bouter, Dirk Thierens, and Peter AN Bosman. 2021. Parameterless Gene-pool Optimal Mixing Evolutionary Algorithms. arXiv preprint arXiv:2109.05259 (2021).Google Scholar
- Anikó Ekárt and Sandor Z. Németh. 2000. A Metric for Genetic Programs and Fitness Sharing. In Proceedings of the European Conference on Genetic Programming. Springer-Verlag, Berlin, Heidelberg, 259--270.Google ScholarDigital Library
- Benjamin P. Evans, Bing Xue, and Mengjie Zhang. 2019. What's inside the Black-Box? A Genetic Programming Method for Interpreting Complex Machine Learning Models. In Proceedings of the Genetic and Evolutionary Computation Conference (Prague, Czech Republic) (GECCO '19). Association for Computing Machinery, New York, NY, USA, 1012--1020. Google ScholarDigital Library
- Leonardo Augusto Ferreira, Frederico Gadelha Guimaraes, and Rodrigo Silva. 2020. Applying genetic programming to improve interpretability in machine learning models. In 2020 IEEE Congress on Evolutionary Computation (CEC). IEEE, 1--8.Google ScholarDigital Library
- Edgar Galván and Marc Schoenauer. 2019. Promoting Semantic Diversity in Multi-Objective Genetic Programming. In Proceedings of the Genetic and Evolutionary Computation Conference (Prague, Czech Republic) (GECCO '19). Association for Computing Machinery, New York, NY, USA, 1021--1029. Google ScholarDigital Library
- J Gerritsma, R Onnink, and A Versluis. 1981. Geometry, resistance and stability of the Delft systematic yacht hull series. International shipbuilding progress 28, 328 (1981), 276--297.Google Scholar
- Ilan Gronau and Shlomo Moran. 2007. Optimal implementations of UPGMA and other common clustering algorithms. Inform. Process. Lett. 104, 6 (2007), 205--210.Google ScholarDigital Library
- David Harrison and Daniel L Rubinfeld. 1978. Hedonic housing prices and the demand for clean air. Journal of Environmental Economics and Management 5, 1 (1978), 81--102. Google ScholarCross Ref
- Herbert J. Klausmeier. 1992. Concept Learning and Concept Teaching. Educational Psychologist 27, 3 (1992), 267--286. arXiv:https://doi.org/10.1207/s15326985ep2703_1 Google ScholarCross Ref
- William La Cava, Patryk Orzechowski, Bogdan Burlacu, Fabrício Olivetti de Franca, Marco Virgolin, Ying Jin, Michael Kommenda, and Jason H Moore. 2021. Contemporary symbolic regression methods and their relative performance. arXiv preprint arXiv:2107.14351 (2021).Google Scholar
- Ngoc Hoang Luong, Han La Poutré, and Peter A.N. Bosman. 2014. Multi-Objective Gene-Pool Optimal Mixing Evolutionary Algorithms. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (Vancouver, BC, Canada) (GECCO '14). Association for Computing Machinery, New York, NY, USA, 357--364. Google ScholarDigital Library
- R I (Bob) McKay. 2000. Fitness Sharing in Genetic Programming. In Proceedings of the 2nd Annual Conference on Genetic and Evolutionary Computation (Las Vegas, Nevada) (GECCO'00). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 435--442.Google ScholarDigital Library
- Christoph Molnar. 2019. Interpretable Machine Learning. https://chrisrtophm.github.io/interpretable-ml-book/Google Scholar
- Michael D. Schmidt and Hod Lipson. 2010. Age-Fitness Pareto Optimization. In Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation (Portland, Oregon, USA) (GECCO '10). Association for Computing Machinery, New York, NY, USA, 543--544. Google ScholarDigital Library
- Dirk Thierens and Peter A.N. Bosman. 2011. Optimal Mixing Evolutionary Algorithms. In Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (Dublin, Ireland) (GECCO '11). Association for Computing Machinery, New York, NY, USA, 617--624. Google ScholarDigital Library
- Ernest O Tuck. 1987. Wave resistance of thin ships and catamarans. Report T8701. Applied Mathematics Department, The University of Adelaide (1987), 15.Google Scholar
- Marco Virgolin, Tanja Alderliesten, Cees Witteveen, and Peter AN Bosman. 2021. Improving model-based genetic programming for symbolic regression of small expressions. Evolutionary Computation 29, 2 (2021), 211--237.Google ScholarCross Ref
- I.-C. Yeh. 1998. Modeling of strength of high-performance concrete using artificial neural networks. Cement and Concrete Research 28, 12 (1998), 1797--1808. Google ScholarCross Ref
Index Terms
- Multi-modal multi-objective model-based genetic programming to find multiple diverse high-quality models
Recommendations
Ordered Incremental Multi-Objective Problem Solving Based on Genetic Algorithms
Many Multiple Objective Genetic Algorithms (MOGAs) have been designed to solve problems with multiple conflicting objectives. Incremental approach can be used to enhance the performance of various MOGAs, which was developed to evolve each objective ...
Exact Schema Theory and Markov Chain Models for Genetic Programming and Variable-length Genetic Algorithms with Homologous Crossover
Genetic Programming (GP) homologous crossovers are a group of operators, including GP one-point crossover and GP uniform crossover, where the offspring are created preserving the position of the genetic material taken from the parents. In this paper we ...
Multi-objective genetic programming for feature learning in face recognition
AbstractFace recognition is a challenging task due to high variations of pose, expression, ageing, and illumination. As an effective approach to face recognition, feature learning can be formulated as a multi-objective optimisation task of ...
Highlights- A new GP representation to automatically learn effective features for classification
Comments