Skip to main content

Advertisement

Log in

Understanding grammatical evolution: initialisation

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

Grammatical evolution is one of the most used variants of genetic programming, and ever since its introduction, several improvements have been suggested. One of these concerns the routine used to create the initial population. In this study, several proposed initialisation routines are compared; based on a detailed analysis of the generated initial populations, and subsequent results obtained on a large set of experiments, a variant of the PTC2 algorithm is shown to consistently outperform all other routines, while a variant of random initialisation provides a good compromise between efficiency and ease of implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. One could question the analysis of tree structures, given that in these experiments, linear genotypes are generating linear phenotypes, meaning trees are never used during evolution. But they remain an easily understandable and visualised structure, particularly when analysing mathematical expressions. Also, the use of tree measures allows for direct comparison with tree-based GP systems, and derivation-tree based grammatical systems.

  2. The use of a ranking system, rather than just the mean generation at which the reference performance was achieved, is required, as some approaches may not reach that performance even after 100 generations.

References

  1. J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT Press, Cambridge, 1992)

    MATH  Google Scholar 

  2. C. Ryan, J. Collins, M. O’Neill, Grammatical evolution: Evolving programs for an arbitrary language, in Genetic Programming, First European Workshop, EuroGP 1998, Paris, France, April 14–15, 1998, Proceedings, LNCS, vol. 1391, ed. by W. Banzhaf, R. Poli, M. Schoenauer, T.C. Fogarty (Springer, Berlin, 1998), pp. 83–95

    Google Scholar 

  3. R. Harper, A. Blair, A structure preserving crossover in grammatical evolution, in IEEE Congress on Evolutionary Computation, CEC 2005, Edinburgh, UK, September 2–5, 2005, Proceedings, vol. 3, (2005), pp. 2537–2544

  4. J. Byrne, M. O’Neill, J. McDermott, A. Brabazon, An analysis of the behaviour of mutation in grammatical evolution, in Genetic Programming, 13th European Conference, EuroGP 2010, Istanbul, Turkey, April 7–9, 2010, Proceedings, LNCS, vol. 6021, ed. by A.I. Esparcia-Alcázar, A. Ekárt, S. Silva, S. Dignum, A.S. Uyar (Springer, Berlin, 2010), pp. 14–25

    Google Scholar 

  5. C. Ryan, A. Azad, Sensible initialisation in grammatical evolution, in Genetic and Evolutionary Computation—GECCO 2003, Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 12–16, 2004, Workshops, Proceedings, ed. by E. Cantú-Paz, J.A. Foster, K. Deb, L. Davis, R. Roy, U.M. O’Reilly, H.G. Beyer, R.K. Standish, G. Kendall, S.W. Wilson, M. Harman, J. Wegener, D. Dasgupta, M.A. Potter, A.C. Schultz, K.A. Dowsland, N. Jonoska, J.F. Miller (AAAI, 2003)

  6. D. Fagan, M. Fenton, M. O’Neill, Exploring position independent initialisation in grammatical evolution, in IEEE Congress on Evolutionary Computation, CEC 2016, Vancouver, Canada, July 24–29, 2016, Proceedings (2016)

  7. S. Forstenlechner, M. Nicolau, D. Fagan, M. O’Neill, Grammar design for derivation tree based genetic programming systems, in Genetic Programming, 19th European Conference, EuroGP 2016, Porto, Portugal, March 30–April 1, 2016, Proceedings, LNCS, vol. 9594, ed. by M.I. Heywood, J. McDermott, M. Castelli, E. Costa, K. Smith (Springer, Berlin, 2016), pp. 199–214

    Google Scholar 

  8. R. Harper, GE, explosive grammars and the lasting legacy of bad initialisation, in IEEE Congress on Evolutionary Computation, CEC 2010, Barcelona, Spain, July 18–23, 2010, Proceedings (2010), pp. 2602–2609

  9. D.J. Montana, Strongly typed genetic programming. Evolut. Comput. 3(2), 199–230 (1995)

    Google Scholar 

  10. P.A. Whigham, Grammatically-based genetic programming, in Workshop on Genetic Programming: From Theory to Real-World Applications, Tahoe City, CA, USA, July 9, 1995, Proceedings, vol. 414, ed. by J.P. Rosca (Morgan Kaufmann, Burlington, 1995), pp. 33–41

    Google Scholar 

  11. M. O’Neill, C. Ryan, M. Keijzer, M. Cattolico, Crossover in grammatical evolution. Genet. Program. Evolv. Mach. 4(1), 67–93 (2003)

    Article  MATH  Google Scholar 

  12. M. Nicolau, Automatic grammar complexity reduction in grammatical evolution, in Genetic and Evolutionary Computation—GECCO 2004, Genetic and Evolutionary Computation Conference, Seattle, WA, USA, June 26–30, 2004, Workshops, Proceedings, ed. by R. Poli, S. Cagnoni, M. Keijzer, E. Costa, F. Pereira, G. Raidl, S.C. Upton, D.E. Goldberg, H. Lipson, E. de Jong, J.R. Koza, H. Suzuki, H. Sawai, I. Parmee, M. Pelikan, K. Sastry, D. Thierens, W. Stolzmann, P.L. Lanzi, S.W. Wilson, M. O’Neill, C. Ryan, T. Yu, J.F. Miller, I. Garibay, G. Holifield, A.S. Wu, T. Riopka, M.M. Meysenburg, A.W. Wright, N. Richter, J.H. Moore, M.D. Ritchie, L. Davis, R. Roy, M. Jakiela (2004)

  13. M. Nicolau, M. Fenton, Managing repetition in grammar-based genetic programming. In: Genetic and Evolutionary Computation—GECCO 2016, Genetic and Evolutionary Computation Conference, Denver, CO, USA, July 20–24, 2016, Proceedings, ed. by T. Friedrich (ACM, 2016), pp. 765–772

  14. A. Thorhauer, On the non-uniform redundancy in grammatical evolution, in Parallel Problem Solving from Nature—PPSN XIV, 14th International Conference, Edinburgh, UK, September 17–21, 2016, Proceedings, LNCS, vol. 9921, ed. by J. Handl, E. Hart, P.R. Lewis, M. López-Ibáñez, G. Ochoa, B. Paechter (Springer, Berlin, 2016), pp. 292–302

    Google Scholar 

  15. M. Fenton, C. McNally, J. Byrne, E. Hemberg, J. McDermott, M. O’Neill, Discrete planar truss optimization by node position variation using grammatical evolution. IEEE Trans. Evolut. Comput. 20(4), 577–589 (2016)

    Article  Google Scholar 

  16. M. O’Neill, C. Ryan, Grammatical Evolution—Evolutionary Automatic Programming in an Arbitrary Language, Genetic Programming, vol. 4 (Kluwer, Dordrecht, 2003)

    MATH  Google Scholar 

  17. I. Tsoulos, D. Gavrilis, E. Glavas, Neural network construction and training using grammatical evolution. Neurocomputing 72(1–3), 269–277 (2008)

    Article  Google Scholar 

  18. M. Nicolau, M. O’Neill, A. Brabazon, Termination in grammatical evolution: Grammar design, wrapping, and tails, in IEEE Congress on Evolutionary Computation, CEC 2012, Brisbane, Australia, June 10–15, 2012, Proceedings (2012), pp. 1–8

  19. E.F. Crane, N.F. McPhee, The effects of size and depth limits on tree based genetic programming, in Genetic Programming Theory and Practice III, ed. by T. Yu, R. Riolo, B. Worzel (Springer, Boston, 2006), pp. 223–240

    Chapter  Google Scholar 

  20. W.B. Langdon, Size fair and homologous tree crossovers for tree genetic programming. Genet. Program. Evolv. Mach. 1(1), 95–119 (2000)

    Article  MATH  Google Scholar 

  21. T. Soule, J.E. Foster, Code size and depth flows in genetic programming, in Genetic Programming 1997: Second Annual Conference, Stanford, USA, July 13–16, 1997, Proceedings, ed. by J.R. Koza, K. Deb, M. Dorigo, D.B. Fogel, M. Garzon, H. Iba, R.L. Riolo (Morgan Kaufmann, Burlington, 1997), pp. 313–320

    Google Scholar 

  22. S. Luke, Two fast tree-creation algorithms for genetic programming. IEEE Trans. Evolut. Comput. 4(3), 274–283 (2000)

    Article  Google Scholar 

  23. D.R. White, J. McDermott, M. Castelli, L. Manzoni, B.W. Goldman, G. Kronberger, W. Jaśkowski, U.M. O’Reilly, S. Luke, Better GP benchmarks: community survey results and proposals. Genet. Program. Evolv. Mach. 14(1), 3–29 (2013)

    Article  Google Scholar 

  24. E. Hemberg, An exploration of grammars in grammatical evolution. Ph.D. Thesis, University College Dublin, University College Dublin (2010)

  25. M. O’Neill, A. Brabazon, Grammatical swarm, in Genetic and Evolutionary Computation - GECCO 2004, Genetic and Evolutionary Computation Conference, Seattle, WA, USA, June 26–30, 2004, Proceedings, Part I, LNCS, vol. 3102, ed. by K. Deb, R. Poli, W. Banzhaf, H.G. Beyer, E.K. Burke, P.J. Darwen, D. Dasgupta, D. Floreano, J.A. Foster, M. Harman, O. Holland, P.L. Lanzi, L. Spector, A. Tettamanzi, D. Thierens, A.M. Tyrrell (Springer, Berlin, 2004), pp. 163–174

    Google Scholar 

  26. J. O’Sullivan, An investigation into the use of different search engines with grammatical evolution. Master’s Thesis, University of Limerick, University of Limerick (2001)

  27. D. Fagan, M. O’Neill, E. Galván-López, A. Brabazon, S. McGarraghy, An analysis of genotype-phenotype maps in grammatical evolution, in Genetic Programming, 13th European Conference, EuroGP 2010, Istanbul, Turkey, April 7–9, 2010, Proceedings, LNCS, vol. 6021, ed. by A.I. Esparcia-Alcázar, A. Ekárt, S. Silva, S. Dignum, A.S. Uyar (Springer, Berlin, 2010), pp. 62–73

    Google Scholar 

  28. M. Keijzer, M. O’Neill, C. Ryan, M. Cattolico, Grammatical evolution rules: The mod and the bucket rule, in Genetic Programming, 5th European Conference, EuroGP 2002, Kinsale, Ireland, April 3–5, 2002, Proceedings, LNCS, vol. 2278, ed. by J.A. Foster, E. Lutton, J. Miller, C. Ryan, A.G. Tettamanzi (Springer, Berlin, 2002), pp. 123–130

    Google Scholar 

  29. N. Lourenço, F.B. Pereira, E. Costa, Unveiling the properties of structured grammatical evolution. Genet. Program. Evolv. Mach. 17(3), 251–289 (2016)

    Article  Google Scholar 

  30. R.M.A. Azad, A.R. Ansari, C. Ryan, M. Walsh, T. McGloughlin, An evolutionary approach to wall shear stress prediction in a grafted artery. Appl. Soft Comput. 4(2), 139–148 (2004)

    Article  Google Scholar 

  31. A. Brabazon, M. O’Neill, Biologically Inspired Algorithms for Financial Modelling (Springer, Berlin, 2006)

    MATH  Google Scholar 

  32. J. McDermott, J. Byrne, J.M. Swafford, M. Hemberg, C. McNally, E. Shotton, E. Hemberg, M. Fenton, M. O’Neill, String-rewriting grammars for evolutionary architectural design. Environ. Plan. B: Plan. Des. 39(4), 713–731 (2012)

    Article  Google Scholar 

  33. R. Harper, Evolving robocode tanks for evo robocode. Genet. Program. Evolv. Mach. 15(4), 403–431 (2014)

    Article  Google Scholar 

  34. M. Nicolau, D. Perez-Liebana, M. O’Neill, A. Brabazon, Evolutionary behavior tree approaches for navigating platform games. IEEE Trans. Comput. Intell. AI Games 99 (2016). doi:10.1109/TCIAIG.2016.2543661

  35. M. Hemberg, U.M. O’Reilly, Extending grammatical evolution to evolve digital surfaces with genr8, in Genetic Programming, 7th European Conference, EuroGP 2004, Coimbra, Portugal, April 5–7, 2004, Proceedings, LNCS, vol. 3003, ed. by M. Keijzer, U.M. O’Reilly, S.M. Lucas, E. Costa, T. Soule (Springer, Berlin, 2004), pp. 299–308

    Google Scholar 

  36. J.E. Murphy, M. O’Neill, H. Carr, Exploring grammatical evolution for horse gait optimisation, in Genetic Programming, 12th European Conference, EuroGP 2009, Tübingen, Germany, April 15–17, 2009, Proceedings, LNCS, vol. 5481, ed. by L. Vanneschi, S. Gustafson, A. Moraglio, I.D. Falco, M. Ebner (Springer, Berlin, 2009), pp. 183–194

    Google Scholar 

  37. M. Nicolau, M. Saunders, M. O’Neill, B. Osborne, A. Brabazon, Evolving interpolating models of net ecosystem CO2 exchange using grammatical evolution, in Genetic Programming, 15th European Conference, EuroGP 2012, Malaga, Spain, April 11–13, 2012, Proceedings, LNCS, vol. 7244, ed. by A. Moraglio, S. Silva, K. Krawiec, P. Machado, C. Cotta (Springer, Berlin, 2012), pp. 134–145

    Google Scholar 

  38. J. Tavares, F.B. Pereira, Automatic design of ant algorithms with grammatical evolution, in Genetic Programming, 15th European Conference, EuroGP 2012, Malaga, Spain, April 11–13, 2012, Proceedings, LNCS, vol. 7244, ed. by A. Moraglio, S. Silva, K. Krawiec, P. Machado, C. Cotta (Springer, Berlin, 2012), pp. 206–217

    Google Scholar 

  39. C. Ryan, M. Keijzer, M. Nicolau, On the avoidance of fruitless wraps in grammatical evolution, in Genetic and Evolutionary Computation—GECCO 2003, Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 12–16, 2003. Proceedings, Part II, LNCS, vol. 2724, ed. by E. Cantú-Paz, J.A. Foster, K. Deb, L. Davis, R. Roy, U.M. O’Reilly, H.G. Beyer, R.K. Standish, G. Kendall, S.W. Wilson, M. Harman, J. Wegener, D. Dasgupta, M.A. Potter, A.C. Schultz, K.A. Dowsland, N. Jonoska, J.F. Miller (Springer, Berlin, 2003), pp. 1752–1763

    Google Scholar 

  40. F. Rothlauf, M. Oetzel, On the locality of grammatical evolution, in Genetic Programming, 9th European Conference, EuroGP 2006, Budapest, Hungary, April 10–12, 2006, Proceedings, LNCS, vol. 3905, ed. by P. Collet, M. Tomassini, M. Ebner, S. Gustafson, A. Ekárt (Springer, Berlin, 2006), pp. 320–330

    Google Scholar 

  41. A. Thorhauer, F. Rothlauf, On the locality of standard search operators in grammatical evolution, in Parallel Problem Solving from Nature - PPSN XIII, 13th International Conference, Ljubljana, Slovenia, September 13–17, 2014, Proceedings, LNCS, vol. 8672, ed. by T. Bartz-Beielstein, J. Branke, B. Filipič J. Smith (Springer, 2014), pp. 465–475

  42. N. Lourenço, J. Ferrer, F.B. Pereira, E. Costa, A comparative study of different grammar-based genetic programming approaches, in Genetic Programming, 20th European Conference, EuroGP 2017, Amsterdam, The Netherlands, April 19–21, 2017, Proceedings, LNCS, vol. 10196, ed. by J. McDermott, M. Castelli, L. Sekanina, E. Haasdijk, P. García-Sánchez (Springer, Berlin, 2017), pp. 311–325

    Google Scholar 

  43. P.A. Whigham, G. Dick, J. Maclaurin, C.A. Owen, Examining the “best of both worlds” of grammatical evolution, in Genetic and Evolutionary Computation—GECCO 2015, Genetic and Evolutionary Computation Conference, Madrid, Spain, July 11–15, 2015, Proceedings, ed. by S. Silva (ACM, 2015), pp. pp. 1111–1118

  44. A. Brabazon, M. O’Neill, S. McGarraghy, Natural Computing Algorithms (Springer, Berlin, 2015)

    Book  MATH  Google Scholar 

  45. M. Keijzer, Improving symbolic regression with interval arithmetic and linear scaling, in Genetic Programming, 6th European Conference, EuroGP 2003, Essex, UK, April 14–16, 2003, Proceedings, LNCS, vol. 2610, ed. by C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli, E. Costa (Springer, Berlin, 2003), pp. 70–82

    Google Scholar 

  46. L. Pagie, P. Hogeweg, Evolutionary consequences of coevolving targets. Evolut. Comput. 5(4), 401–418 (1997)

    Article  Google Scholar 

  47. E.J. Vladislavleva, G.F. Smits, D. den Hertog, Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans. Evolut. Comput. 13(2), 333–349 (2009)

    Article  Google Scholar 

  48. M.F. Korns, Accuracy in symbolic regression, in Genetic Programming Theory and Practice IX, Genetic and Evolutionary Computation, ed. by R. Riolo, E. Vladislavleva, J.H. Moore (Springer, New York, 2011), pp. 129–151

    Chapter  Google Scholar 

  49. P. Cortez, A. Morais, A data mining approach to predict forest fires using meteorological data, in New Trends in Artificial Intelligence, Portuguese Conference on Artificial Intelligence, EPIA 2007, Guimaraes, Portugal, December 2007, Proceedings, ed. by J. Neves, M.F. Santos, J. Machado (APPIA, New York, 2007), pp. 512–523

    Google Scholar 

  50. M. Lichman, UCI machine learning repository (2013). http://archive.ics.uci.edu/ml

  51. P. Cortez, A. Cerdeira, F. Almeida, T. Matos, J. Reis, Modeling wine preferences by data mining from physicochemical properties. Decis. Support Syst. 47(4), 547–553 (2009)

    Article  Google Scholar 

  52. M. O’Neill, J.M. Swafford, J. McDermott, J. Byrne, A. Brabazon, E. Shotton, C. McNally, M. Hemberg, Shape grammars and grammatical evolution for evolutionary design, in Genetic and Evolutionary Computation—GECCO 2009, Genetic and Evolutionary Computation Conference, Montreal, Canada, July 8–12, 2009, Proceedings, ed. by G.R. et al. (ACN, 2009), pp. 1035–1042

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Miguel Nicolau.

Appendices

Appendix 1: Regression grammars

See Table 8.

Table 8 Regression grammars used, *-g2 versions (*-g1 (through reduction of non-terminal symbols [12]) and *-g3 versions were also used)

Appendix 2: Classification grammars

See Table 9.

Table 9 Classification grammars used, *-g2 versions (*-g1 (through reduction of non-terminal symbols [12]) and *-g3 versions were also used (*-g2 and *-g3 only for BreastCancerW, WineQualityRed and WineQualityWhite problems))

Appendix 3: Shape grammar

See Table 10.

Table 10 Shape grammar used, *-g2 version (*-g3 version was also used)

Appendix 4: Initialiser parameters for regression grammars

See Table 11.

Table 11 Initialisation procedures setup, for regression grammars (genome length (GL), minimum and maximum depth (mD/MD), and minimum and maximum grammar expansions (mE/ME))

Appendix 5: Initialiser parameters for classification grammars

See Table 12.

Table 12 Initialisation procedures setup, for classification grammars BreastCancer and WineQuality problems have no corresponding one terminal symbol grammar (g1)

Appendix 6: Initialiser parameters for shape grammar

See Table 13.

Table 13 Initialisation procedures setup, for shape grammar; no corresponding one terminal symbol grammar (g1)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nicolau, M. Understanding grammatical evolution: initialisation. Genet Program Evolvable Mach 18, 467–507 (2017). https://doi.org/10.1007/s10710-017-9309-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-017-9309-9

Keywords

Navigation