Skip to main content
SpringerLink
Log in

Parsimony Pressure Made Easy: Solving the Problem of Bloat in GP

  • Chapter
  • First Online:
Book cover Theory and Principled Methods for the Design of Metaheuristics

Part of the book series: Natural Computing Series ((NCS))

Abstract

The parsimony pressure method is perhaps the simplest and most frequently used method to control bloat in genetic programming (GP). In this chapter we first reconsider the size evolution equation for genetic programming developed in Poli and McPhee (Evol Comput 11(2):169–206, 2003) and rewrite it in a form that shows its direct relationship to Price’s theorem. We then use this new formulation to derive theoretical results that show how to practically and optimally set the parsimony coefficient dynamically during a run so as to achieve complete control over the growth of the programs in a population. Experimental results confirm the effectiveness of the method, as we are able to tightly control the average program size under a variety of conditions. These include such unusual cases as dynamically varying target sizes so that the mean program size is allowed to grow during some phases of a run, while being forced to shrink in others.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In a symmetric operator the probability of selecting particular crossover points in the parents does not depend on the order in which the parents are drawn from the population.

  2. 2.

    Naturally, while f p is used to guide evolution, one needs to still use the original fitness function f to recognise solutions and stop runs.

  3. 3.

    We talk about size control pressure rather than parsimony pressure because μ(t) can drift both above and below μ(0).

References

  1. L. Altenberg, The Schema Theorem, Price’s Theorem, in Foundations of Genetic Algorithms 3, Estes Park, ed. by L.D. Whitley, M.D. Vose (Morgan Kaufmann, 1994), pp. 23–49. http://dynamics.org/~altenber/PAPERS/STPT/. Published 1995

  2. P.J. Angeline, An investigation into the sensitivity of genetic programming to the frequency of leaf selection during subtree crossover, in Genetic Programming 1996: Proceedings of the First Annual Conference, Stanford University, ed. by J.R. Koza, D.E. Goldberg, D.B. Fogel, R.L. Riolo (MIT, 1996), pp. 21–29

    Google Scholar 

  3. W. Banzhaf, W.B. Langdon, Some considerations on the reason for bloat. Genet. Program. Evol. Mach. 3(1), 81–91 (2002). doi:10.1023/A:1014548204452. http://web.cs.mun.ca/~banzhaf/papers/genp_bloat.pdf

  4. R. Crawford-Marks, L. Spector, Size control via size fair genetic operators in the PushGP genetic programming system, in GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, New York, ed. by W.B. Langdon, E. Cantú-Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M.A. Potter, A.C. Schultz, J.F. Miller, E. Burke, N. Jonoska (Morgan Kaufmann, 2002), pp. 733–739

    Google Scholar 

  5. S. Dignum, R. Poli, Generalisation of the limiting distribution of program sizes in tree-based genetic programming and analysis of its effects on bloat, in GECCO ’07: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, London, vol. 2, ed. by D. Thierens, H.G. Beyer, J. Bongard, J. Branke, J.A. Clark, D. Cliff, C.B. Congdon, K. Deb, B. Doerr, T. Kovacs, S. Kumar, J.F. Miller, J. Moore, F. Neumann, M. Pelikan, R. Poli, K. Sastry, K.O. Stanley, T. Stützle, R.A. Watson, I. Wegener (ACM, 2007), pp. 1588–1595. http://www.cs.bham.ac.uk/~wbl/biblio/gecco2007/docs/p1588.pdf

  6. S. Dignum, R. Poli, Crossover, sampling, bloat and the harmful effects of size limits, in Proceedings of the 11th European Conference on Genetic Programming, EuroGP 2008, Naples, ed. by M. O’Neill, L. Vanneschi, S. Gustafson, A.I. Esparcia Alcazar, I. De Falco, A. Della Cioppa, E. Tarantino. Lecture Notes in Computer Science, vol. 4971 (Springer, 2008), pp. 158–169. doi:10.1007/978-3-540-78671-9_14

  7. A. Ekart, S.Z. Nemeth, Selection based on the Pareto nondomination criterion for controlling code growth in genetic programming. Genet. Program. Evol. Mach. 2(1), 61–73 (2001). doi:10.1023/A:1010070616149

    Article  MATH  Google Scholar 

  8. H. Iba, Complexity-based fitness evaluation for variable length representation (1997). Position paper at the Workshop on Evolutionary Computation with Variable Size Representation at ICGA-97

    Google Scholar 

  9. H. Iba, H. de Garis, T. Sato, Genetic programming using a minimum description length principle, in Advances in Genetic Programming, Chap. 12, ed. by K.E. Kinnear Jr. (MIT, 1994), pp. 265–284

  10. H. Iba, H. de Garis, T. Sato, Temporal data processing using genetic programming, in Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA95), Pittsburgh, ed. by L. Eshelman (Morgan Kaufmann, 1995), pp. 279–286

    Google Scholar 

  11. K.E. Kinnear Jr., Evolving a sort: lessons in genetic programming, in Proceedings of the 1993 International Conference on Neural Networks, San Francisco, vol. 2 (IEEE, 1993), pp. 881–888. doi:10.1109/ICNN.1993.298674. http://www.cs.ucl.ac.uk/staff/W.Langdon/ftp/ftp.io.com/papers/kinnear.icnn93.ps.Z

  12. K.E. Kinnear Jr., Fitness landscapes and difficulty in genetic programming, in Proceedings of the 1994 IEEE World Conference on Computational Intelligence, Orlando, vol. 1 (IEEE, 1994), pp. 142–147. doi:10.1109/ICEC.1994.350026. http://www.cs.ucl.ac.uk/staff/W.Langdon/ftp/ftp.io.com/papers/kinnear.wcci.ps.Z

  13. M. Kotanchek, G. Smits, E. Vladislavleva, Pursuing the Pareto paradigm tournaments, algorithm variations & ordinal optimization, in Genetic Programming Theory and Practice IV, Ann Arbor, ed. by R.L. Riolo, T. Soule, B. Worzel. Genetic and Evolutionary Computation, vol. 5 (Springer, 2006), pp. 167–186

    Google Scholar 

  14. J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT, Cambridge, 1992)

    MATH  Google Scholar 

  15. W.B. Langdon, The evolution of size in variable length representations, in 1998 IEEE International Conference on Evolutionary Computation, Anchorage, Alaska (IEEE, 1998), pp. 633–638. doi:10.1109/ICEC.1998.700102. http://www.cs.bham.ac.uk/~wbl/ftp/papers/WBL.wcci98_bloat.pdf

  16. W.B. Langdon, Size fair and homologous tree genetic programming crossovers. Genet. Program. Evol. Mach. 1(1/2), 95–119 (2000). doi:10.1023/A:1010024515191. http://www.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/WBL_fairxo.pdf

  17. W.B. Langdon, R. Poli, Fitness causes bloat, in Soft Computing in Engineering Design and Manufacturing, ed. by P.K. Chawdhry, R. Roy, R.K. Pant (Springer, London, 1997), pp. 13–22. http://www.cs.bham.ac.uk/~wbl/ftp/papers/WBL.bloat_wsc2.ps.gz

  18. W.B. Langdon, R. Poli, Foundations of Genetic Programming (Springer, 2002). http://www.cs.ucl.ac.uk/staff/W.Langdon/FOGP/

  19. W.B. Langdon, T. Soule, R. Poli, J.A. Foster, The evolution of size and shape, in Advances in Genetic Programming 3, ed. by L. Spector, W.B. Langdon, U.M. O’Reilly, P.J. Angeline, Chap. 8 (MIT, Cambridge, 1999). pp. 163–190. http://www.cs.bham.ac.uk/~wbl/aigp3/ch08.pdf

  20. S. Luke, Two fast tree-creation algorithms for genetic programming. IEEE Trans. Evol. Comput. 4(3), 274–283 (2000). http://ieeexplore.ieee.org/iel5/4235/18897/00873237.pdf

    Google Scholar 

  21. N.F. McPhee, N.J. Hopper, M.L. Reierson, Sutherland: an extensible object-oriented software framework for evolutionary computation, in Genetic Programming 1998: Proceedings of the Third Annual Conference, University of Wisconsin, Madison, ed. by J.R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D.B. Fogel, M.H. Garzon, D.E. Goldberg, H. Iba, R. Riolo (Morgan Kaufmann, 1998), p. 241. http://www.mrs.umn.edu/~mcphee/Research/Sutherland/sutherland_gp98_announcement.ps.gz

  22. N.F. McPhee, J.D. Miller, Accurate replication in genetic programming, in Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA95), Pittsburgh, ed. by L. Eshelman (Morgan Kaufmann, 1995), pp. 303–309. http://www.mrs.umn.edu/~mcphee/Research/Accurate_replication.ps

  23. N.F. McPhee, R. Poli, Using schema theory to explore interactions of multiple operators, in GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, New York, ed. by W.B. Langdon, E. Cantú-Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K. Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M.A. Potter, A.C. Schultz, J.F. Miller, E. Burke, N. Jonoska (Morgan Kaufmann, 2002), pp. 853–860. http://www.cs.bham.ac.uk/~wbl/biblio/gecco2002/GP139.pdf

  24. R. Poli, General schema theory for genetic programming with subtree-swapping crossover, in Genetic Programming, Proceedings of EuroGP 2001, Lake Como. Lecture Notes in Computer Science (Springer, Milan, 2001)

    Google Scholar 

  25. R. Poli, A simple but theoretically-motivated method to control bloat in genetic programming, in Genetic Programming, Proceedings of the 6th European Conference, EuroGP 2003, Essex, ed. by C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli, E. Costa. Lecture Notes in Computer Science (Springer, 2003), pp. 211–223

    Google Scholar 

  26. R. Poli, W.B. Langdon, S. Dignum, On the limiting distribution of program sizes in tree-based genetic programming, in Proceedings of the 10th European Conference on Genetic Programming, Valencia, vol. 4445, ed. by M. Ebner, M. O’Neill, A. Ekárt, L. Vanneschi, A.I. Esparcia-Alcázar. Lecture Notes in Computer Science (Springer, 2007), pp. 193–204. doi:10.1007/978-3-540-71605-1_18

  27. R. Poli, W.B. Langdon, N.F. McPhee, A field guide to genetic programming. Published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk (2008). (With contributions by J. R. Koza)

  28. R. Poli, N.F. McPhee, General schema theory for genetic programming with subtree-swapping crossover: Part II. Evol. Comput. 11(2), 169–206 (2003). doi:10.1162/106365603766646825. http://cswww.essex.ac.uk/staff/rpoli/papers/ecj2003partII.pdf

  29. G.R. Price, Selection and covariance. Nature 227, August 1, 520–521 (1970). http://www.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/price_nature.pdf

  30. J. Rosca, Generality versus size in genetic programming, in Genetic Programming 1996: Proceedings of the First Annual Conference, Stanford University, ed. by J.R. Koza, D.E. Goldberg, D.B. Fogel, R.L. Riolo (MIT, 1996), pp. 381–387. ftp://ftp.cs.rochester.edu/pub/u/rosca/gp/96.gp.ps.gz

  31. J. Rosca, A probabilistic model of size drift, in Genetic Programming Theory and Practice, Chap. 8, ed. by R.L. Riolo, B. Worzel (Springer, New York, 2003), pp. 119–136

    Google Scholar 

  32. J.P. Rosca, Analysis of complexity drift in genetic programming, in Genetic Programming 1997: Proceedings of the Second Annual Conference, Stanford University, ed. by J.R. Koza, K. Deb, M. Dorigo, D.B. Fogel, M. Garzon, H. Iba, R.L. Riolo (Morgan Kaufmann, 1997), pp. 286–294. ftp://ftp.cs.rochester.edu/pub/u/rosca/gp/97.gp.ps.gz

  33. J.P. Rosca, D.H. Ballard, Complexity drift in evolutionary computation with tree representations. Technical Report NRL5, University of Rochester, Computer Science Department, Rochester, 1996. ftp://ftp.cs.rochester.edu/pub/u/rosca/gp/96.drift.ps.gz

  34. J.P. Rosca, D.H. Ballard, Rooted-tree schemata in genetic programming, in Advances in Genetic Programming 3, Chap. 11, ed. by L. Spector, W.B. Langdon, U.M. O’Reilly, P.J. Angeline (MIT, Cambridge, 1999), pp. 243–271. http://www.cs.bham.ac.uk/~wbl/aigp3/ch11.pdf

  35. J.E. Rowe, N.F. McPhee, The effects of crossover and mutation operators on variable length linear structures, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), San Francisco, ed. by L. Spector, E.D. Goodman, A. Wu, W.B. Langdon, H.M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M.H. Garzon, E. Burke (Morgan Kaufmann, San Francisco, 2001), pp. 535–542. http://www.cs.bham.ac.uk/~wbl/biblio/gecco2001/d03b.pdf

  36. T. Soule, J.A. Foster, Effects of code growth and parsimony pressure on populations in genetic programming. Evol. Comput. 6(4), 293–309 (1998). doi:10.1162/evco.1998.6.4.293

    Article  Google Scholar 

  37. T. Soule, J.A. Foster, Removal bias: a new cause of code growth in tree based evolutionary programming, in 1998 IEEE International Conference on Evolutionary Computation, Anchorage, Alaska (IEEE, 1998), pp. 781–186

    Google Scholar 

  38. B.T. Zhang, H. Mühlenbein, Balancing accuracy and parsimony in genetic programming. Evol. Comput. 3(1), 17–38 (1995). doi:10.1162/evco.1995.3.1.17

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Electronic Engineering, University of Essex, Colchester, Essex, UK

    Riccardo Poli

  2. Division of Science and Mathematics, University of Minnesota, Morris, MN, USA

    Nicholas Freitag McPhee

Authors
  1. Riccardo Poli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicholas Freitag McPhee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Riccardo Poli .

Editor information

Editors and Affiliations

  1. VisualDNA, London, United Kingdom

    Yossi Borenstein

  2. University of Birmingham School of Computer Science, Birmingham, United Kingdom

    Alberto Moraglio

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Poli, R., McPhee, N.F. (2014). Parsimony Pressure Made Easy: Solving the Problem of Bloat in GP. In: Borenstein, Y., Moraglio, A. (eds) Theory and Principled Methods for the Design of Metaheuristics. Natural Computing Series. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33206-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-33206-7_9

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33205-0

  • Online ISBN: 978-3-642-33206-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation