Skip to main content
Log in

Exact Schema Theory for Genetic Programming and Variable-Length Genetic Algorithms with One-Point Crossover

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

A few schema theorems for genetic programming (GP) have been proposed in the literature in the last few years. Since they consider schema survival and disruption only, they can only provide a lower bound for the expected value of the number of instances of a given schema at the next generation rather than an exact value. This paper presents theoretical results for GP with one-point crossover which overcome this problem. First, we give an exact formulation for the expected number of instances of a schema at the next generation in terms of microscopic quantities. Due to this formulation we are then able to provide an improved version of an earlier GP schema theorem in which some (but not all) schema creation events are accounted for. Then, we extend this result to obtain an exact formulation in terms of macroscopic quantities which makes all the mechanisms of schema creation explicit. This theorem allows the exact formulation of the notion of effective fitness in GP and opens the way to future work on GP convergence, population sizing, operator biases, and bloat, to mention only some of the possibilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. L. Altenberg, “Emergent phenomena in genetic programming,” in Evolutionary Programming-Proc. Third Ann. Conf., A. V. Sebald and L. J. Fogel (eds.), World Scientific Publishing: Singapore, 1994, pp. 233–241.

    Google Scholar 

  2. L. Altenberg, “The schema theorem and Price's theorem,” in Foundations of Genetic Algorithms, L. D. Whitley and M. D. Vose (eds.), Estes Park, Colorado, USA, 31 July-2 Aug, 1994. Morgan Kaufmann: San Francisco, 1995, vol. 3, pp. 23–49.

    Google Scholar 

  3. T. Bäck and D. B. Fogel, “Glossary,” in Evolutionary Computation 1: Basic Algorithms and Operators, T. Bäck, D. B. Fogel, and T. Michalewicz, (eds.), Institute of Physics Publishing: Bristol and Philadelphia, 2000.

    Google Scholar 

  4. W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic Programming-An Introduction; On the Automatic Evolution of Computer Programs and its Applications, Morgan Kaufmann: San Francisco, 1998.

    Google Scholar 

  5. S. W. Chung and R. A. Perez, “The schema theorem considered insufficient,” in Proc. Sixth IEEE Int. Conf. Tools with Artif. Intell., New Orleans, IEEE Press: Piscataway, NJ, Nov 6–9 1994, pp. 748–751.

    Google Scholar 

  6. T. E. Davis and J. C. Principe, “AMarkov chain framework for the simple genetic algorithm,” Evol. Comput., vol. 1(3) pp. 269–288, 1993.

    Google Scholar 

  7. K. A. De Jong, W. M. Spears, and D. F. Gordon, “Using Markov chains to analyze GAFOs,” in Foundations of Genetic Algorithms, L. D. Whitley and M. D. Vose (eds.), Morgan Kaufmann: San Francisco, CA, 1995, vol. 3, pp. 115–137.

    Google Scholar 

  8. D. B. Fogel and A. Ghozeil, “Schema processing under proportional selection in the presence of random effects,” IEEE Trans. Evol. Comput., vol. 1(4), pp. 290–293, 1997.

    Google Scholar 

  9. D. B. Fogel and A. Ghozeil, “The schema theorem and the misallocation of trials in the presence of stochastic effects,” in Evolutionary Programming VII: Proc. of the 7th Ann. Conf. on Evolutionary Programming, V. W. Porto, N. Saravanan, D. Waagen, and A. E. Eiben (eds.), Springer: Berlin, 1998, pp. 313–321.

    Google Scholar 

  10. D. E. Goldberg. Genetic algorithms and Walsh fuctions: II. Deception and its analysis. Complex Systems, vol. 3(2) pp. 153–171, 1989.

    Google Scholar 

  11. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley: Reading, MA, 1989.

    Google Scholar 

  12. J. J. Grefenstette, “Deception considered harmful,” in Foundations of Genetic Algorithms, L. D. Whitley (ed.), San Mateo, CA, Morgan Kaufman: San Francisco, 1993, vol. 2.

    Google Scholar 

  13. J. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press: Ann Arbor, USA, 1975.

    Google Scholar 

  14. J. H. Holland, “Building blocks, cohort genetic algorithms, and hyperplane-defined functions,” Evol. Comput., vol. 8(4) pp. 373–391, 2000.

    Google Scholar 

  15. J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press: Cambridge, MA, 1992.

    Google Scholar 

  16. W. B. Langdon, “Size fair and homologous tree genetic programming crossovers,” in Proc. Genetic and Evolutionary Computation Conference, W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (eds.), Orlando, FL, USA, Morgan Kaufmann: San Francisco, July 13–17 1999, vol. 2, pp. 1092–1097.

    Google Scholar 

  17. W. B. Langdon, “Size fair and homologous tree genetic programming crossovers,” Genet. Program. Evol. Mach., vol. 1(1/2) pp. 95–119, 2000.

    Google Scholar 

  18. W. B. Langdon and R. Poli, “Why ants are hard,” in Genetic Programming 1998: Proc. Third Ann. Conf., J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), University of Wisconsin, Madison, Wisconsin, USA, Morgan Kaufmann: San Francisco, July 22–25 1998, pp. 193–201.

    Google Scholar 

  19. N. F. McPhee and R. Poli, “Aschema theory analysis of the evolution of size in genetic programming with linear representations,” in Genetic Programming, Proceedings of EuroGP 2001, LNCS, Milan, Springer-Verlag: Berlin, April 18–20 2001, vol. 2038, pp. 108–125.

    Google Scholar 

  20. N. F. McPhee, R. Poli, and J. E. Rowe, “Aschema theory analysis of mutation size biases in genetic programming with linear representations,” Technical Report CSRP–00–24, University of Birmingham, School of Computer Science, December 2000.

  21. A. E. Nix and M. D. Vose, “Modeling genetic algorithms with Markov chains,” Ann. Math. Artif. Intell., vol. 5 pp. 79–88, 1992.

    Google Scholar 

  22. P. Nordin and W. Banzhaf, “Complexity compression and evolution,” in Genetic Algorithms: Proc. Sixth Int. Conf. (ICGA95), L. Eshelman (ed.), Pittsburgh, PA, USA, Morgan Kaufmann: San Francisco, July 15–19 1995, pp. 310–317.

    Google Scholar 

  23. P. Nordin, F. Francone, and W. Banzhaf, “Explicitly defined introns and destructive crossover in genetic programming,” in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, J. P. Rosca (ed.), Tahoe City, California, USA, July 9 1995, pp. 6–22.

  24. U.-M. O'Reilly and F. Oppacher, “The troubling aspects of a building block hypothesis for genetic programming,” in Foundations of Genetic Algorithms, L. D. Whitley and M. D. Vose (eds.), Estes Park, Colorado, USA, Morgan Kaufmann: San Francisco, 31 July-2 August 1994, 1995, vol. 3, pp. 73–88.

    Google Scholar 

  25. J. Page, R. Poli, and W. B. Langdon, “Smooth uniform crossover with smooth point mutation in genetic programming: Apreliminary study,” in Genetic Programming, Proceedings of EuroGP'99, R. Poli, P. Nordin, W. B. Langdon, and T. C. Fogarty (eds.), volume 1598 of LNCS, Goteborg, Sweden, Springer-Verlag: Berlin, May 26–27 1999, pp. 39–49.

    Google Scholar 

  26. R. Poli, “Schema theorems without expectations,” in Proceedings of the Genetic and Evolutionary Computation Conference, W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (eds.), Orlando, FL, USA, Morgan Kaufmann: San Francisco, July 13–17 1999, vol. 1, p. 806.

    Google Scholar 

  27. R. Poli, “Exact schema theorem and effective fitness for GP with one-point crossover,” in Proc. Genet. Evol. Comput. Conf., D. Whitley, D. Goldberg, E. Cantu-Paz, L. Spector, I. Parmee, and H.-G. Beyer (eds.), Las Vegas, Morgan Kaufmann: San Francisco, July 2000, pp. 469–476.

    Google Scholar 

  28. R. Poli, “Hyperschema theory for GP with one-point crossover, building blocks, and some new results in GAtheory,” in Genetic Programming, Proceedings of EuroGP'2000, R. Poli, W. Banzhaf, W. B. Langdon, J. F. Miller, P. Nordin, and T. C. Fogarty (eds.), volume 1802 of LNCS, Edinburgh, Springer-Verlag: Berlin, April 15–16 2000, pp. 163–180.

    Google Scholar 

  29. R. Poli, “Recursive conditional schema theorem, convergence and population sizing in genetic algorithms,” in Proceedings of the Foundations of Genetic Algorithms Workshop (FOGA 6), W. M. Spears and W. Martin (eds.), Charlottesville, VA, USA, July 2000, in press.

  30. R. Poli. “Why the schema theorem is correct also in the presence of stochastic effects,” in Proc. Congr. Evol. Comput. (CEC 2000), San Diego, USA, IEEE: Piscataway, NJ, July 2000, pp. 487–492.

    Google Scholar 

  31. R. Poli, “General schema theory for genetic programming with subtree-swapping crossover,” in Genetic Programming, Proceedings of EuroGP 2001, LNCS, Milan, Springer-Verlag: Berlin, April 18–20 2001, vol. 2038, pp. 143–159.

    Google Scholar 

  32. R. Poli and W. B. Langdon, “An experimental analysis of schema creation, propagation and disruption in genetic programming,” in Genetic Algorithms: Proc. Seventh Int. Conf., T. Back (ed.), Michigan State University, East Lansing, MI, USA, Morgan Kaufmann: San Francisco, July 19–23 1997, pp. 18–25.

    Google Scholar 

  33. R. Poli and W. B. Langdon, “Anew schema theory for genetic programming with one-point crossover and point mutation,” in Genetic Programming 1997: Proc. Sec. Annu. Conf., J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (eds.), Stanford University, CA, USA, Morgan Kaufmann: San Francisco, July 13–16 1997, pp. 278–285.

    Google Scholar 

  34. R. Poli and W. B. Langdon, “On the search properties of different crossover operators in genetic programming,” in Genetic Programming 1998: Proc. Third Annu. Conf., J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), University of Wisconsin, Madison, Wisconsin, USA, Morgan Kaufmann: San Francisco, July 22–25 1998, pp. 293–301.

    Google Scholar 

  35. R. Poli and W. B. Langdon, “Areview of theoretical and experimental results on schemata in genetic programming,” in Proceedings of the First European Workshop on Genetic Programming, W. Banzhaf, R. Poli, M. Schoenauer, and T. C. Fogarty (eds.), volume 1391 of LNCS, Paris, Springer-Verlag: Berlin, April 14–15 1998, pp. 1–15.

    Google Scholar 

  36. R. Poli and W. B. Langdon, “Schema theory for genetic programming with one-point crossover and point mutation,” Evol. Comput., vol. 6(3) pp. 231–252, 1998.

    Google Scholar 

  37. R. Poli, W. B. Langdon, and U.-M. O'Reilly, “Analysis of schema variance and short term extinction likelihoods,” in Genetic Programming 1998: Proc. Third Annu. Conf., J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), University of Wisconsin, Madison, Wisconsin, USA, Morgan Kaufmann: San Francisco, July 22–25 1998, pp. 284–292.

    Google Scholar 

  38. R. Poli and N. F. McPhee, “Exact GP schema theory for headless chicken crossover and subtree mutation,” Technical Report CSRP–00–23, University of Birmingham, School of Computer Science, December 2000.

  39. R. Poli and N. F. McPhee, “Exact schema theorems for GP with one-point and standard crossover operating on linear structures and their application to the study of the evolution of size,” in Genetic Programming, Proceedings of EuroGP 2001, LNCS, Milan, Springer-Verlag: Berlin, April 18–20 2001, vol. 2038, pp. 126–142.

    Google Scholar 

  40. R. Poli and J. Page, “Solving high-order boolean parity problems with smooth uniform crossover, sub-machine code GP and demes,” Genet. Program. Evol. Mach., vol. 1(1/2) pp. 37–56, 2000.

    Google Scholar 

  41. R. Poli, J. Page, and W. B. Langdon, “Smooth uniform crossover, sub-machine code GP and demes: Arecipe for solving high-order boolean parity problems,” in Proc. Genet. Evol. Comput. Conf., W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (eds.), Orlando, FL, USA, Morgan Kaufmann: San Francisco, July 13–17 1999, vol. 2, pp. 1162–1169.

    Google Scholar 

  42. A. Prögel-Bennett and J. L. Shapiro, “An analysis of genetic algorithms using statistical mechanics,” Physical Review Letters, vol. 72 pp. 1305–1309, 1994.

    Google Scholar 

  43. N. J. Radcliffe, “Schema processing,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), IOP Press: NY, 1997, pp. B2–5/1–10.

    Google Scholar 

  44. J. P. Rosca, “Analysis of complexity drift in genetic programming,” in Genetic Programming 1997: Proc. Second Annu. Conf., J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (eds.), Stanford University, CA, USA, Morgan Kaufmann: San Francisco, July 13–16 1997, pp. 286–294.

    Google Scholar 

  45. J. E. Rowe, “Population fixed-points for functions of unitation,” in Foundations of Genetic Algorithms, W. Banzhaf and C. Reeves (eds.), Morgan Kaufmann: San Francisco, 1999, vol. 5 pp. 69–84.

    Google Scholar 

  46. G. Rudolph, “Convergence analysis of canonical genetic algorithm,” IEEE Trans. Neural Networks, vol. 5(1) pp. 96–101, 1994.

    Google Scholar 

  47. G. Rudolph, “Genetic algorithms,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), IOP Press: NY, 1997, pp. B2.4–20–27.

    Google Scholar 

  48. G. Rudolph, “Models of stochastic convergence,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), IOP Press: NY, 1997, pp. B2.3–1–3.

    Google Scholar 

  49. G. Rudolph, “Stochastic processes,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), IOP Press: New York, 1997, pp. B2.2–1–8.

    Google Scholar 

  50. W. M. Spears, “Aggregating models of evolutionary algorithms,” in Proc. Congr. Evol. Comput., P. J. Angeline, Z. Michalewicz, M. Schoenauer, X. Yao, and A. Zalzala, (eds.), Mayflower Hotel, Washington D.C., USA, IEEE Press: Piscataway, NJ, July 6–9 1999, vol. 1, pp. 631–638.

    Google Scholar 

  51. M. R. Spiegel, Probability and Statistics, McGraw-Hill: New York, 1975.

    Google Scholar 

  52. C. R. Stephens and J. M. Vargas, “Effective fitness as an alternative paradigm for evolutionary computation I: general formalism,” Genet. Program. Evol. Mach., vol. 1(4) pp. 363–378, 2000.

    Google Scholar 

  53. C. R. Stephens and J. M. Vargas, “Effective fitness as an alternative paradigm for evolutionary computation II: examples and applications,” Genet. Program. Evol. Mach., 2001, vol. 2(1), March, pp. 1–32.

    Google Scholar 

  54. C. R. Stephens and H. Waelbroeck, “Effective degrees of freedom in genetic algorithms and the block hypothesis,” in Proc. Seventh Int. Conf. Genet. Algo. (ICGA97), T. Bäck (ed.), East Lansing, Morgan Kaufmann: San Francisco, 1997, pp. 34–40.

    Google Scholar 

  55. C. R. Stephens and H. Waelbroeck, “Schemata evolution and building blocks,” Evol. Comput., vol. 7(2) pp. 109–124, 1999.

    Google Scholar 

  56. C. R. Stephens, H. Waelbroeck, and R. Aguirre, “Schemata as building blocks: Does size matter? in Foundations of Genetic Algorithms, W. Banzhaf and C. Reeves (eds.), Morgan Kaufmann: San Francisco, CA, 1999, vol. 5, pp. 117–133.

    Google Scholar 

  57. M. D. Vose, The Simple Genetic Algorithm: Foundations and Theory. MIT Press: Cambridge, MA, 1999.

    Google Scholar 

  58. P. A. Whigham, “A schema theorem for context-free grammars,” in 1995 IEEE Conf. Evol. Comput., Perth, Australia, IEEE Press: Piscataway, NJ, 29 November-1 December 1995, vol. 1, pp. 178–181.

    Google Scholar 

  59. P. A. Whigham, “Grammatical bias for evolutionary learning,” PhD thesis, School of Computer Science, University College, University of New South Wales, Australian Defence Force Academy, October 14 1996.

  60. D. fWhitley, “Agenetic algorithm tutorial,” Technical Report CS–93–103, Department of Computer Science, Colorado State University, August 1993.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Poli, R. Exact Schema Theory for Genetic Programming and Variable-Length Genetic Algorithms with One-Point Crossover. Genetic Programming and Evolvable Machines 2, 123–163 (2001). https://doi.org/10.1023/A:1011552313821

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011552313821

Navigation