Skip to main content

Advertisement

Log in

Automatic programming: The open issue?

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

Automatic programming, the automatic generation of a computer program given a high-level statement of the program’s desired behaviour, is a stated objective of the field of genetic programming. As the general solution to a computational problem is to write a computer program, and given that genetic programming can automatically generate a computer program, researchers in the field of genetic programming refer to its ability to automatically solve problems. Genetic programming has also been described as an “invention machine” that is capable of generating human-competitive solutions. We argue that the majority of success and focus of our field has not actually been as a result of automatic programming. We set out to challenge the genetic programming community to refocus our research towards the objective of automatic programming, and to do so in a manner that embraces a wider perspective encompassing the related fields of, for example, artificial intelligence, machine learning, analytics, optimisation and software engineering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Annual “humies” awards for human-competitive results. http://www.human-competitive.org/. Accessed 21 Apr 2019

  2. Z. Manna, R. Waldinger, A deductive approach to program synthesis, in Automatic Program Construction Techniques, ed. by A. Bierman, G. Guiho, Y. Kodratoff (Macmillan Publishing Company, 1984)

  3. A. Arcuri, X. Yao, Co-evolutionary automatic programming for software development. Inf. Sci. 259, 412–432 (2014)

    Article  Google Scholar 

  4. M. Balog, A.L. Gaunt, M. Brockschmidt, S. Nowozin, D. Tarlow, Deepcoder: learning to write programs, in Proceedings International Conference on Learning Representations 2017. OpenReviews.net (2017). https://openreview.net/pdf?id=rkE3y85ee. Accessed 21 Apr 2019

  5. A. Bierman, G. Guiho, Y. Kodratoff (eds.), Automatic Program Construction Techniques, (Macmillan Publishing Company, 1984)

  6. I. Bladek, K. Krawiec, J. Swan, Counterexample-driven genetic programming: heuristic program synthesis from formal specifications. Evolut. Comput. 26(3), 441–469 (2018). https://doi.org/10.1162/evco_a_00228

    Article  Google Scholar 

  7. N.L. Cramer, A representation for the adaptive generation of simple sequential programs, in Proceedings of an International Conference on Genetic Algorithms and the Applications, ed. by J.J. Grefenstette (Carnegie-Mellon University, Pittsburgh, 1985), pp. 183–187

    Google Scholar 

  8. K.A. De Jong, On using genetic algorithms to search program spaces, in Proceedings of the Second International Conference on Genetic Algorithms on Genetic Algorithms and Their Application (L. Erlbaum Associates Inc., Hillsdale, 1987), pp. 210–216. http://dl.acm.org/citation.cfm?id=42512.42540. Accessed 21 Apr 2019

  9. L. Floridi et al., AI4people—an ethical framework for a good AI society: opportunities, risks, principles and recommendations. Minds Mach. 28, 689–707 (2018)

    Article  Google Scholar 

  10. R. Forsyth, BEAGLE a Darwinian approach to pattern recognition. Kybernetes 10(3), 159–166 (1981). https://doi.org/10.1108/eb005587

    Article  Google Scholar 

  11. R.M. Friedberg, A learning machine: part I. IBM J. Res. Dev. 2(1), 2–13 (1958). https://doi.org/10.1147/rd.21.0002

    Article  Google Scholar 

  12. R.M. Friedberg, B. Dunham, J.H. North, A learning machine: part II. IBM J. Res. Dev. 3(3), 282–287 (1959). https://doi.org/10.1147/rd.33.0282

    Article  Google Scholar 

  13. C. Fujiki, J. Dickinson, Using the genetic algorithm to generate LISP source code to solve the prisoner’s dilemma, in Proceedings of the 2nd International Conference on Genetic Algorithms, Cambridge (1987), pp. 236–240

  14. A.L. Gaunt, M. Brockschmidt, R. Singh, N. Kushman, P. Kohli, J. Taylor, D. Tarlow, Terpret: a probabilistic programming language for program induction (2016). CoRR arXiv:1608.04428

  15. S. Gulwani, Automating string processing in spreadsheets using input–output examples. SIGPLAN Not. 46(1), 317–330 (2011). https://doi.org/10.1145/1925844.1926423

    Article  MATH  Google Scholar 

  16. S. Gulwani, W.R. Harris, R. Singh, Spreadsheet data manipulation using examples. Commun. ACM 55(8), 97–105 (2012). https://doi.org/10.1145/2240236.2240260

    Article  Google Scholar 

  17. M. Harman, S.A. Mansouri, Y. Zhang, Search-based software engineering: trends, techniques and applications. ACM Comput. Surv. 45(1), 11:1–11:61 (2012). https://doi.org/10.1145/2379776.2379787

    Article  Google Scholar 

  18. T. Helmuth, N.F. McPhee, E. Pantridge, L. Spector, Improving generalization of evolved programs through automatic simplification, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’17 (ACM, Berlin, 2017), pp. 937–944. https://doi.org/10.1145/3071178.3071330

  19. T. Helmuth, L. Spector, General program synthesis benchmark suite, in GECCO ’15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, ed. by S. Silva, A.I. Esparcia-Alcazar, M. Lopez-Ibanez, S. Mostaghim, J. Timmis, C. Zarges, L. Correia, T. Soule, M. Giacobini, R. Urbanowicz, Y. Akimoto, T. Glasmachers, F.F. de Vega, A. Hoover, P. Larranaga, M. Soto, C. Cotta, F.B. Pereira, J. Handl, J. Koutnik, A. Gaspar-Cunha, H. Trautmann, J.B. Mouret, S. Risi, E. Costa, O. Schuetze, K. Krawiec, A. Moraglio, J.F. Miller, P. Widera, S. Cagnoni, J. Merelo, E. Hart, L. Trujillo, M. Kessentini, G. Ochoa, F. Chicano, C. Doerr (ACM, Madrid, 2015), pp. 1039–1046. https://doi.org/10.1145/2739480.2754769

    Chapter  Google Scholar 

  20. T. Helmuth, L. Spector, J. Matheson, Solving uncompromising problems with lexicase selection. IEEE Trans. Evolut. Comput. 19(5), 630–643 (2015). https://doi.org/10.1109/TEVC.2014.2362729

    Article  Google Scholar 

  21. T.M. Helmuth, General program synthesis from examples using genetic programming with parent selection based on random lexicographic orderings of test cases. Ph.D. thesis, College of Information and Computer Sciences, University of Massachusetts Amherst, USA (2015). https://web.cs.umass.edu/publication/details.php?id=2398. Accessed 21 Apr 2019

  22. High Level Expert Group on Artificial Intelligence, Ethics guidelines for trustworthy AI. Technical report, European Commission (2019)

  23. T.H. Hoang, D. Essam, R.I.B. McKay, N.X. Hoai, Developmental evaluation in genetic programming: the TAG-based frame work. Int. J. Knowl. Based Intell. Eng. Syst. 12(1), 69–82 (2008). https://doi.org/10.3233/KES-2008-12106

    Article  Google Scholar 

  24. J.H. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence (The University of Michigan, Ann Arbor, 1975)

    MATH  Google Scholar 

  25. G.S. Hornby, H. Lipson, J.B. Pollack, Generative representations for the automated design of modular physical robots. IEEE Trans. Robot. Autom. 19(4), 709–713 (2003). https://doi.org/10.1109/TRA.2003.814502

    Article  Google Scholar 

  26. C.G. Johnson, What can automatic programming learn from theoretical computer science?, in The 2002 U.K. Workshop on Computational Intelligence (UKCI’02) ed. by X. Yao (Birmingham, 2002). http://kar.kent.ac.uk/id/eprint/13729. Accessed 21 Apr 2019

  27. W. Kantschik, W. Banzhaf, Linear-graph GP: a new GP structure, in Genetic Programming, Proceedings of the 5th European Conference, EuroGP 2002, vol. 2278, Lecture Notes in Computer Science, ed. by J.A. Foster, E. Lutton, J. Miller, C. Ryan, A.G.B. Tettamanzi (Springer, Kinsale, 2002), pp. 83–92. https://doi.org/10.1007/3-540-45984-7_8

    Chapter  Google Scholar 

  28. S. Katayama, Recent improvements of magichaskeller, in Approaches and Applications of Inductive Programming, ed. by U. Schmid, E. Kitzelmann, R. Plasmeijer (Springer, Berlin, 2010), pp. 174–193

    Chapter  Google Scholar 

  29. R.E. Keller, W. Banzhaf, The evolution of genetic code in genetic programming, in Proceedings of the Genetic and Evolutionary Computation Conference, vol. 2, ed. by W. Banzhaf, J. Daida, A.E. Eiben, M.H. Garzon, V. Honavar, M. Jakiela, R.E. Smith (Morgan Kaufmann, Orlando, 1999), pp. 1077–1082

    Google Scholar 

  30. S. Kelly, M.I. Heywood, Emergent tangled graph representations for Atari game playing agents, in EuroGP 2017: Proceedings of the 20th European Conference on Genetic Programming, vol. 10196, Lecture Notes in Computer Science, ed. by M. Castelli, J. McDermott, L. Sekanina (Springer, Amsterdam, 2017), pp. 64–79. https://doi.org/10.1007/978-3-319-55696-3_5

    Chapter  Google Scholar 

  31. E. Kitzelmann, Data-driven induction of recursive functions from input/output-examples, in Proceedings of the ECML/PKDD 2007 Workshop on Approaches and Applications of Inductive Programming (AAIP 2007) (2007), pp. 15–26

  32. J.R. Koza, Hierarchical genetic algorithms operating on populations of computer programs, in Proceedings of the 11th International Joint Conference on Artificial Intelligence IJCAI-89, vol. 1, ed. by N.S. Sridharan (Morgan Kaufmann, Detroit, 1989), pp. 768–774

    Google Scholar 

  33. J.R. Koza, Human-competitive results produced by genetic programming. Genet. Program. Evol. Mach. 11(3/4), 251–284 (2010). https://doi.org/10.1007/s10710-010-9112-3

    Article  Google Scholar 

  34. J.R. Koza, D. Andre, F.H. Bennett III, M. Keane, Genetic Programming III: Darwinian Invention and Problem Solving (Morgan Kaufman, Burlington, 1999)

    MATH  Google Scholar 

  35. J.R. Koza, M.A. Keane, M.J. Streeter, W. Mydlowec, J. Yu, G. Lanza, Genetic Programming IV: Routine Human-Competitive Machine Intelligence (Kluwer Academic Publishers, Dordrecht, 2003)

    MATH  Google Scholar 

  36. W. La Cava, K. Danai, L. Spector, Inference of compact nonlinear dynamic models by epigenetic local search. Eng. Appl. Artif. Intell. 55, 292–306 (2016). https://doi.org/10.1016/j.engappai.2016.07.004

    Article  Google Scholar 

  37. W.G. La Cava, Automatic development and adaptation of concise nonlinear models for system identification. Doctoral dissertations May 2014-current, vol. 731 (2016). http://scholarworks.umass.edu/dissertations_2/731/. Accessed 21 Apr 2019

  38. W.B. Langdon, Genetic Programming and Data Structures: Genetic Programming + Data Structures = Automatic Programming!, vol. 1, Genetic Programming (Kluwer, Boston, 1998). https://doi.org/10.1007/978-1-4615-5731-9

    Book  MATH  Google Scholar 

  39. C. Le Goues, S. Forrest, W. Weimer, Current challenges in automatic software repair. Softw. Qual. J. 21, 421–443 (2013). https://doi.org/10.1007/s11219-013-9208-0

    Article  Google Scholar 

  40. J.D. Lohn, G. Hornby, D.S. Linden, Human-competitive evolved antennas. Artif. Intell. Eng. Des. Anal. Manuf. 22(3), 235–247 (2008). https://doi.org/10.1017/S0890060408000164

    Article  Google Scholar 

  41. J.C. Mallery, Thinking about foreign policy: finding an appropriate role for artificially intelligent computers, in The 1988 Annual Meeting of the International Studies Association (1988)

  42. R.I. McKay, N.X. Hoai, P.A. Whigham, Y. Shan, M. O’Neill, Grammar-based genetic programming: a survey. Genet. Program. Evol. Mach. 11(3/4), 365–396 (2010). https://doi.org/10.1007/s10710-010-9109-y

    Article  Google Scholar 

  43. J.F. Miller (ed.), Cartesian Genetic Programming, Natural Computing Series (Springer, Berlin, 2011). https://doi.org/10.1007/978-3-642-17310-3

    Book  MATH  Google Scholar 

  44. S. Muggleton, Inductive logic programming: issues, results and the challenge of learning language in logic. Artif. Intell. 114, 283–296 (1999)

    Article  Google Scholar 

  45. R. Olsson, Inductive functional programming using incremental program transformation. Artif. Intell. 74(1), 55–81 (1995)

    Article  Google Scholar 

  46. M. O’Neill, Automatic programming in an arbitrary language: evolving programs with grammatical evolution. Ph.D. thesis, University of Limerick, Ireland (2001). http://www.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/oneill/MichaelONeillThesis.ps.gz. Accessed 21 Apr 2019

  47. M. O’Neill, C. Ryan, Grammatical evolution by grammatical evolution: the evolution of grammar and genetic code, in Genetic Programming 7th European Conference, EuroGP 2004. Proceedings, vol. 3003, Lecture Notes in Computer Science, ed. by M. Keijzer, U.M. O’Reilly, S.M. Lucas, E. Costa, T. Soule (Springer, Coimbra, 2004), pp. 138–149. https://doi.org/10.1007/978-3-540-24650-3_13

    Chapter  Google Scholar 

  48. M. O’Neill, L. Vanneschi, S. Gustafson, W. Banzhaf, Open issues in genetic programming. Genet. Program. Evol. Mach. 11(3/4), 339–363 (2010). https://doi.org/10.1007/s10710-010-9113-2

    Article  Google Scholar 

  49. M. O’Neill, L. Vanneschi, S. Gustafson, W. Banzhaf, Open issues in genetic programming, in Tutorial on Open Issues in Genetic Programming at GECCO 2013 (The Netherlands, Amsterdam, 2013)

  50. M. Orlov, M. Sipper, FINCH: a system for evolving Java (bytecode), in Genetic Programming Theory and Practice VIII, Genetic and Evolutionary Computation, chap. 1, vol. 8, ed. by R. Riolo, T. McConaghy, E. Vladislavleva (Springer, Ann Arbor, 2010), pp. 1–16

    Google Scholar 

  51. E. Pantridge, T. Helmuth, N.F. McPhee, L. Spector, On the difficulty of benchmarking inductive program synthesis methods, in Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’17 (ACM, Berlin, 2017), pp. 1589–1596. https://doi.org/10.1145/3067695.3082533

  52. E. Pantridge, L. Spector, PyshGP: PushGP in python, in Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’17 (ACM, Berlin, 2017), pp. 1255–1262. https://doi.org/10.1145/3067695.3082468

  53. T.P. Pawlak, K. Krawiec, Synthesis of constraints for mathematical programming with one-class genetic programming. IEEE Trans. Evolut. Comput. (2018). https://doi.org/10.1109/TEVC.2018.2835565

    Article  Google Scholar 

  54. J. Petke, S.O. Haraldsson, M. Harman, W.B. Langdon, D.R. White, J.R. Woodward, Genetic improvement of software: a comprehensive survey. IEEE Trans. Evolut. Comput. 22(3), 415–432 (2018). https://doi.org/10.1109/TEVC.2017.2693219

    Article  Google Scholar 

  55. C. Rich, R.C. Waters, Automatic programming: myths and prospects. Computer 21(8), 40–51 (1988). https://doi.org/10.1109/2.75

    Article  Google Scholar 

  56. F. Rothlauf, Representations for Genetic and Evolutionary Algorithms (Springer, Berlin, 2006)

    Book  Google Scholar 

  57. A.L. Samuel, Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 3(3), 210–229 (1959). https://doi.org/10.1147/rd.33.0210

    Article  MathSciNet  Google Scholar 

  58. M. Schmidt, H. Lipson, Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2009). https://doi.org/10.1126/science.1165893

    Article  Google Scholar 

  59. L. Spector, A. Robinson, Genetic programming and autoconstructive evolution with the push programming language. Genet. Program. Evol. Mach. 3(1), 7–40 (2002). https://doi.org/10.1023/A:1014538503543

    Article  MATH  Google Scholar 

  60. M. O’Neill, D. Fagan, The Elephant in the room: Towards the application of genetic programming to automatic programming. in Genetic Programming Theory and Practice XVI (Springer, 2019), pp. 179–192.

  61. K.O. Stanley, R. Miikkulainen, Evolving neural networks through augmenting topologies. Evolut. Comput. 10(2), 99–127 (2002). https://doi.org/10.1162/106365602320169811

    Article  Google Scholar 

  62. A. Teller, Turing completeness in the language of genetic programming with indexed memory, in Proceedings of the 1994 IEEE World Congress on Computational Intelligence, vol. 1 (IEEE Press, Orlando, 1994), pp. 136–141. https://doi.org/10.1109/ICEC.1994.350027

  63. P.S. Thomas, B.C. da Silva, A.G. Barto, E. Brunskill, On ensuring that intelligent machines are well-behaved. CoRR arXiv:1708.05448 (2017)

  64. M. Vechev, E. Yahav, Programming with “big code”. Found. Trends Program. Lang. 3(4), 231–284 (2016). https://doi.org/10.1561/2500000028

    Article  Google Scholar 

  65. W. Weimer, S. Forrest, C. Le Goues, T. Nguyen, Automatic program repair with evolutionary computation. Commun. ACM 53(5), 109–116 (2010). https://doi.org/10.1145/1735223.1735249

    Article  Google Scholar 

  66. S.M. West, M. Whittaker, K. Crawford, Discriminating systems: gender, race and power in AI. Technical report (2019)

  67. P.A. Whigham, Grammatically-based genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications ed. by J.P. Rosca, Tahoe City, California, USA, pp. 33–41 (1995). http://divcom.otago.ac.nz/sirc/Peterw/Publications/ml95.zip. Accessed 21 Apr 2019

  68. D.G. Wilson, S. Cussat-Blanc, H. Luga, J.F. Miller, Evolving simple programs for playing Atari games, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’18 (ACM, New York, 2018), pp. 229–236. https://doi.org/10.1145/3205455.3205578

  69. J. Woodward, Evolving turing complete representations, in Proceedings of the 2003 Congress on Evolutionary Computation CEC2003, ed. by R. Sarker, R. Reynolds, H. Abbass, K.C. Tan, B. McKay, D. Essam, T. Gedeon (IEEE Press, Canberra, 2003), pp. 830–837. https://doi.org/10.1109/CEC.2003.1299753

    Chapter  Google Scholar 

Download references

Acknowledgements

We would like to thank the reviewers for their constructive feedback. MO’N is supported by the Science Foundation Ireland under Grants 13/IA/1850 and 13/RC/2094. This material is based upon work supported by the National Science Foundation under Grant No. 1617087. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michael O’Neill.

Additional information

Handled by Dr. W. B. Langdon and Dr. Nicholas Freitag McPhee.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

O’Neill, M., Spector, L. Automatic programming: The open issue?. Genet Program Evolvable Mach 21, 251–262 (2020). https://doi.org/10.1007/s10710-019-09364-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-019-09364-2

Keywords

Navigation