Abstract
Automatic programming, the automatic generation of a computer program given a high-level statement of the program’s desired behaviour, is a stated objective of the field of genetic programming. As the general solution to a computational problem is to write a computer program, and given that genetic programming can automatically generate a computer program, researchers in the field of genetic programming refer to its ability to automatically solve problems. Genetic programming has also been described as an “invention machine” that is capable of generating human-competitive solutions. We argue that the majority of success and focus of our field has not actually been as a result of automatic programming. We set out to challenge the genetic programming community to refocus our research towards the objective of automatic programming, and to do so in a manner that embraces a wider perspective encompassing the related fields of, for example, artificial intelligence, machine learning, analytics, optimisation and software engineering.
Similar content being viewed by others
References
Annual “humies” awards for human-competitive results. http://www.human-competitive.org/. Accessed 21 Apr 2019
Z. Manna, R. Waldinger, A deductive approach to program synthesis, in Automatic Program Construction Techniques, ed. by A. Bierman, G. Guiho, Y. Kodratoff (Macmillan Publishing Company, 1984)
A. Arcuri, X. Yao, Co-evolutionary automatic programming for software development. Inf. Sci. 259, 412–432 (2014)
M. Balog, A.L. Gaunt, M. Brockschmidt, S. Nowozin, D. Tarlow, Deepcoder: learning to write programs, in Proceedings International Conference on Learning Representations 2017. OpenReviews.net (2017). https://openreview.net/pdf?id=rkE3y85ee. Accessed 21 Apr 2019
A. Bierman, G. Guiho, Y. Kodratoff (eds.), Automatic Program Construction Techniques, (Macmillan Publishing Company, 1984)
I. Bladek, K. Krawiec, J. Swan, Counterexample-driven genetic programming: heuristic program synthesis from formal specifications. Evolut. Comput. 26(3), 441–469 (2018). https://doi.org/10.1162/evco_a_00228
N.L. Cramer, A representation for the adaptive generation of simple sequential programs, in Proceedings of an International Conference on Genetic Algorithms and the Applications, ed. by J.J. Grefenstette (Carnegie-Mellon University, Pittsburgh, 1985), pp. 183–187
K.A. De Jong, On using genetic algorithms to search program spaces, in Proceedings of the Second International Conference on Genetic Algorithms on Genetic Algorithms and Their Application (L. Erlbaum Associates Inc., Hillsdale, 1987), pp. 210–216. http://dl.acm.org/citation.cfm?id=42512.42540. Accessed 21 Apr 2019
L. Floridi et al., AI4people—an ethical framework for a good AI society: opportunities, risks, principles and recommendations. Minds Mach. 28, 689–707 (2018)
R. Forsyth, BEAGLE a Darwinian approach to pattern recognition. Kybernetes 10(3), 159–166 (1981). https://doi.org/10.1108/eb005587
R.M. Friedberg, A learning machine: part I. IBM J. Res. Dev. 2(1), 2–13 (1958). https://doi.org/10.1147/rd.21.0002
R.M. Friedberg, B. Dunham, J.H. North, A learning machine: part II. IBM J. Res. Dev. 3(3), 282–287 (1959). https://doi.org/10.1147/rd.33.0282
C. Fujiki, J. Dickinson, Using the genetic algorithm to generate LISP source code to solve the prisoner’s dilemma, in Proceedings of the 2nd International Conference on Genetic Algorithms, Cambridge (1987), pp. 236–240
A.L. Gaunt, M. Brockschmidt, R. Singh, N. Kushman, P. Kohli, J. Taylor, D. Tarlow, Terpret: a probabilistic programming language for program induction (2016). CoRR arXiv:1608.04428
S. Gulwani, Automating string processing in spreadsheets using input–output examples. SIGPLAN Not. 46(1), 317–330 (2011). https://doi.org/10.1145/1925844.1926423
S. Gulwani, W.R. Harris, R. Singh, Spreadsheet data manipulation using examples. Commun. ACM 55(8), 97–105 (2012). https://doi.org/10.1145/2240236.2240260
M. Harman, S.A. Mansouri, Y. Zhang, Search-based software engineering: trends, techniques and applications. ACM Comput. Surv. 45(1), 11:1–11:61 (2012). https://doi.org/10.1145/2379776.2379787
T. Helmuth, N.F. McPhee, E. Pantridge, L. Spector, Improving generalization of evolved programs through automatic simplification, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’17 (ACM, Berlin, 2017), pp. 937–944. https://doi.org/10.1145/3071178.3071330
T. Helmuth, L. Spector, General program synthesis benchmark suite, in GECCO ’15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, ed. by S. Silva, A.I. Esparcia-Alcazar, M. Lopez-Ibanez, S. Mostaghim, J. Timmis, C. Zarges, L. Correia, T. Soule, M. Giacobini, R. Urbanowicz, Y. Akimoto, T. Glasmachers, F.F. de Vega, A. Hoover, P. Larranaga, M. Soto, C. Cotta, F.B. Pereira, J. Handl, J. Koutnik, A. Gaspar-Cunha, H. Trautmann, J.B. Mouret, S. Risi, E. Costa, O. Schuetze, K. Krawiec, A. Moraglio, J.F. Miller, P. Widera, S. Cagnoni, J. Merelo, E. Hart, L. Trujillo, M. Kessentini, G. Ochoa, F. Chicano, C. Doerr (ACM, Madrid, 2015), pp. 1039–1046. https://doi.org/10.1145/2739480.2754769
T. Helmuth, L. Spector, J. Matheson, Solving uncompromising problems with lexicase selection. IEEE Trans. Evolut. Comput. 19(5), 630–643 (2015). https://doi.org/10.1109/TEVC.2014.2362729
T.M. Helmuth, General program synthesis from examples using genetic programming with parent selection based on random lexicographic orderings of test cases. Ph.D. thesis, College of Information and Computer Sciences, University of Massachusetts Amherst, USA (2015). https://web.cs.umass.edu/publication/details.php?id=2398. Accessed 21 Apr 2019
High Level Expert Group on Artificial Intelligence, Ethics guidelines for trustworthy AI. Technical report, European Commission (2019)
T.H. Hoang, D. Essam, R.I.B. McKay, N.X. Hoai, Developmental evaluation in genetic programming: the TAG-based frame work. Int. J. Knowl. Based Intell. Eng. Syst. 12(1), 69–82 (2008). https://doi.org/10.3233/KES-2008-12106
J.H. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence (The University of Michigan, Ann Arbor, 1975)
G.S. Hornby, H. Lipson, J.B. Pollack, Generative representations for the automated design of modular physical robots. IEEE Trans. Robot. Autom. 19(4), 709–713 (2003). https://doi.org/10.1109/TRA.2003.814502
C.G. Johnson, What can automatic programming learn from theoretical computer science?, in The 2002 U.K. Workshop on Computational Intelligence (UKCI’02) ed. by X. Yao (Birmingham, 2002). http://kar.kent.ac.uk/id/eprint/13729. Accessed 21 Apr 2019
W. Kantschik, W. Banzhaf, Linear-graph GP: a new GP structure, in Genetic Programming, Proceedings of the 5th European Conference, EuroGP 2002, vol. 2278, Lecture Notes in Computer Science, ed. by J.A. Foster, E. Lutton, J. Miller, C. Ryan, A.G.B. Tettamanzi (Springer, Kinsale, 2002), pp. 83–92. https://doi.org/10.1007/3-540-45984-7_8
S. Katayama, Recent improvements of magichaskeller, in Approaches and Applications of Inductive Programming, ed. by U. Schmid, E. Kitzelmann, R. Plasmeijer (Springer, Berlin, 2010), pp. 174–193
R.E. Keller, W. Banzhaf, The evolution of genetic code in genetic programming, in Proceedings of the Genetic and Evolutionary Computation Conference, vol. 2, ed. by W. Banzhaf, J. Daida, A.E. Eiben, M.H. Garzon, V. Honavar, M. Jakiela, R.E. Smith (Morgan Kaufmann, Orlando, 1999), pp. 1077–1082
S. Kelly, M.I. Heywood, Emergent tangled graph representations for Atari game playing agents, in EuroGP 2017: Proceedings of the 20th European Conference on Genetic Programming, vol. 10196, Lecture Notes in Computer Science, ed. by M. Castelli, J. McDermott, L. Sekanina (Springer, Amsterdam, 2017), pp. 64–79. https://doi.org/10.1007/978-3-319-55696-3_5
E. Kitzelmann, Data-driven induction of recursive functions from input/output-examples, in Proceedings of the ECML/PKDD 2007 Workshop on Approaches and Applications of Inductive Programming (AAIP 2007) (2007), pp. 15–26
J.R. Koza, Hierarchical genetic algorithms operating on populations of computer programs, in Proceedings of the 11th International Joint Conference on Artificial Intelligence IJCAI-89, vol. 1, ed. by N.S. Sridharan (Morgan Kaufmann, Detroit, 1989), pp. 768–774
J.R. Koza, Human-competitive results produced by genetic programming. Genet. Program. Evol. Mach. 11(3/4), 251–284 (2010). https://doi.org/10.1007/s10710-010-9112-3
J.R. Koza, D. Andre, F.H. Bennett III, M. Keane, Genetic Programming III: Darwinian Invention and Problem Solving (Morgan Kaufman, Burlington, 1999)
J.R. Koza, M.A. Keane, M.J. Streeter, W. Mydlowec, J. Yu, G. Lanza, Genetic Programming IV: Routine Human-Competitive Machine Intelligence (Kluwer Academic Publishers, Dordrecht, 2003)
W. La Cava, K. Danai, L. Spector, Inference of compact nonlinear dynamic models by epigenetic local search. Eng. Appl. Artif. Intell. 55, 292–306 (2016). https://doi.org/10.1016/j.engappai.2016.07.004
W.G. La Cava, Automatic development and adaptation of concise nonlinear models for system identification. Doctoral dissertations May 2014-current, vol. 731 (2016). http://scholarworks.umass.edu/dissertations_2/731/. Accessed 21 Apr 2019
W.B. Langdon, Genetic Programming and Data Structures: Genetic Programming + Data Structures = Automatic Programming!, vol. 1, Genetic Programming (Kluwer, Boston, 1998). https://doi.org/10.1007/978-1-4615-5731-9
C. Le Goues, S. Forrest, W. Weimer, Current challenges in automatic software repair. Softw. Qual. J. 21, 421–443 (2013). https://doi.org/10.1007/s11219-013-9208-0
J.D. Lohn, G. Hornby, D.S. Linden, Human-competitive evolved antennas. Artif. Intell. Eng. Des. Anal. Manuf. 22(3), 235–247 (2008). https://doi.org/10.1017/S0890060408000164
J.C. Mallery, Thinking about foreign policy: finding an appropriate role for artificially intelligent computers, in The 1988 Annual Meeting of the International Studies Association (1988)
R.I. McKay, N.X. Hoai, P.A. Whigham, Y. Shan, M. O’Neill, Grammar-based genetic programming: a survey. Genet. Program. Evol. Mach. 11(3/4), 365–396 (2010). https://doi.org/10.1007/s10710-010-9109-y
J.F. Miller (ed.), Cartesian Genetic Programming, Natural Computing Series (Springer, Berlin, 2011). https://doi.org/10.1007/978-3-642-17310-3
S. Muggleton, Inductive logic programming: issues, results and the challenge of learning language in logic. Artif. Intell. 114, 283–296 (1999)
R. Olsson, Inductive functional programming using incremental program transformation. Artif. Intell. 74(1), 55–81 (1995)
M. O’Neill, Automatic programming in an arbitrary language: evolving programs with grammatical evolution. Ph.D. thesis, University of Limerick, Ireland (2001). http://www.cs.ucl.ac.uk/staff/W.Langdon/ftp/papers/oneill/MichaelONeillThesis.ps.gz. Accessed 21 Apr 2019
M. O’Neill, C. Ryan, Grammatical evolution by grammatical evolution: the evolution of grammar and genetic code, in Genetic Programming 7th European Conference, EuroGP 2004. Proceedings, vol. 3003, Lecture Notes in Computer Science, ed. by M. Keijzer, U.M. O’Reilly, S.M. Lucas, E. Costa, T. Soule (Springer, Coimbra, 2004), pp. 138–149. https://doi.org/10.1007/978-3-540-24650-3_13
M. O’Neill, L. Vanneschi, S. Gustafson, W. Banzhaf, Open issues in genetic programming. Genet. Program. Evol. Mach. 11(3/4), 339–363 (2010). https://doi.org/10.1007/s10710-010-9113-2
M. O’Neill, L. Vanneschi, S. Gustafson, W. Banzhaf, Open issues in genetic programming, in Tutorial on Open Issues in Genetic Programming at GECCO 2013 (The Netherlands, Amsterdam, 2013)
M. Orlov, M. Sipper, FINCH: a system for evolving Java (bytecode), in Genetic Programming Theory and Practice VIII, Genetic and Evolutionary Computation, chap. 1, vol. 8, ed. by R. Riolo, T. McConaghy, E. Vladislavleva (Springer, Ann Arbor, 2010), pp. 1–16
E. Pantridge, T. Helmuth, N.F. McPhee, L. Spector, On the difficulty of benchmarking inductive program synthesis methods, in Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’17 (ACM, Berlin, 2017), pp. 1589–1596. https://doi.org/10.1145/3067695.3082533
E. Pantridge, L. Spector, PyshGP: PushGP in python, in Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’17 (ACM, Berlin, 2017), pp. 1255–1262. https://doi.org/10.1145/3067695.3082468
T.P. Pawlak, K. Krawiec, Synthesis of constraints for mathematical programming with one-class genetic programming. IEEE Trans. Evolut. Comput. (2018). https://doi.org/10.1109/TEVC.2018.2835565
J. Petke, S.O. Haraldsson, M. Harman, W.B. Langdon, D.R. White, J.R. Woodward, Genetic improvement of software: a comprehensive survey. IEEE Trans. Evolut. Comput. 22(3), 415–432 (2018). https://doi.org/10.1109/TEVC.2017.2693219
C. Rich, R.C. Waters, Automatic programming: myths and prospects. Computer 21(8), 40–51 (1988). https://doi.org/10.1109/2.75
F. Rothlauf, Representations for Genetic and Evolutionary Algorithms (Springer, Berlin, 2006)
A.L. Samuel, Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 3(3), 210–229 (1959). https://doi.org/10.1147/rd.33.0210
M. Schmidt, H. Lipson, Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2009). https://doi.org/10.1126/science.1165893
L. Spector, A. Robinson, Genetic programming and autoconstructive evolution with the push programming language. Genet. Program. Evol. Mach. 3(1), 7–40 (2002). https://doi.org/10.1023/A:1014538503543
M. O’Neill, D. Fagan, The Elephant in the room: Towards the application of genetic programming to automatic programming. in Genetic Programming Theory and Practice XVI (Springer, 2019), pp. 179–192.
K.O. Stanley, R. Miikkulainen, Evolving neural networks through augmenting topologies. Evolut. Comput. 10(2), 99–127 (2002). https://doi.org/10.1162/106365602320169811
A. Teller, Turing completeness in the language of genetic programming with indexed memory, in Proceedings of the 1994 IEEE World Congress on Computational Intelligence, vol. 1 (IEEE Press, Orlando, 1994), pp. 136–141. https://doi.org/10.1109/ICEC.1994.350027
P.S. Thomas, B.C. da Silva, A.G. Barto, E. Brunskill, On ensuring that intelligent machines are well-behaved. CoRR arXiv:1708.05448 (2017)
M. Vechev, E. Yahav, Programming with “big code”. Found. Trends Program. Lang. 3(4), 231–284 (2016). https://doi.org/10.1561/2500000028
W. Weimer, S. Forrest, C. Le Goues, T. Nguyen, Automatic program repair with evolutionary computation. Commun. ACM 53(5), 109–116 (2010). https://doi.org/10.1145/1735223.1735249
S.M. West, M. Whittaker, K. Crawford, Discriminating systems: gender, race and power in AI. Technical report (2019)
P.A. Whigham, Grammatically-based genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications ed. by J.P. Rosca, Tahoe City, California, USA, pp. 33–41 (1995). http://divcom.otago.ac.nz/sirc/Peterw/Publications/ml95.zip. Accessed 21 Apr 2019
D.G. Wilson, S. Cussat-Blanc, H. Luga, J.F. Miller, Evolving simple programs for playing Atari games, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’18 (ACM, New York, 2018), pp. 229–236. https://doi.org/10.1145/3205455.3205578
J. Woodward, Evolving turing complete representations, in Proceedings of the 2003 Congress on Evolutionary Computation CEC2003, ed. by R. Sarker, R. Reynolds, H. Abbass, K.C. Tan, B. McKay, D. Essam, T. Gedeon (IEEE Press, Canberra, 2003), pp. 830–837. https://doi.org/10.1109/CEC.2003.1299753
Acknowledgements
We would like to thank the reviewers for their constructive feedback. MO’N is supported by the Science Foundation Ireland under Grants 13/IA/1850 and 13/RC/2094. This material is based upon work supported by the National Science Foundation under Grant No. 1617087. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Additional information
Handled by Dr. W. B. Langdon and Dr. Nicholas Freitag McPhee.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
O’Neill, M., Spector, L. Automatic programming: The open issue?. Genet Program Evolvable Mach 21, 251–262 (2020). https://doi.org/10.1007/s10710-019-09364-2
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10710-019-09364-2