Robert J. Smith
Malcolm I. Heywood
ABSTRACT
Scaling genetic programming to organize large complex combinations of programs remains an under investigated topic in general. This work revisits the issue by first demonstrating the respective contributions of coevolution and diversity maintenance. Competitive coevolution is employed to organize a task in such a way that the most informative training cases are retained. Cooperative coevolution helps discover modularity in the solutions discovered and, in this work, is fundamental to constructing complex structures of programs that still execute efficiently (the policy tree). The role of coevolution and diversity maintenance is first independently established under the task of discovering reinforcement learning policies for solving Rubik's Cubes scrambled with 5-twists. With this established, a combined approach is then adopted for building large organizations of code for representing policies that solve 5 to 8-twist combinations of the Cube. The resulting 'deep' policy tree organizes hundreds of programs to provide optimal solutions to tens of millions of test cube configurations.
- Isidro M. Alvarez, Will N. Browne, and Mengjie Zhang. 2016. Human-inspired Scaling in Learning Classifier Systems: Case Study on the n-bit Multiplexer Problem Set. In ACM Genetic and Evolutionary Computation Conference. 429--436. Google Scholar
Digital Library
- S. J. Arfaee, S. Zilies, and R. C. Holte. 2011. Learning heuristic functions for large state spaces. Artificial Intelligence 175 (2011), 2075--2098. Google ScholarDigital Library
- E. B. Baum and I. Durdanovic. 2000. Evolution of cooperative problem solving in an artificial economy. Neural Computation 12 (2000), 2743--2775. Google ScholarDigital Library
- E. D. de Jong. 2007. A monotonic archive for Pareto-coevolution. Evolutionary Computation 15, 1 (2007), 61--93. Google ScholarDigital Library
- J. Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1 (2006), 1--30. Google ScholarDigital Library
- J. A. Doucette, P. Lichodzijewski, and M. I. Heywood. 2012. Hierarchical task decomposition through symbiosis in reinforcement learning. In ACM Genetic and Evolutionary Computation Conference. 97--104. Google ScholarDigital Library
- J. A. Doucette, A. R. McIntyre, P. Lichodzijewski, and M. I. Heywood. 2012. Symbiotic coevolutionary genetic programming: A benchmarking study under large attribute spaces. Genetic Programming and Evolvable Machines 13 (2012), 71--101. Google ScholarDigital Library
- N. El-Sourani, S. Hauke, and M. Borschbach. 2010. An evolutionary approach for solving the Rubik's Cube incorporating exact methods. In EvoApplications: Part I (LNCS), Vol. 6024. 80--89. Google ScholarDigital Library
- S. G. Ficici and J. B. Pollack. 2001. Pareto optimality in coevolutionary learning. In European Conference on Advances in Artificial Life (LNAI), Vol. 2159. 316--325. Google ScholarDigital Library
- F. Gomez and R. Miikkulainen. 1997. Incremental evolution of complex general behavior. Adaptive Behavior 5, 3--4 (1997), 317--342. Google ScholarDigital Library
- A. Hauptman, A. Elyasaf, M. Sipper, and A. Karmon. 2009. GP-Rush: Using genetic programming to evolve solvers for the rush hour puzzle. In ACM Genetic and Evolutionary Computation Conference. 955--962. Google ScholarDigital Library
- William H. Hsu, Scott J. Harmon, Edwin Rodriguez, and Christopher Zhong. 2004. Empirical comparison of incremental reuse strategies in genetic programming for keep-away soccer. In Genetic and Evolutionary Computation Conference - Late Breaking Papers.Google Scholar
- W. Jaśkowski, K. Krawiec, and B. Wieloch. 2007. Knowledge reuse in genetic programming applied to visual learning. In Proceedings of the ACM Genetic and Evolutionary Computation Conference. 1790--1797. Google ScholarDigital Library
- M. Keijzer, C. Ryan, and M. Cattolico. 2004. Run transferable libraries - learning functional bias in problem domains. In Proceedings of the Genetic and Evolutionary Computation Conference (LNCS), Vol. 3103. 531--542.Google Scholar
- S. Kelly and M. I. Heywood. 2014. On Diversity, Teaming, and Hierarchical Policies: Observations from the Keepaway Soccer Task. In European Conference on Genetic Programming (LNCS), Vol. 8599. Springer, 75--86. Google ScholarDigital Library
- S. Kelly and M. I. Heywood. 2015. Knowledge Transfer from Keepaway Soccer to Half-field Offense Through Program Symbiosis: Building Simple Programs for a Complex Task. In ACM Genetic and Evolutionary Computation Conference. 1143--1150. Google ScholarDigital Library
- S. Kelly, P. Lichodzijewski, and M. I. Heywood. 2012. On run time libraries and hierarchical symbiosis. In IEEE Congress on Evolutionary Computation. 3245--3252.Google Scholar
- R. E. Korf. 1997. Finding optimal solutions to Rubik's Cube using pattern databases. In AAAI Conference on Artificial Intelligence. 700--705. Google ScholarDigital Library
- Y. LeCun, Y. Bengio, and G. Hinton. 2015. Deep Learning. Nature 521 (2015), 436--444.Google ScholarCross Ref
- P. Lichodzijewski and M. Heywood. 2010. The Rubic's Cube and GP temporal sequence learning: An initial study. In Genetic Programming Theory and Practice VIII. Springer, Chapter 3, 35--54.Google Scholar
- P. Lichodzijewski and M. I. Heywood. 2008. Managing team-based problem solving with Symbiotic Bid-based Genetic Programming. In ACM Genetic and Evolutionary Computation Conference. 363--370. Google ScholarDigital Library
- C. D. Rosin and R. K. Belew. 1997. New methods for competitive coevolution. Evolutionary Computation 5 (1997), 1--29. Google ScholarDigital Library
- Robert J. Smith, Stephen Kelly, and Malcolm I. Heywood. 2016. Discovering Rubik's Cube Subgroups using Coevolutionary GP: A Five Twist Experiment. In ACM Genetic and Evolutionary Computation Conference. 789--796. Google ScholarDigital Library
- L. Spector, K. Harrington, and T. Helmuth. 2012. Tag-based modularity in tree-based genetic programming. In Proceedings of the ACM Genetic and Evolutionary Computation Conference. 815--826. Google ScholarDigital Library
- P. Stone. 2000. Layered learning in multiagent systems. MIT Press. Google ScholarDigital Library
- Matthew E. Taylor and Peter Stone. 2011. An Introduction to Inter-task Transfer for Reinforcement Learning. AI Magazine 32, 1 (2011), 15--34.Google ScholarCross Ref
- S. Wu and W. Banzhaf. 2011. Rethinking multilevel selection in genetic programming. In Proceedings of the ACM Genetic and Evolutionary Computation Conference. 1403--1410. Google ScholarDigital Library
Index Terms
- Coevolving deep hierarchies of programs to solve complex tasks
Recommendations
Evolutionary consequences of coevolving targets
Most evolutionary optimization models incorporate a fitness evaluation that is based on a predefined static set of test cases or problems. In the natural evolutionary process, selection is of course not based on a static fitness evaluation. Organisms do ...
Investigating the success of spatial coevolution
GECCO '05: Proceedings of the 7th annual conference on Genetic and evolutionary computationWe investigate the results of coevolution of spatially distributed populations. In particular, we describe work in which a simple function approximation problem is used to compare different spatial evolutionary methods. Our work shows that, on this ...
Discovering Rubik's Cube Subgroups using Coevolutionary GP: A Five Twist Experiment
GECCO '16: Proceedings of the Genetic and Evolutionary Computation Conference 2016This work reports on an approach to direct policy discovery (a form of reinforcement learning) using genetic programming (GP) for the 3 by 3 by 3 Rubik's Cube. Specifically, a synthesis of two approaches is proposed: 1) a previous group theoretic ...
Comments