Skip to main content
SpringerLink
Log in
Book cover

Computational Intelligence pp 177–201Cite as

Exploring Hyper-heuristic Methodologies with Genetic Programming

  • Chapter

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 1))

Abstract

Hyper-heuristics represent a novel search methodology that is motivated by the goal of automating the process of selecting or combining simpler heuristics in order to solve hard computational search problems. An extension of the original hyper-heuristic idea is to generate new heuristics which are not currently known. These approaches operate on a search space of heuristics rather than directly on a search space of solutions to the underlying problem which is the case with most meta-heuristics implementations. In the majority of hyper-heuristic studies so far, a framework is provided with a set of human designed heuristics, taken from the literature, and with good measures of performance in practice. A less well studied approach aims to generate new heuristics from a set of potential heuristic components. The purpose of this chapter is to discuss this class of hyper-heuristics, in which Genetic Programming is the most widely used methodology. A detailed discussion is presented including the steps needed to apply this technique, some representative case studies, a literature review of related work, and a discussion of relevant issues. Our aim is to convey the exciting potential of this innovative approach for automating the heuristic design process.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   259.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Asmuni, H., Burke, E.K., Garibaldi, J.M., Mccollum, B.: A novel fuzzy approach to evaluate the quality of examination timetabling. In: Proceedings of the 6th International Conference on the Practice and Theory of Automated Timetabling, pp. 82–102 (2006)

    Google Scholar 

  2. Bäck, T.: An overview of parameter control methods by self-adaption in evolutionary algorithms. Fundam. Inf. 35(1-4), 51–66 (1998)

    MATH  Google Scholar 

  3. Bader-El-Din, M.B., Poli, R.: Generating SAT local-search heuristics using a GP hyper-heuristic framework. In: Monmarché, N., Talbi, E.-G., Collet, P., Schoenauer, M., Lutton, E. (eds.) EA 2007. LNCS, vol. 4926, pp. 37–49. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  4. Bai, R., Kendall, G.: An investigation of automated planograms using a simulated annealing based hyper-heuristic. In: Ibaraki, T., Nonobe, K., Yagiura, M. (eds.) Metaheuristics: Progress as Real Problem Solver. Operations Research/Computer Science Interface Serices, vol. 32, pp. 87–108. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  5. Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming – An Introduction; On the Automatic Evolution of Computer Programs and its Applications. Morgan Kaufmann, San Francisco (1998)

    MATH  Google Scholar 

  6. Battiti, R.: Reactive search: Toward self–tuning heuristics. In: Rayward-Smith, V.J., Osman, I.H., Reeves, C.R., Smith, G.D. (eds.) Modern Heuristic Search Methods, pp. 61–83. John Wiley & Sons Ltd, Chichester (1996)

    Google Scholar 

  7. Battiti, R., Brunato, M.: Reactive search: Machine learning for memory-based heuristics. In: Gonzalez, T.F. (ed.) Approximation Algorithms and Metaheuristics, ch. 21, pp. 1–17. Taylor and Francis Books/CRC Press, Washington (2007)

    Google Scholar 

  8. Birattari, M.: The problem of tuning metaheuristics as seen from a machine learning perspective. Ph.D. thesis, Universite Libre de Bruxelles (2004)

    Google Scholar 

  9. Burke, E.K., Hart, E., Kendall, G., Newall, J., Ross, P., Schulenburg, S.: Hyper-heuristics: An emerging direction in modern search technology. In: Glover, F., Kochenberger, G. (eds.) Handbook of Metaheuristics, pp. 457–474. Kluwer, Dordrecht (2003)

    Google Scholar 

  10. Burke, E.K., Hyde, M.R., Kendall, G.: Evolving bin packing heuristics with genetic programming. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 860–869. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  11. Burke, E.K., Hyde, M.R., Kendall, G., Woodward, J.: Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one. In: Thierens, D., et al. (eds.) Proceedings of the 9th annual conference on Genetic and evolutionary computation GECCO 2007, vol. 2, pp. 1559–1565. ACM Press, London (2007)

    Chapter  Google Scholar 

  12. Burke, E.K., Hyde, M.R., Kendall, G., Woodward, J.R.: The scalability of evolved on line bin packing heuristics. In: Srinivasan, D., Wang, L. (eds.) 2007 IEEE Congress on Evolutionary Computation, pp. 2530–2537. IEEE Computational Intelligence Society/IEEE Press, Singapore (2007)

    Chapter  Google Scholar 

  13. Burke, E.K., Kendall, G., Soubeiga, E.: A tabu-search hyperheuristic for timetabling and rostering. J. of Heuristics 9(6), 451–470 (2003)

    Article  Google Scholar 

  14. Burke, E.K., McCollum, B., Meisels, A., Petrovic, S., Qu, R.: A graph-based hyper-heuristic for educational timetabling problems. Eur. J. of Oper. Res. 176, 177–192 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  15. Burke, E.K., Petrovic, S., Qu, R.: Case based heuristic selection for timetabling problems. J. of Sched. 9(2), 115–132 (2006)

    Article  MATH  Google Scholar 

  16. Coffman Jr., E.G., Galambos, G., Martello, S., Vigo, D.: Bin packing approximation algorithms: Combinatorial analysis. In: Du, D.Z., Pardalos, P.M. (eds.) Handbook of Combinatorial Optimization, pp. 151–207. Kluwer, Dordrecht (1998)

    Google Scholar 

  17. Cowling, P., Kendall, G., Soubeiga, E.: A hyperheuristic approach to scheduling a sales summit. In: Burke, E., Erben, W. (eds.) PATAT 2000. LNCS, vol. 2079, pp. 176–190. Springer, Heidelberg (2001) (selected papers)

    Chapter  Google Scholar 

  18. Dowsland, K.A., Soubeiga, E., Burke, E.K.: A simulated annealing hyper-heuristic for determining shipper sizes. Eur. J. of Oper. Res. 179(3), 759–774 (2007)

    Article  MATH  Google Scholar 

  19. Drechsler, N., Schmiedle, F., Grosse, D., Drechsler, R.: Heuristic learning based on genetic programming. In: Miller, J., Tomassini, M., Lanzi, P.L., Ryan, C., Tetamanzi, A.G.B., Langdon, W.B. (eds.) EuroGP 2001. LNCS, vol. 2038, pp. 1–10. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  20. Drechsler, R., Becker, B.: Learning heuristics by genetic algorithms. In: ASP-DAC 1995: Proceedings of the 1995 conference on Asia Pacific design automation (CD-ROM), p. 53. ACM, New York (1995)

    Chapter  Google Scholar 

  21. Eiben, A.E., Hinterding, R., Michalewicz, Z.: Parameter control in Evolutionary Algorithms. IEEE Trans. on Evol. Comput. 3(2), 124–141 (1999)

    Article  Google Scholar 

  22. Falkenauer, E., Delchambre, A.: A genetic algorithm for bin packing and line balancing. In: Proc. of the IEEE 1992 International Conference on Robotics and Automation, pp. 1186–1192 (1992)

    Google Scholar 

  23. Fang, H.L., Ross, P., Corne, D.: A promising hybrid GA/heuristic approach for open-shop scheduling problems. In: Eur. Conference on Artificial Intelligence (ECAI 2004), pp. 590–594 (1994)

    Google Scholar 

  24. Fukunaga, A.S.: Automated discovery of composite sat variable-selection heuristics. In: AAAI/IAAI, pp. 641–648 (2002)

    Google Scholar 

  25. Fukunaga, A.S.: Automated discovery of local search heuristics for satisfiability testing. Evol. Comput. 16(1), 31–61 (2008)

    Article  Google Scholar 

  26. Gagliolo, M., Schmidhuber, J.: Learning dynamic algorithm portfolios. Ann. of Math. and Artif. Intell. 47(3-4), 295–328 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  27. Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. Series of Books in the Mathematical Sciences. W. H. Freeman, New York (1979)

    MATH  Google Scholar 

  28. Geiger, C.D., Uzsoy, R., Aytug, H.: Rapid modeling and discovery of priority dispatching rules: An autonomous learning approach. J. of Sched. 9(1), 7–34 (2006)

    Article  MATH  Google Scholar 

  29. Hart, E., Ross, P., Nelson, J.: Solving a real-world problem using an evolving heuristically driven schedule builder. Evol. Comput. 6(1), 61–80 (1998)

    Article  Google Scholar 

  30. Hoos, H.H., Sttzle, T.: Stochastic Local Search: Foundations and Applications. Morgan Kaufmann / Elsevier (2005)

    Google Scholar 

  31. Huberman, B.A., Lukose, R.M., Hogg, T.: An economics approach to hard computational problems. Sci. 275, 51–54 (1997)

    Article  Google Scholar 

  32. Hutter, F., Hamadi, Y., Hoos, H.H., Leyton-Brown, K.: Performance prediction and automated tuning of randomized and parametric algorithms. In: Benhamou, F. (ed.) CP 2006. LNCS, vol. 4204, pp. 213–228. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  33. Hutter, F., Hoos, H.H., Stützle, T.: Automatic algorithm configuration based on local search. In: AAAI, pp. 1152–1157. AAAI Press, Menlo Park (2007)

    Google Scholar 

  34. Jakob, W.: HyGLEAM – an approach to generally applicable hybridization of evolutionary algorithms. In: Guervós, J.J.M., Adamidis, P.A., Beyer, H.-G., Fernández-Villacañas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 527–536. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  35. Jakob, W.: Towards an adaptive multimeme algorithm for parameter optimisation suiting the engineers’ needs. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 132–141. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  36. Johnson, D., Demers, A., Ullman, J., Garey, M., Graham, R.: Worst-case performance bounds for simple one-dimensional packaging algorithms. SIAM J. on Comput. 3(4), 299–325 (1974)

    Article  MathSciNet  Google Scholar 

  37. Keller, R.E., Poli, R.: Linear genetic programming of parsimonious metaheuristics. In: Srinivasan, D., Wang, L. (eds.) 2007 IEEE Congress on Evolutionary Computation, pp. 4508–4515. IEEE Computational Intelligence Society/IEEE Press, Singapore (2007)

    Chapter  Google Scholar 

  38. Koza, J.R.: Genetic Programming: on the Programming of Computers by Means of Natural Selection. The MIT Press, Boston (1992)

    MATH  Google Scholar 

  39. Koza, J.R.: Genetic Programming II: Automatic Discovery of Reusable Programs. The MIT Press, Cambridge (1994)

    MATH  Google Scholar 

  40. Koza, J.R., Bennett III, F.H., Andre, D., Keane, M.A.: Genetic Programming III: Darwinian Invention and Problem solving. Morgan Kaufmann, San Francisco (1999)

    MATH  Google Scholar 

  41. Koza, J.R., Keane, M.A., Streeter, M.J., Mydlowec, W., Yu, J., Lanza, G.: Genetic Programming IV: Routine Human-Competitive Machine Intelligence (Genetic Programming). Springer, Heidelberg (2003)

    MATH  Google Scholar 

  42. Koza, J.R., Poli, R.: Genetic programming. In: Burke, E.K., Kendall, G. (eds.) Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, pp. 127–164. Springer, Boston (2005)

    Google Scholar 

  43. Krasnogor, N., Gustafson, S.: A study on the use of ’self-generation’ in memetic algorithms. Nat. Comput. 3(1), 53–76 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  44. Krasnogor, N., Smith, J.E.: Emergence of profitable search strategies based on a simple inheritance mechanism. In: Proceedings of the 2001 Genetic and Evolutionary Computation Conference, pp. 432–439. Morgan Kaufmann, San Francisco (2001)

    Google Scholar 

  45. Mockus, J.: Application of bayesian approach to numerical methods of global and stochastic optimization. J. of Glob. Optim. 4(4), 347–366 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  46. Nareyek, A.: Choosing search heuristics by non-stationary reinforcement learning. In: Resende, M.G.C., de Sousa, J.P. (eds.) Metaheuristics: Computer Decision-Making, ch. 9, pp. 523–544. Kluwer, Dordrecht (2003)

    Google Scholar 

  47. Ong, Y.S., Keane, A.J.: Meta-lamarckian learning in memetic algorithms. IEEE Trans. on Evol. Comput. 8, 99–110 (2004)

    Article  Google Scholar 

  48. Ong, Y.S., Lim, M.H., Zhu, N., Wong, K.W.: Classification of adaptive memetic algorithms: a comparative study. IEEE Trans. on Syst. Man and Cybern. Part B 36(1), 141–152 (2006)

    Article  Google Scholar 

  49. Ozcan, E., Bilgin, B., Korkmaz, E.E.: A comprehensive survey of hyperheuristics. Intell. Data Anal. 12(1), 3–23 (2008)

    Google Scholar 

  50. Poli, W.B.R., Langdon, N.F.M.: A Field Guide to Genetic Programming. Lulu Enterprises, UK (2008)

    Google Scholar 

  51. Rhee, W.T., Talagrand, M.: On line bin packing with items of random size. Math. Oper. Res. 18, 438–445 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  52. Ropke, S., Pisinger, D.: A unified heuristic for a large class of vehicle routing problems with backhauls. Eur. J. of Oper. Res. 171(3), 750–775 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  53. Ross, P.: Hyper-heuristics. In: Burke, E.K., Kendall, G. (eds.) Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, ch. 17, pp. 529–556. Springer, Heidelberg (2005)

    Google Scholar 

  54. Ross, P., Marin-Blazquez, J.G., Schulenburg, S., Hart, E.: Learning a procedure that can solve hard bin-packing problems: A new ga-based approach to hyper-heuristics. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2003, pp. 1295–1306. Springer, Heidelberg (2003)

    Google Scholar 

  55. Seiden, S.S.: On the online bin packing problem. J. ACM 49(5), 640–671 (2002)

    Article  MathSciNet  Google Scholar 

  56. Smith, J.E.: Co-evolving memetic algorithms: A review and progress report. IEEE Trans. in Syst. Man and Cybern. Part B 37(1), 6–17 (2007)

    Article  Google Scholar 

  57. Stephenson, M., O’Reilly, U., Martin, M., Amarasinghe, S.: Genetic programming applied to compiler heuristic optimisation. In: Proceedings of the Eur. Conference on Genetic Programming, pp. 245–257. Springer, Heidelberg (2003)

    Google Scholar 

  58. Terashima-Marin, H., Flores-Alvarez, E.J., Ross, P.: Hyper-heuristics and classifier systems for solving 2D-regular cutting stock problems. In: Beyer, H.G., O’Reilly, U.M. (eds.) Proceedings of Genetic and Evolutionary Computation Conference, GECCO 2005, Washington DC, USA, June 25-29, pp. 637–643. ACM, New York (2005)

    Chapter  Google Scholar 

  59. Terashima-Marin, H., Ross, P., Valenzuela-Rendon, M.: Evolution of constraint satisfaction strategies in examination timetabling. In: Proc. of the Genetic and Evolutionary Computation Conf. GECCO 1999, pp. 635–642. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  60. Thrun, S., Pratt, L.: Learning to learn: Introduction and overview. In: Thrun, S., Pratt, L. (eds.) Learning To Learn. Kluwer Academic Publishers, Dordrecht (1998)

    Google Scholar 

  61. Vazquez-Rodriguez, J.A., Petrovic, S., Salhi, A.: A combined meta-heuristic with hyper-heuristic approach to the scheduling of the hybrid flow shop with sequence dependent setup times and uniform machines. In: Proceedings of the 3rd Multidisciplinary International Scheduling Conference: Theory and Applications (MISTA 2007), pp. 506–513 (2007)

    Google Scholar 

  62. Wah, B.W., Ieumwananonthachai, A.: Teacher: A genetics-based system for learning and for generalizing heuristics. In: Yao, X. (ed.) Evol. Comput., pp. 124–170. World Scientific Publishing Co. Pte. Ltd, Singapore (1999)

    Google Scholar 

  63. Wah, B.W., Ieumwananonthachai, A., Chu, L.C., Aizawa, A.: Genetics-based learning of new heuristics: Rational scheduling of experiments and generalization. IEEE Trans. on Knowl. and Data Eng. 7(5), 763–785 (1995)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, University of Nottingham, Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, UK

    Edmund K. Burke, Mathew R. Hyde, Graham Kendall, Gabriela Ochoa & Ender Ozcan

  2. University of Nottingham, 199 Taikang East Road, Ningbo, 315100, China

    John R. Woodward

Authors
  1. Edmund K. Burke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathew R. Hyde
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Graham Kendall
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gabriela Ochoa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ender Ozcan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. John R. Woodward
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computer Science, Cardiff University , 5 The Parade,Roath, CF24 3AA, Cardiff, UK

    Christine L. Mumford

  2. University of South Australia Adelaide , Mawson Lakes Campus, South Australia, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Burke, E.K., Hyde, M.R., Kendall, G., Ochoa, G., Ozcan, E., Woodward, J.R. (2009). Exploring Hyper-heuristic Methodologies with Genetic Programming. In: Mumford, C.L., Jain, L.C. (eds) Computational Intelligence. Intelligent Systems Reference Library, vol 1. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01799-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01799-5_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01798-8

  • Online ISBN: 978-3-642-01799-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation