Skip to main content
Log in

Grammar-based generation of variable-selection heuristics for constraint satisfaction problems

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

We propose a grammar-based genetic programming framework that generates variable-selection heuristics for solving constraint satisfaction problems. This approach can be considered as a generation hyper-heuristic. A grammar to express heuristics is extracted from successful human-designed variable-selection heuristics. The search is performed on the derivation sequences of this grammar using a strongly typed genetic programming framework. The approach brings two innovations to grammar-based hyper-heuristics in this domain: the incorporation of if-then-else rules to the function set, and the implementation of overloaded functions capable of handling different input dimensionality. Moreover, the heuristic search space is explored using not only evolutionary search, but also two alternative simpler strategies, namely, iterated local search and parallel hill climbing. We tested our approach on synthetic and real-world instances. The newly generated heuristics have an improved performance when compared against human-designed heuristics. Our results suggest that the constrained search space imposed by the proposed grammar is the main factor in the generation of good heuristics. However, to generate more general heuristics, the composition of the training set and the search methodology played an important role. We found that increasing the variability of the training set improved the generality of the evolved heuristics, and the evolutionary search strategy produced slightly better results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The real-world benchmark problems used can be found at http://www.cril.univ-artois.fr/lecoutre/benchmarks.html, under the names ‘RLFAP-graphs’ and ‘jobShop-e0ddr1’

References

  1. D. Achlioptas, M.S.O. Molloy, L.M. Kirousis, Y.C. Stamatiou, E. Kranakis, D. Krizanc, Random Constraint Satisfaction: A More Accurate Picture. Constraints. 6(4), 329–344 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  2. M. Bader-El-Den, R. Poli, A GP-based hyper-heuristic framework for evolving 3-SAT heuristics, in Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO ’7 (2007), p. 1749

  3. M. Bader-El-Den, R. Poli, S. Fatima, Evolving timetabling heuristics using a grammar-based genetic programming hyper-heuristic framework. Memet. Comput. 1(3), 205–219 (2009)

    Article  Google Scholar 

  4. S. Bain, J. Thornton, A. Sattar, Evolving algorithms for constraint satisfaction, in Congress on Evolutionary Computation (2004), pp. 265–272

  5. S. Bain, J. Thornton, A. Sattar, Methods of automatic algorithm generation, in PRICAI 2004: Trends in Artificial Intelligence (2004), pp. 1–10

  6. E.B. Baum, Iterated descent: a better algorithm for local search in combinatorial optimization problems. Technical report (Caltech, Pasadena, CA, 1986)

  7. J. Berlier, J. McCollum, A constraint satisfaction algorithm for microcontroller selection and pin assignment, in Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon) (2010), pp. 348–351

  8. S.C. Brailsford, C.N. Potts, B.M. Smith, Constraint satisfaction problems: algorithms and applications. Eur. J. Oper. Res. 119(3), 557–581 (1999)

    Article  MATH  Google Scholar 

  9. D. Brelaz, New methods to colour the vertices of a graph. Commun. ACM 22, 251–256 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  10. E.K. Burke, T. Curtois, M. Hyde, G. Kendall, G.a. Ochoa, S. Petrovic, J.A. Vázquez-Rodríguez, M. Gendreau, Iterated local search vs. hyper-heuristics: Towards general-purpose search algorithms. IEEE Congress on Evolutionary Computation. Barcelona (2010)

  11. E.K. Burke, M. Gendreau, M. Hyde, G. Kendall, G. Ochoa, E. Ozcan, R. Qu, Hyper-heuristics: a survey of the state of the art. J. Oper. Res. Soc 64(12), 1695–1724 (2013). doi:10.1057/jors.2013.71

    Article  Google Scholar 

  12. E.K. Burke, M. Hyde, G. Kendall, G. Ochoa, E. Özcan, J. Woodward, Exploring hyper-heuristic methodologies with genetic programming, in Computational Intelligence: Collaboration, Fusion and Emergence, Intelligent Systems Reference Library, ed. by C. Mumford, L. Jain (Springer, Berlin, 2009), pp. 177–201

    Chapter  Google Scholar 

  13. E.K. Burke, M.R. Hyde, G. Kendall, S. Member, Grammatical evolution of local search heuristics. Trans. Evol. Comput. 16(3), 406–417 (2012)

    Article  Google Scholar 

  14. E.K. Burke, M.R. Hyde, G. Kendall, G. Ochoa, E. Özcan, J. Woodward, A Classification of Hyper-heuristic Approaches. International Series in Operations Research & Management Science 146, pp. 449–468 . Springer US (2010). doi:10.1007/978-1-4419-1665-5_15

  15. R. Dechter, I. Meiri, Experimental evaluation of preprocessing algorithms for constraint satisfaction problems. Artif. Intell. 38(2), 211–242 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  16. N. Dunkin, S. Allen, Frequency assignment problems: representations and solutions. Technical report CSD-TR-97-14, (University of London, 1997)

  17. J. Gaschnig, Experimental case studies of backtrack versus waltz-type versus new algorithms for satisfying assignment problems, in Second Biennial Conference of the Canadian Society for Computational Studies of Intelligence, ed. by C.I.P. Society, Toronto (1978)

  18. I. Gent, E. MacIntyre, P. Prosser, B. Smith, T. Walsh, An empirical study of dynamic variable ordering heuristics for the constraint satisfaction problem, in Proceedings of the International Conference on Principles and Practice of Constraint Programming (CP’96) (1996), pp. 179–193

  19. R.M. Haralick, G.L. Elliott, Increasing tree search efficiency for constraint satisfaction problems. Artif. Intell. 14(3), 263–313 (1980)

    Article  Google Scholar 

  20. P. Hell, J. Nesetril, Colouring, constraint satisfaction, and complexity. Comput. Sci. Rev. 2(3), 143–163 (2008)

    Article  MATH  Google Scholar 

  21. R. Keller, R. Poli, Linear genetic programming of parsimonious metaheuristics, in IEEE Congress on Evolutionary Computation 2007. CEC 2007 (2007), pp. 4508–4515

  22. R.E. Keller, R. Poli, Self-adaptive hyperheuristic and greedy search, in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) (2008), pp. 3801–3808

  23. V. Kumar, Algorithms for constraint satisfaction problems: a survey. AI Mag. 13(1), 32–44 (1992)

    Google Scholar 

  24. H.R. Lourenco, O. Martin, T. Stutzle, Iterated local search. Technical report (Kluwer Academic, Norwell, 2002)

  25. A. Mackworth, Consistency in networks of relations. Artif. Intell. 8(1), 99–118 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  26. O. Martin, S.W. Otto, E.W. Felten, Large-step Markov chains for the traveling salesman problem. Complex Syst. 5(3), 299–326 (1991)

    MathSciNet  MATH  Google Scholar 

  27. R.I. McKay, N.X. Hoai, P.A. Whigham, Y. Shan, M. O’Neill, Grammar-based genetic programming: a survey. Genet. Program. Evol. Mach. 11(3–4), 365–396 (2010)

    Article  Google Scholar 

  28. S. Minton, An analytic learning system for specializing heuristics, in IJCAI (1993), pp. 922–928

  29. S. Minton, M.D. Johnston, A.B. Philips, P. Laird, Minimizing conflicts: a heuristic repair method for constraint satisfaction and scheduling problems. Artif. Intell. 58, 161–205 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  30. M. Mouhoub, B. Jafari, Heuristic techniques for variable and value ordering in csps, in Proceedings of the 13th annual conference on Genetic and evolutionary computation-GECCO ’11 (2011), p. 457

  31. J.C. Ortiz Bayliss, Exploring hyper-heuristic approaches for solving constraint satisfaction problems. Ph.D. thesis, Tecnológico de Monterrey (2011)

  32. J.C. Ortiz-Bayliss, J.H. Moreno-Scott, H. Terashima-Marín, Automatic generation of heuristics for constraint satisfaction problems, in International Workshop on Nature Inspired Cooperative Strategies for Optimization (NICSO 2014) (2014), pp. 1–14

  33. J.C. Ortiz-Bayliss, E. Özcan, A.J. Parkes, H. Terashima-Marin, Mapping the performance of heuristics for constraint satisfaction, in CEC’10: Proceedings of the Congress on Evolutionary Computation IEEE (2010)

  34. J.C. Ortiz-Bayliss, A.J. Parkes, H. Terashima-Marín, A genetic programming hyper-heuristic: turning features into heuristics for constraint satisfaction, in Workshop on Computational Intelligence (2013)

  35. J.C. Ortiz-Bayliss, H. Terashima-Marín, S.E. Conant-Pablos, Learning vector quantization for variable ordering in constraint satisfaction problems. Pattern Recognit. Lett. 34(4), 423–432 (2013)

    Article  Google Scholar 

  36. F.J. Ovalle-Martínez, J. Solano-González, I. Stojmenovic, A parallel hill climbing algorithm for pushing dependent data in clients–providers–servers systems. Mob. Netw. Appl. 9, 257–264 (2004)

    Article  Google Scholar 

  37. G. Pappa, G. Ochoa, M. Hyde, A. Freitas, J. Woodward, J. Swan, Contrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms. Genet. Program. Evol. Mach. 15(1), 3–35 (2014). doi:10.1007/s10710-013-9186-9

    Article  Google Scholar 

  38. P.W. Purdom, Search rearrangement backtracking and polynomial average time. Artif. Intell. 21, 117–133 (1983)

    Article  Google Scholar 

  39. N. Sabar, M. Ayob, G. Kendall, R. Qu, Grammatical evolution hyper-heuristic for combinatorial optimization problems. IEEE Trans. Evol. Comput. 17(6), 840–861 (2013). doi:10.1109/TEVC.2013.2281527

    Article  Google Scholar 

  40. J.A. Soria-Alcaraz, G. Ochoa, J. Swan, Effective learning hyper-heuristics for the course timetabling problem. Eur. J. Oper. Res. 238(1), 77–86 (2014)

    Article  MathSciNet  Google Scholar 

  41. A. Sosa-Ascencio, H. Terashima-Marín, M. Valenzuela-Rendón, Grammar-based genetic programming for evolving variable ordering heuristics, in IEEE Congress on Evolutionary Computation (2013), pp. 1154–1161

  42. L. Spector, Towards Practical Autoconstructive Evolution: Self-Evolution of Problem-Solving Genetic Programming Systems, vol. 8 (Springer, Berlin, 2010)

    Google Scholar 

  43. H. Terashima-Marín, J.C. Ortiz-Bayliss, P. Ross, M. Valenzuela-Rendón, Using hyper-heuristics for the dynamic variable ordering in binary constraint satisfaction problems, in 7th Mexican International Conference on Artificial Intelligence, Lecture Notes in Computer Science, vol. 5317, ed. by A. Gelbukh, E. Morales (Springer, Berlin, 2008), pp. 407–417

  44. J.D. Walker, G. Ochoa, M. Gendreau, E.K. Burke, Vehicle routing and adaptive iterated local search within the hyflex hyper-heuristic framework, in Learning and Intelligent Optimization, Lecture Notes in Computer Science ed. by Y. Hamadi, M. Schoenauer (Springer, Berlin, 2012), pp. 265–276

  45. R. Wallace, Analysis of heuristic synergies, in Recent Advances in Constraints, Lecture Notes in Computer Science, vol. 3978, ed. by B. Hnich, M. Carlsson, F. Fages, F. Rossi (Springer, Berlin, 2006), pp. 73–87

    Chapter  Google Scholar 

  46. P.A. Whigham, Grammatically-based genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, ed. by J.P. Rosca (Tahoe City, CA, 1995), pp. 33–41

  47. C.P. Williams, T. Gogg, Using deep structure to locate hard problems, in Proceedings of AAAI (1992), pp. 472–477

  48. K. Xu, W. Li, Exact phase transitions in random constraint satisfaction problems. J. Artif. Intell. Res. 12, 93–103 (2000)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This research was supported in part by Tecnológico de Monterrey under the strategic project PRY075 and the Research Group with Strategic Focus in Intelligent Systems, and the CONACyT Projects (Basic Science) under Grants 99695 and 241461. G. Ochoa acknowledges funding from the Engineering and Physical Sciences Research Council, UK (EPSRC) Grant Number EP/J017515.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alejandro Sosa-Ascencio.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sosa-Ascencio, A., Ochoa, G., Terashima-Marin, H. et al. Grammar-based generation of variable-selection heuristics for constraint satisfaction problems. Genet Program Evolvable Mach 17, 119–144 (2016). https://doi.org/10.1007/s10710-015-9249-1

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-015-9249-1

Keywords

Navigation