Skip to main content

Advertisement

Log in

Improving genetic search in XCS-based classifier systems through understanding the evolvability of classifier rules

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Learning classifier systems (LCSs), an established evolutionary computation technique, are over 30 years old with much empirical testing and foundations of theoretical understanding. XCS is a well-tested LCS model that generates optimal (i.e., maximally general and accurate) classifier rules in the final solution. Previous work has hypothesized the evolution mechanisms in XCS by identifying the bounds of learning and population requirements. However, no work has shown exactly how an optimum rule is evolved or especially identifies whether the methods within an LCS are being utilized effectively. In this paper, we introduce a method to trace the evolution of classifier rules generated in an XCS-based classifier system. Specifically, we introduce the concept of a family tree, termed parent-tree, for each individual classifier rule generated in the system during training, which describes the whole generational process for that classifier. Experiments are conducted on two sample Boolean problem domains, i.e., multiplexer and count ones problems using two XCS-based systems, i.e., standard XCS and XCS with code-fragment actions. The analysis of parent-trees reveals, for the first time in XCS, that no matter how specific or general the initial classifiers are, all the optimal classifiers are converged through the mechanism ‘be specific then generalize’ near the final stages of evolution. Populations where the initial classifiers were slightly more specific than the known ‘ideal’ specificity in the target solutions evolve faster than either very specific, ideal or more general starting classifier populations. Consequently introducing the ‘flip mutation’ method and reverting the conventional wisdom back to apply rule discovery in the match set has demonstrated benefits in binary classification problems, which has implications in using XCS for knowledge discovery tasks. It is further concluded that XCS does not directly utilize all relevant information or all breeding strategies to evolve the optimum solution, indicating areas for performance and efficiency improvement in XCS-based systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. We also adopted the code-fragment action-based approach in XCSRCFA (Iqbal et al. 2012) to compute continuous actions. Further, in XCSSMA (Iqbal et al. 2013b), cyclic graphs have been incorporated in the action of a classifier rule to evolve compact solutions that could solve any size problems in a number of important domains, such as parity problems.

  2. If the classifier population size grows larger than the specified limit, then one of the classifier rules has to be deleted so that the new rule can be inserted.

References

  • Altenberg L (1994) The evolution of evolvability in genetic programming. In: Advances in genetic programming. MIT Press, Massachusetts, pp 47–74

  • Behdad M, Barone L, French T, Bennamoun M (2012) On XCSR for electronic fraud detection. Evol Intell 5(2):139–150

    Article  Google Scholar 

  • Bull L (2004) Applications of learning classifier systems. Springer, Heidelberg

  • Bull L, Kovacs T (2005) Foundations of learning classifier systems: an introduction. Springer, Berlin

  • Butz MV (2006) Rule-based evolutionary online learning systems: a principal approach to lcs analysis and design. Springer, Berlin

    Google Scholar 

  • Butz MV, Kovacs T, Lanzi PL, Wilson SW (2004) Toward a theory of generalization and learning in XCS. IEEE Trans Evol Comput 8(1):28–46

    Article  Google Scholar 

  • Butz MV, Wilson SW (2002) An algorithmic description of XCS. Soft Comput 6(3–4):144–153

    Article  MATH  Google Scholar 

  • Carreira J, Li F, Sminchisescu C (2012) Object recognition by sequential figure-ground ranking. Int J Comput Vision 98(3):243–262

    Article  MathSciNet  Google Scholar 

  • Drugowitsch J (2008) Design and analysis of learning classifier systems: a probabilistic approach. Springer, Berlin

    Google Scholar 

  • Galván-López E, Poli R (2006) An empirical investigation of how and why neutrality affects evolutionary search. In: Proceedings of the genetic and evolutionary computation conference, pp 1149–1156

  • Hart E, Ross P (2001) GAVEL—a new tool for genetic algorithm visualization. IEEE Trans Evol Comput 5(4):335–348

    Article  Google Scholar 

  • Hemberg E, Berzan C, Veeramachaneni K, O’Reilly UM (2013) Introducing graphical models to analyze genetic programming dynamics. In: Proceedings of the twelfth workshop on foundations of genetic algorithms, pp 75–86

  • Holland JH, Booker LB, Colombetti M, Dorigo M, Goldberg DE, Forrest S, Riolo RL, Smith RE, Lanzi PL, Stolzmann W, Wilson SW (2000) What is a learning classifier system? In: Learning classifier systems, from foundations to applications. Springer, New York, pp 3–32

  • Hu T, Banzhaf W, Moore JH (2013) Robustness and evolvability of recombination in linear genetic programming. In: Genetic programming. Springer, New York, pp 97–108

  • Hu T, Payne JL, Banzhaf W, Moore JH (2012) Evolutionary dynamics on multiple scales: a quantitative analysis of the interplay between genotype, phenotype, and fitness in linear genetic programming. Genetic Program Evolv Mach 13(3):305–337

    Article  Google Scholar 

  • Ioannides C, Browne WN (2007) Investigating scaling of an abstracted LCS utilising ternary and S-expression alphabets. In: Proceedings of the genetic and evolutionary computation conference, pp 2759–2764

  • Iqbal M, Browne WN, Zhang M (2012) XCSR with computed continuous action. In: Proceedings of the Australasian joint conference on artificial intelligence, pp 350–361

  • Iqbal M, Browne WN, Zhang M (2013a) Evolving optimum populations with XCS classifier systems. Soft Comput 17(3):503–518

  • Iqbal M, Browne WN, Zhang M (2013b) Extending learning classifier system with cyclic graphs for scalability on complex, large-scale boolean problems. In: Proceedings of the genetic and evolutionary computation conference, pp 1045–1052

  • Iqbal M, Browne WN, Zhang M (2013c) Learning complex, overlapping and niche imbalance Boolean problems using XCS-based classifier systems. Evol Intell 6(2):73–91

  • Iqbal M, Browne WN, Zhang M (2013d) Reusing building blocks of extracted knowledge to solve complex, large-scale Boolean problems. IEEE Trans Evol Comput. doi:10.1109/TEVC.2013.2281537

  • Izquierdo EJ, Fernando CT (2008) The evolution of evolvability in gene transcription networks. Artif Life 11:265–273

    Google Scholar 

  • Kovacs T (1996) Evolving optimal populations with XCS classifier systems. Technical Report CSR-96-17 and CSRP-9617, University of Birmingham, UK

  • Poli R, Langdon WB, McPhee NF (2008) A field guide to genetic programming. http://lulu.com and freely available at http://www.gp-field-guide.org.uk (with contributions by J. R. Koza)

  • Shafi K, Kovacs T, Abbass HA, Zhu W (2009) Intrusion detection with evolutionary learning classifier systems. Nat Comput 8(1):3–27

    Article  MATH  MathSciNet  Google Scholar 

  • Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127

    Article  Google Scholar 

  • Wagner A (2008) Robustness and evolvability: a paradox resolved. Proc R Soc B 275(1630):91–100

    Article  Google Scholar 

  • Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175

    Article  Google Scholar 

  • Wilson SW (1998) Generalization in the XCS classifier system. In: Proceedings of the genetic programming conference, pp 665–674

  • Xie H, Zhang M (2013) Parent selection pressure auto-tuning for tournament selection in genetic programming. IEEE Trans Evol Comput 17(1):1–19

    Article  Google Scholar 

  • Xie H, Zhang M, Andreae P (2006) A study of good predecessor programs for reducing fitness evaluation cost in genetic programming. In: Proceedings of the IEEE congress on evolutionary computation, pp 2661–2668

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Muhammad Iqbal.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Iqbal, M., Browne, W.N. & Zhang, M. Improving genetic search in XCS-based classifier systems through understanding the evolvability of classifier rules. Soft Comput 19, 1863–1880 (2015). https://doi.org/10.1007/s00500-014-1369-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-014-1369-7

Keywords

Navigation