Skip to main content
SpringerLink
Log in
Book cover

Asia-Pacific Conference on Simulated Evolution and Learning

SEAL 2012: Simulated Evolution and Learning pp 320–330Cite as

A Genetic Programming Approach to Hyper-Heuristic Feature Selection

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7673))

Abstract

Feature selection is the task of finding a subset of original features which is as small as possible yet still sufficiently describes the target concepts. Feature selection has been approached through both heuristic and meta-heuristic approaches. Hyper-heuristics are search methods for choosing or generating heuristics or components of heuristics, to solve a range of optimisation problems. This paper proposes a genetic-programming-based hyper-heuristic approach to feature selection. The proposed method evolves new heuristics using some basic components (building blocks). The evolved heuristics act as new search algorithms that can search the space of subsets of features. The classification performance (accuracy) of classifiers are improved by using small subsets of features found by evolved heuristics.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Neshatian, K.: Feature Manipulation with Genetic Programming. PhD thesis, Victoria University of Wellington, New Zealand (2010)

    Google Scholar 

  2. Kira, K., Rendell, L.A.: The feature selection problem: Traditional methods and a new algorithm. In: Swartout, W.R. (ed.) pp. 129–134. AAAI, AAAI Press / The MIT Press (1992)

    Google Scholar 

  3. Burke, E.K., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., Woodward, J.: Exploring Hyper-heuristic Methodologies with Genetic Programming. In: Computational Intelligence: Collaboration, Fusion and Emergence, pp. 177–201. Springer (2009)

    Google Scholar 

  4. Blum, C., Roli, A.: Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Comput. Surv. 35(3), 268–308 (2003)

    Article  Google Scholar 

  5. Burke, E., Kendall, G.: Introduction. In: Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, pp. 5–18. Springer, Heidelberg (2003)

    Google Scholar 

  6. Almuallim, H., Dietterich, T.G.: Learning with many irrelevant features. In: Proceedings of the Ninth National Conference on Artificial Intelligence, pp. 547–552. AAAI Press (1991)

    Google Scholar 

  7. Whitney, A.W.: A direct method of nonparametric measurement selection. IEEE Trans. Comput. 20(9), 1100–1103 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  8. Caruana, R., Freitag, D.: Greedy attribute selection. In: Proceedings of the Eleventh International Conference on Machine Learning, pp. 28–36. Morgan Kaufmann (1994)

    Google Scholar 

  9. Gutlein, M., Frank, E., Hall, M., Karwath, A.: Large-scale attribute selection using wrappers. In: IEEE CIDM, pp. 332–339 (2009)

    Google Scholar 

  10. Vafaie, H., De Jong, K.A.: Genetic algorithms as a tool for feature selection in machine learning. In: ICTAI, pp. 200–203 (1992)

    Google Scholar 

  11. Xue, B., Zhang, M., Browne, W.N.: Single feature ranking and binary particle swarm optimisation based feature subset ranking for feature selection. In: Proceedings of the 35th Australasian Computer Science Conference, pp. 27–36 (2012)

    Google Scholar 

  12. Ross, P.: Hyper-heuristics. In: Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, pp. 529–556. Springer (2003)

    Google Scholar 

  13. Abdullah, S., Sabar, N.R., Nazri, M.Z.A., Turabieh, H., McCollum, B.: A constructive hyper-heuristics for rough set attribute reduction. In: IEEE ISDA, pp. 1032–1035 (2010)

    Google Scholar 

  14. Marill, T., Green, D.: On the effectiveness of receptors in recognition systems. IEEE Transactions on Information Theory 9(1), 11–17 (1963)

    Article  Google Scholar 

  15. Neshatian, K., Zhang, M., Andreae, P.: Genetic Programming for Feature Ranking in Classification Problems. In: Li, X., Kirley, M., Zhang, M., Green, D., Ciesielski, V., Abbass, H.A., Michalewicz, Z., Hendtlass, T., Deb, K., Tan, K.C., Branke, J., Shi, Y. (eds.) SEAL 2008. LNCS, vol. 5361, pp. 544–554. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  16. Neshatian, K., Zhang, M.: Genetic Programming for Feature Subset Ranking in Binary Classification Problems. In: Vanneschi, L., Gustafson, S., Moraglio, A., De Falco, I., Ebner, M. (eds.) EuroGP 2009. LNCS, vol. 5481, pp. 121–132. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  17. Burke, E.K., Hyde, M., Kendall, G., Ochoa, G., Ozcan, E., Woodward, J.R.: A classification of hyper-heuristics approaches. In: Gendreau, M., Potvin, J.Y. (eds.) Handbook of Metaheuristics, pp. 449–468. Springer (2010)

    Google Scholar 

  18. Fukunaga, A.S.: Automated discovery of local search heuristics for satisfiability testing. Evol. Comput. 16(1), 31–61 (2008)

    Article  Google Scholar 

  19. Burke, E.K., Hyde, M.R., Kendall, G., Woodward, J.: Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, pp. 1559–1565. ACM, New York (2007)

    Chapter  Google Scholar 

  20. Nguyen, S., Zhang, M., Johnston, M.: A genetic programming based hyper-heuristic approach for combinatorial optimisation. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, GECCO 2011, pp. 1299–1306. ACM, New York (2011)

    Chapter  Google Scholar 

  21. Smith, M.G., Bull, L.: Genetic programming with a genetic algorithm for feature construction and selection. Genetic Programming and Evolvable Machines 6, 265–281 (2005); Published online: August 17, 2005

    Article  Google Scholar 

  22. Oh, I.S., Lee, J.S., Moon, B.R.: Hybrid genetic algorithms for feature selection. IEEE Transactions on Pattern Analysis and Machine Intellignece, 1424–1437 (2004)

    Google Scholar 

  23. Yu, L., Liu, H.: Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research 5, 1205–1224 (2004)

    MATH  Google Scholar 

  24. Kohavi, R., John, G.: Wrappers for feature subset selection. Artificial Intelligence 97, 273–324 (1997)

    Article  MATH  Google Scholar 

  25. Frank, A., Asuncion, A.: UCI Machine Learning Repository (2010)

    Google Scholar 

  26. Luke, S.: ECJ 20 – A Java-based evolutionary computation research system

    Google Scholar 

  27. Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming – An Introduction; On the Automatic Evolution of Computer Programs and its Applications. Morgan Kaufmann, San Francisco (1998)

    MATH  Google Scholar 

  28. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explor. Newsl. 11, 10–18 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics, Statistics and Operations Research, Victoria University of Wellington, PO Box 600, Wellington, New Zealand

    Rachel Hunt

  2. School of Engineering and Computer Science, Victoria University of Wellington, PO Box 600, Wellington, New Zealand

    Mengjie Zhang

  3. Department of Computer Science and Software Engineering, University of Canterbury, Private Bag 4800, Christchurch, New Zealand

    Kourosh Neshatian

Authors
  1. Rachel Hunt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kourosh Neshatian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Information Technology, Le Quy Don Technical University, 100 Hoang Quoc Viet Street, Cau Giay District,, Hanoi, Vietnam

    Lam Thu Bui

  2. School of Computer Engineering, Nanyang Technological University, Block N4, 2b-39, Nanyang Avenue, 639798,, Singapore, Singapore

    Yew Soon Ong

  3. Faculty of Information Technology,, Hanoi University, km9 Nguyen Trai Road, Hanoi, Vietnam

    Nguyen Xuan Hoai

  4. Graduate School of Engineering, Department of Computer Science, Osaka Prefecture University, 1-1 Gakuen-cho, Nakaku, 599-8531, Sakai, Osaka, Japan

    Hisao Ishibuchi

  5. School of Electrical and Electronic Engineering, Nanyang Technological University, Block N4, 2b-39, Nanyang Avenue, 639798, Singapore, Singapore

    Ponnuthurai Nagaratnam Suganthan

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Hunt, R., Neshatian, K., Zhang, M. (2012). A Genetic Programming Approach to Hyper-Heuristic Feature Selection. In: Bui, L.T., Ong, Y.S., Hoai, N.X., Ishibuchi, H., Suganthan, P.N. (eds) Simulated Evolution and Learning. SEAL 2012. Lecture Notes in Computer Science, vol 7673. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34859-4_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34859-4_32

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34858-7

  • Online ISBN: 978-3-642-34859-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation