ABSTRACT
While support vector machines (SVMs) have shown great promise in supervised classification problems, researchers have had to rely on expert domain knowledge when choosing the SVM's kernel function. This project seeks to replace this expert with a genetic programming (GP) system. Using strongly typed genetic programming and principled kernel closure properties, we introduce a new algorithm, called KGP, which finds near-optimal kernels. The algorithm shows wide applicability, but the combined computational overhead of GP and SVMs remains a major unresolved issue.
- G. H. Baker, L. Bottou, and J. Weston. Breaking SVM complexity with cross-training. In L. Saul, Y. Weiss, and L. Bottou, editors, Proceedings of Advances in Neural Information Processing Systems, pages 81--88, 2005.Google Scholar
- C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.Google Scholar
- O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters forsupport vector machines. Machine Learning, 46:131--159, 2002. Google ScholarDigital Library
- K. Crammer, J. Keshet, and Y. Singer. Kernel design using boosting. In S. Becker, S. Thrun, and K. Obermayer, editors, Proceedings of Advanced in Neural Information Processing Systems, pages 537--544, Vancouver, Canada, 2002. MIT Press.Google Scholar
- N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola. On kernel-target alignment. In T. Diettrich, S. Becker, and Z. Ghahramani, editors, Proceedings of Advanced in Neural Information Processing Systems, pages 367--373, Vancouver, Canada, 2001. MIT Press.Google Scholar
- B. F. de Souza, A. C. de Carvalho, R. Calvo, and R. P. Ishii. Multiclass SVM model selection using particle swarm optimization. In Proceedings of the Sixth International Conference on Hybrid Intelligent Systems, 2006. Google ScholarDigital Library
- C. B. D.J. Newman, S. Hettich and C. Merz. UCI repository of machine learning databases, 1998.Google Scholar
- F. Friedrichs and C. Igel. Evolutionary tuning of multiple SVM parameters. Neurocomputing, 64:107--117, 2005. Google ScholarDigital Library
- C. Gagne, M. Schoenauer, M. Sebag, and M. Tomassini. Genetic programming for kernel based learning with co-evolving subsets selection. In Proceedings of Parallel Problem Solving in Nature, 2006. Google ScholarDigital Library
- T. Howley and M. G. Madden. The genetic kernel support vector machine: Description and evaluation. Artificial Intelligence Review, 24(3 - 4):379--395, 2005. Google ScholarDigital Library
- C.-W. Hsu, C.-C. Chang, and C.-J. Lin. A practical guide to support vector classification. Technical report, Department of Computer Science, National Taiwan University, 2003.Google Scholar
- C.-L. Huang, M.-C. Chen, and C.-J. Wang. Credit scoring with a data mining approach based on support vector machines. Expert Systems with Applications, 2006. Google ScholarDigital Library
- C.-L. Huang and C.-J. Wang. A GA-based feature selection and parameter optimization for support vector machines. Expert Systems with Applications, 31:231 - 240, 2006.Google Scholar
- S. S. Keerthi. Efficient turing of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transactions on Neural Networks, 13(5):1225--1229, 2002. Google ScholarDigital Library
- S. Luke. ECJ 13: A java-based computation and genetic programming research system. http://cs.gmu.edu/~eclab/projects/ecj/, 2005.Google Scholar
- S. Luke and L. Panait. Lexicographic parsimony pressure. In W. B. Langdon, editor, Proceedings of the Genetic and Evolutionary Computation Conference, pages 829--836, 2002. Google ScholarDigital Library
- I. Mierswa. Evolutionary learning with kernels: A generic solution for large margin problems. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1553--1560, 2006. Google ScholarDigital Library
- D. J. Montana. Strongly typed genetic programming. Evolutionary Computation, 3(2):199--230, 11995. Google ScholarDigital Library
- C. S. Ong, A. J. Smola, and R. C. Williamson. Hyperkernels. In S. Becker, S. Thrun, and K. Obermayer, editors, Proceedings of Advanced in Neural Information Processing Systems, pages 478--485, Vancouver, Canada, 2002. MIT Press.Google Scholar
- T. Phienthrakul and B. Kijsirikul. Evolutionary strategies for multi-scale radial basis function kernels in support vector machines. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 905--911, Washington, DC, June 2005. ACM Press. Google ScholarDigital Library
- T. P. Runarsson and S. Sigurdsson. Asynchronous parallel evolutionary model selection for support vector machines. Neural Information Processing - Letters and Reviews, 3(3):59--67, June 2004.Google Scholar
- B. Schölkopf and A. J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2002.Google ScholarDigital Library
- B. Schölkopf, A. J. Smola, R. C. Williamson, and P. L. Bartlett. New support vector algorithms. Neural Computation, 12:1207--1245, 2000. Google ScholarDigital Library
- J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University, 2004. Google ScholarDigital Library
- S. Sonnenburg, R. Gunnar, and S. Christin. A general and efficient multiple kernel learning algorithm. In Y. Weiss, B. Schölkopf, and J. Platt, editors,Proceedings of Advances in Neural Information Processing Systems, pages 1273--1280, Vancouver, Canada, 2005. MIT Press.Google Scholar
- I. Steinwart. Sparseness of support vector machines - some asymptotically sharp bounds. In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in Neural Information Processing Systems, volume 16, 2004.Google Scholar
Index Terms
Evolving kernels for support vector machine classification
Recommendations
Extending twin support vector machine classifier for multi-category classification problems
Twin support vector machine classifier TWSVM was proposed by Jayadeva et al., which was used for binary classification problems. TWSVM not only overcomes the difficulties in handling the problem of exemplar unbalance in binary classification problems, ...
Self-Universum support vector machine
In this paper, for an improved twin support vector machine (TWSVM), we give it a theoretical explanation based on the concept of Universum and then name it Self-Universum support vector machine (SUSVM). For the binary classification problem, SUSVM takes ...
Twin Support Vector Machines for Pattern Classification
We propose Twin SVM, a binary SVM classifier that determines two nonparallel planes by solving two related SVM-type problems, each of which is smaller than in a conventional SVM. The Twin SVM formulation is in the spirit of proximal SVMs via generalized ...
Comments