GP-based secondary classifiers
Introduction
In this paper, we present a new technique to improve the recognition accuracy of a digit recognizer without significantly increasing the computational effort. It involves two levels of classification. The primary classification is the regular run of a K-nearest-neighbor template-based recognizers for the BHA data set and genetic programming (GP)-based recognizers for the NIST digit data set. It is the secondary level of classification that is the focus of this article.
The approach we propose has some similarities with the method of serial combination of classifiers. Serial classifiers combination has emerged as a new direction in pattern recognition with the potential to improve classification performance by combining the strengths and overcoming the limitations of individual classifiers. When multiple classifiers or experts are available, recognition performance can be improved by combining classifiers at the decision level. The parallel combination of classifiers has received considerable attention in recent times. Voting schemes, rank-based schemes such as Borda count, and theories of evidence combination such as Bayes, logistic regression, fuzzy integrals and Dempster–Shafer have been applied to combine classifiers at the abstract, rank and measurement levels [1], [2].
In contrast, the serial combination of classifiers has not received much systematic study. In the typical serial combination of two classifiers, a fast classifier—the reducer—is used to reduce or filter the lexicon as a preprocessing step for a better and generally slower classifier—the recognizer. Serial combinations have been used in the past primarily for reasons of efficiency.
The motivation for our two-stage classification strategy is presented in Fig. 1. For example, the question could be as follows: Which of the four digits is the appropriate class label for the input image? Such a question is often posed in problems like zip-code recognition or check processing. What is called for here is not simply using the regular recognizer and looking for the confidence ranks of the numerals of interest but building a specific classifier which focuses on only those features which are useful in telling the numerals in the subclass apart. The subclasses can vary dynamically in an application.
Biologically inspired genetic techniques like GA and GP have been used previously for feature selection [3], [4]. These techniques strive to provide a global optima in terms of the feature subset required for a particular task of classification. We employ GP to provide us with secondary classifiers that are helpful in making a sharper decision when confusion between top choices from the first stage is very high.
Section snippets
Empirical analysis for secondary classification
Using a set of isolated digit images (BHA set), we analyzed the decisions of a standard K-nearest-neighbor recognizer. A study of the correct rate of the recognizer regardless of recognition confidence shows the confusion matrix in Table 1.
The second data set used is the NIST data set which contains isolated digit images. Table 2 shows the confusion matrix for this data set.
For the BHA test-set:
Total digit .
The top 1 hit ratio is 0.9809.
The hit ratio is 0.9913.
For the
Methodology
Recognition of handwritten characters by a computer has been a topic of intense research for many years [5]. Due to its pivotal role in many applications such as postal addresses interpretation, bank checks, tax forms and census forms reading, considerable research has been conducted in the area of handwritten character and numeral recognition [6].
Due to the nature of handwritten characters, there is probably no one single method that can achieve high recognition and reliability rates for all
Combination of GSC and GP classifiers for BHA set
There are two stages in the classification process. The first stage is a regular classifier based on GP or other methods like the K-nearest-neighbor technique. Each input to this classifier is a feature vector rejected by the first-level classifier. Since the first-level classification behaves differently on different classes, a rejection scheme using different thresholds on different classes is used. The design of the secondary classifier uses a set of GP trees each trained for a particular
Conclusion
In this paper, we proposed a multiple-stage classification algorithm for two data sets frequently used for building handwritten digit recognizers. The first-level classification routine decides simple patterns quickly and reliably. The confusing digit feature vectors are difficult to classify reliably at first stage. Hence such digits are presented to the secondary classifier routine. The secondary classifier uses a sub-set of the original feature set. It helps determine the correct output
Dr. Ankur Teredesai is an Assistant Professor of Computer Science and Director of the Laboratory for Discovery Informatics at the Rochester Institute of Technology. He received his B-Engg from the M.S. University, Baroda, India in 1997, and his Ph.D. degree in Computer Science and Engineering from the University at Buffalo, State University of New York in 2003.
References (11)
- et al.
Handwritten phrase recognition as applied to street name images
Pattern Recognition
(1998) - et al.
Methods of combining multiple classifiers and their applications to handwriting recognition
IEEE Trans. Syst. Man Cybern.
(1992) - et al.
Decision combination in multiple classifier systems
IEEE Trans. Pattern Anal. Mach. Intel.
(1994) - et al.
Feature subset selection using a genetic algorithm
IEEE Intel. Syst.
(1998) - A. Jain, D. Zongker, Feature selection: evaluation, application and small sample performance, IEEE Trans. Pattern Anal....
Cited by (10)
Linguistic cost-sensitive learning of genetic fuzzy classifiers for imprecise data
2011, International Journal of Approximate ReasoningTowards the experimental evaluation of novel supervised fuzzy adaptive resonance theory for pattern classification
2008, Pattern Recognition LettersVision and evolution: State of the art
2016, Natural Computing SeriesA minimum-risk genetic fuzzy classifier based on low quality data
2009, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)Genetic programming: An introduction and tutorial, with a survey of techniques and applications
2008, Studies in Computational IntelligenceEvolutionary computer vision
2007, Proceedings of GECCO 2007: Genetic and Evolutionary Computation Conference, Companion Material
Dr. Ankur Teredesai is an Assistant Professor of Computer Science and Director of the Laboratory for Discovery Informatics at the Rochester Institute of Technology. He received his B-Engg from the M.S. University, Baroda, India in 1997, and his Ph.D. degree in Computer Science and Engineering from the University at Buffalo, State University of New York in 2003.
Dr. Teredesai's research focuses on pattern recognition, data mining algorithms and data management in resource constrained environments geared towards discovery informatics. He is particularly interested in combined multimedia mining, semi-structured data sources (text, web, etc.) and novel applications and algorithms for these domains. He has co-authored several scientific papers and has two U.S. patents pending. He serves as a reviewer for the IEEE Transactions on Pattern Analysis and Machine Intelligence, the IEEE Transactions on Systems, Man, and Cybernetics (B), the IEEE Transactions on Evolutionary Computation and the IEEE Transactions for Knowledge and Data Engineering. He also serves on several national and international conference program committees. He recieved the Sigma-Xi Research Competition Award for excellence in research during the year 2001 for his dissertation thesis.