Elsevier

Pattern Recognition

Volume 38, Issue 4, April 2005, Pages 505-512
Pattern Recognition

GP-based secondary classifiers

https://doi.org/10.1016/j.patcog.2004.06.010Get rights and content

Abstract

Genetic programming (GP) is used to evolve secondary classifiers for disambiguating between pairs of handwritten digit images. The inherent property of feature selection accorded by GP is exploited to make sharper decision between conflicting classes. Classification can be done in several steps with an available feature set and a mixture of strategies. A two-step classification strategy is presented in this paper. After the first step of the classification using the full feature set, the high confidence recognition result will lead to an end of the recognition process. Otherwise a secondary classifier designed using a sub-set of the original feature set and the information available from the earlier classification step will help classify the input further. The feature selection mechanism employed by GP selects important features that provide maximum separability between classes under consideration. In this way, a sharper decision on fewer classes is obtained at the secondary classification stage. The full feature set is still available in both stages of classification to retain complete information. An intuitive motivation and detailed analysis using confusion matrices between digit classes is presented to describe how this strategy leads to improved recognition performance. In comparison with the existing methods, our method is aimed for increasing recognition accuracy and reliability. Results are reported for the BHA test-set and the NIST test-set of handwritten digits.

Introduction

In this paper, we present a new technique to improve the recognition accuracy of a digit recognizer without significantly increasing the computational effort. It involves two levels of classification. The primary classification is the regular run of a K-nearest-neighbor template-based recognizers for the BHA data set and genetic programming (GP)-based recognizers for the NIST digit data set. It is the secondary level of classification that is the focus of this article.

The approach we propose has some similarities with the method of serial combination of classifiers. Serial classifiers combination has emerged as a new direction in pattern recognition with the potential to improve classification performance by combining the strengths and overcoming the limitations of individual classifiers. When multiple classifiers or experts are available, recognition performance can be improved by combining classifiers at the decision level. The parallel combination of classifiers has received considerable attention in recent times. Voting schemes, rank-based schemes such as Borda count, and theories of evidence combination such as Bayes, logistic regression, fuzzy integrals and Dempster–Shafer have been applied to combine classifiers at the abstract, rank and measurement levels [1], [2].

In contrast, the serial combination of classifiers has not received much systematic study. In the typical serial combination of two classifiers, a fast classifier—the reducer—is used to reduce or filter the lexicon as a preprocessing step for a better and generally slower classifier—the recognizer. Serial combinations have been used in the past primarily for reasons of efficiency.

The motivation for our two-stage classification strategy is presented in Fig. 1. For example, the question could be as follows: Which of the four digits {1,4,7,9} is the appropriate class label for the input image? Such a question is often posed in problems like zip-code recognition or check processing. What is called for here is not simply using the regular recognizer and looking for the confidence ranks of the numerals of interest but building a specific classifier which focuses on only those features which are useful in telling the numerals in the subclass apart. The subclasses can vary dynamically in an application.

Biologically inspired genetic techniques like GA and GP have been used previously for feature selection [3], [4]. These techniques strive to provide a global optima in terms of the feature subset required for a particular task of classification. We employ GP to provide us with secondary classifiers that are helpful in making a sharper decision when confusion between top choices from the first stage is very high.

Section snippets

Empirical analysis for secondary classification

Using a set of isolated digit images (BHA set), we analyzed the decisions of a standard K-nearest-neighbor recognizer. A study of the correct rate of the recognizer regardless of recognition confidence shows the confusion matrix in Table 1.

The second data set used is the NIST data set which contains isolated digit images. Table 2 shows the confusion matrix for this data set.

  • For the BHA test-set:

  • Total digit images=12888.

  • The top 1 hit ratio is 0.9809.

  • The (top1+top2) hit ratio is 0.9913.

  • For the

Methodology

Recognition of handwritten characters by a computer has been a topic of intense research for many years [5]. Due to its pivotal role in many applications such as postal addresses interpretation, bank checks, tax forms and census forms reading, considerable research has been conducted in the area of handwritten character and numeral recognition [6].

Due to the nature of handwritten characters, there is probably no one single method that can achieve high recognition and reliability rates for all

Combination of GSC and GP classifiers for BHA set

There are two stages in the classification process. The first stage is a regular classifier based on GP or other methods like the K-nearest-neighbor technique. Each input to this classifier is a feature vector rejected by the first-level classifier. Since the first-level classification behaves differently on different classes, a rejection scheme using different thresholds on different classes is used. The design of the secondary classifier uses a set of GP trees each trained for a particular

Conclusion

In this paper, we proposed a multiple-stage classification algorithm for two data sets frequently used for building handwritten digit recognizers. The first-level classification routine decides simple patterns quickly and reliably. The confusing digit feature vectors are difficult to classify reliably at first stage. Hence such digits are presented to the secondary classifier routine. The secondary classifier uses a sub-set of the original feature set. It helps determine the correct output

Dr. Ankur Teredesai is an Assistant Professor of Computer Science and Director of the Laboratory for Discovery Informatics at the Rochester Institute of Technology. He received his B-Engg from the M.S. University, Baroda, India in 1997, and his Ph.D. degree in Computer Science and Engineering from the University at Buffalo, State University of New York in 2003.

References (11)

  • G. Kim et al.

    Handwritten phrase recognition as applied to street name images

    Pattern Recognition

    (1998)
  • L. Xu et al.

    Methods of combining multiple classifiers and their applications to handwriting recognition

    IEEE Trans. Syst. Man Cybern.

    (1992)
  • T. Ho et al.

    Decision combination in multiple classifier systems

    IEEE Trans. Pattern Anal. Mach. Intel.

    (1994)
  • J. Yang et al.

    Feature subset selection using a genetic algorithm

    IEEE Intel. Syst.

    (1998)
  • A. Jain, D. Zongker, Feature selection: evaluation, application and small sample performance, IEEE Trans. Pattern Anal....
There are more references available in the full text version of this article.

Cited by (10)

  • Vision and evolution: State of the art

    2016, Natural Computing Series
  • A minimum-risk genetic fuzzy classifier based on low quality data

    2009, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
  • Evolutionary computer vision

    2007, Proceedings of GECCO 2007: Genetic and Evolutionary Computation Conference, Companion Material
View all citing articles on Scopus

Dr. Ankur Teredesai is an Assistant Professor of Computer Science and Director of the Laboratory for Discovery Informatics at the Rochester Institute of Technology. He received his B-Engg from the M.S. University, Baroda, India in 1997, and his Ph.D. degree in Computer Science and Engineering from the University at Buffalo, State University of New York in 2003.

Dr. Teredesai's research focuses on pattern recognition, data mining algorithms and data management in resource constrained environments geared towards discovery informatics. He is particularly interested in combined multimedia mining, semi-structured data sources (text, web, etc.) and novel applications and algorithms for these domains. He has co-authored several scientific papers and has two U.S. patents pending. He serves as a reviewer for the IEEE Transactions on Pattern Analysis and Machine Intelligence, the IEEE Transactions on Systems, Man, and Cybernetics (B), the IEEE Transactions on Evolutionary Computation and the IEEE Transactions for Knowledge and Data Engineering. He also serves on several national and international conference program committees. He recieved the Sigma-Xi Research Competition Award for excellence in research during the year 2001 for his dissertation thesis.

View full text