Elsevier

Applied Soft Computing

Volume 51, February 2017, Pages 83-95
Applied Soft Computing

Genetic programming for evolving figure-ground segmentors from multiple features

https://doi.org/10.1016/j.asoc.2016.07.055Get rights and content

Highlights

  • A novel figure-ground segmentation system based on GP is proposed.

  • The proposed method can be successfully applied to diverse types of images.

  • Intensity based features are not sufficient for complex images.

  • Effectiveness of various features for GP to produce capable segmentors is studied.

Abstract

Figure-ground segmentation is a crucial preprocessing step for many image processing and computer vision tasks. Since different object classes need specific segmentation rules, the top-down approach, which learns from the object information, is more suitable to solve segmentation problems than the bottom-up approach. A problem faced by most existing top-down methods is that they require much human work/intervention, meanwhile introducing human bias. As genetic programming (GP) does not require users to specify the structure of solutions, we apply it to evolve segmentors that can conduct the figure-ground segmentation automatically and accurately. This paper aims to determine what kind of image information is necessary for GP to evolve capable segmentors (especially for images with high variations, e.g. varied object shapes or cluttered backgrounds). Therefore, seven different terminal sets are exploited to evolve segmentors, and images from four datasets (bitmap, Brodatz texture, Weizmann and Pascal databases), which are increasingly difficult for segmentation tasks, are selected for testing. Results show that the proposed GP based method can be successfully applied to diverse types of images. In addition, intensity based features are not sufficient for complex images, whereas features containing spectral and statistical information are necessary. Compared with four widely-used segmentation techniques, our method obtains consistently better segmentation performance.

Introduction

Figure-ground segmentation is a basic computer vision task, which aims to separate foreground objects or regions of interest from their backgrounds. The results of figure-ground segmentations can be the input to many higher-level tasks, e.g. object recognition, object tracking and image editing [1].

There are two ways to conduct figure-ground segmentation: bottom-up approach and top-down approach [2]. The former aims to segment the image into regions first, then recognize image regions that belong to objects or the background based on the image-based criteria (e.g. texture uniformity and continuity of contours). However, this approach is likely to segment an object to multiple parts, and merge the object and background parts together as they are similar based on the image-based criteria [2], [3], thus reducing the segmentation accuracy. In contrast, unlike the bottom-up approach, top-down approach does not rely on the image criteria, but learns from the prior knowledge of objects, such as object classes and shape fragments, to direct the figure-ground segmentation. As different types of images may contain different classes of objects, they require specific segmentation rules. Therefore, it is preferable to apply top-down approach that can learn rules from the object information provided by the training samples [2] rather than attempt to manually design rules.

Existing top-down figure-ground segmentation approach can be divided into two branches: model-based methods [4], [5], [6] and fragment-based methods [2], [7], [8]. Model-based methods include deformable templates, active shape models, active contour models, etc. They normally match a deformable model to an image by minimizing the image energy, which is a function of image features [9]. Fragment-based methods extract a fragment set in the training stage, which captures shape and appearance information of the common object parts. Each fragment's figure-ground segmentation is then generated and used to match objects in the testing stage. However, model-based methods require a lot of human intervention to locate the initial models, and only when the initial model is located close to the target is it likely to obtain accurate segmentation results. Fragment-based methods require much work from researchers to collect informative fragments. Furthermore, the more involvement of human work, the higher probability of involving human bias, which should be avoided as much as possible.

As an evolutionary computation technique, genetic programming (GP) can evolve computer programs to solve problems automatically, so users are not required to specify the form or structure of solutions [10]. If GP can be introduced to evolve segmentation algorithms, the GP based methods will require less human work than the model-based and fragment-based methods. Moreover, since GP can combine input features in complex non-linear forms, GP is more flexible and more likely to evolve better methods than those defined by experts [11].

Although GP has already been introduced in the area of image segmentation since the 1990s, only a limited number of related papers have been published. Based on these works, GP is used in two ways for image segmentation – one is to apply GP directly to evolve segmentors [11], [12], [13], [14], [15], [16]; the other is to combine GP with other conventional segmentation techniques (e.g. clustering methods [17]) and utilize GP as an optimization technique [17]. The first one is a pure GP approach and can evolve segmentors automatically, so our paper will also follow this approach.

The existing work, using GP to evolve segmentors, actually transforms the segmentation tasks into an extension of classification problems and evolves pixelwise classifiers. Poli [11] proposes a method using GP to evolve filters, based on which the pixelwise classification can be conducted to fulfill segmentation tasks. The terminal set contains average intensity values of small image regions, and the function set consists of simple arithmetic functions. The evolved algorithm is tested on several medical images, achieving much higher scores in sensitivity and specificity than an artificial neural network approach. The GP based segmentation method has also been utilized on texture segmentation [12], [13]. As the terminal set only includes raw pixel values, this method does not involve feature extraction, so it is efficient. The function set consists of arithmetic and logic operators (such as >=, <=, = =). The method can segment texture images accurately. A recent work [16] builds a GP based segmentation system adapted from works of Song et al. [12], [13]. It employs GP to evolve segmentors from local binary patterns (LBP) to segment the corpora lutea on 30 medical images.

Image operators can be added to the function set (such as dilation, thresholding and histogram equalization). Singh et al. [14] evolve Matlab programs from intensities and primitive image operators, which produce consistently good results on cell images. Roberts [15] employs not only arithmetic functions but also image processing operators. To tackle the problem of a high computational cost caused by image operators, a caching mechanism was introduced. The GP based method achieves 92.3% in sensitivity and 97.2% in specificity on mole images.

These works on GP based segmentation achieve promising results. However, there are still several drawbacks. Firstly, they are only tested on simple images in a limited number of domains, including texture images [12], [13], and medical images [11], [14], [15]. As the tested medical images all have clean backgrounds, they are considered as simple images. Therefore, whether GP-evolved segmentors can deal with a wide range of images, especially complex images with variations in background, is still not clear. Secondly, only intensity based features [12], [13], [14], [15], intensity statistics [11] and LBP features [16] have been used as input features to evolve segmentors. As the input of GP has a great influence on its performance [18], we assume that if the terminal set contains certain kinds of features known to suit target image domains (e.g. histogram statistics and Gabor features), better performing segmentors will be evolved. Thirdly, limited comparisons with other segmentation techniques have been made. GP-evolved segmentors have been compared only with neural networks [11] and a GA-based algorithm [14]. To test the effectiveness of segmentors, comparisons with the widely-used approaches, e.g. thresholding, clustering and region-based methods, are necessary.

This paper is an extension of our former work [19], which utilizes GP to evolve segmentors from three simple feature sets, i.e. intensities, histogram statistics and “histogram statistics + spatial moments + gradient statistics”. In this paper, we exploit more feature descriptors, which can extract all the three types of general image properties (color, texture and shape) consisting of seven feature sets. This is the first work that employs a wide range of image features to evolve figure-ground image segmentors by GP and investigates what kind of image information is necessary for GP to produce capable segmentors for difficult segmentation tasks (e.g. images with high variations in objects and/or backgrounds). The explicit objectives are as follows.

  • 1.

    Explore the effectiveness of different image features that are used to evolve segmentors by GP.

  • 2.

    Investigate whether the GP evolved segmentors can have a consistent performance across diverse ranges of images, which have different difficulty levels for the segmentation task.

  • 3.

    Investigate whether the GP based segmentation method can outperform widely-used conventional techniques.

The rest of this paper is organized as follows. In Section 2, the basic knowledge of GP is provided. Section 3 describes our GP based segmentation system and the seven feature descriptors selected to evolve segmentors. Section 4 introduces experiment preparations of test images and evaluation measures. Section 5 displays results obtained on four databases. In Section 6, further discussions about the evolved segmentors and comparisons with conventional segmentation techniques are made. Conclusions are drawn in Section 7.

Section snippets

Background – GP

GP is one of the evolutionary computation techniques, which are inspired by the biological evolution [10]. It can evolve a population of computer programs by transforming populations of programs into new, normally better, populations. Fig. 1 displays the flowchart of GP, and the basic steps of GP are described as follows.

  • 1.

    Initialize a GP population.

    This is the first step to perform a GP run, in which a variety of individuals are created randomly for the later evolution. There are three common

The GP based image segmentation system

According to the existing works [11], [12], [13], [14], [15], [16], which use GP to evolve algorithms, image segmentation is handled as a supervised classification problem at the pixel level. In Song's work [12], [13], a pixel-classification-based segmentation framework is developed, which is a common way to use GP in image-related tasks. Our paper also follows this framework. We exploit seven terminal sets to discover what kind of image information is useful to segment diverse types of images,

Experiment preparation

We select images from four databases (Section 3.1), i.e. bitmap patterns [13], Brodatz texture database [29], Weizmann horse database [30], [2] and Pascal Visual Object Classes 2012 (Pascal VOC2012) dataset [31], to test our GP based method. This paper aims to investigate whether the GP-evolved segmentors can deal with diverse types of images. These four datasets are selected as they are increasingly difficult for the segmentation task. Three types of evaluation methods (Section 3.2), the

Experiment results

This section tests the segmentors evolved from seven kinds of features on four datasets. For bitmap and texture images, there are two different patterns or textures in each image, so we choose one pattern as the object and the other one as background randomly. In addition to GP parameters, the size of the sliding window and its shifting step are also needed by the system, which are set based on our initial experimental tests. As the window captures sub-images, the window size must guarantee

Comparisons and analyses of segmentors

To find out the influence of input features on the performance of evolved segmentors, the results on Brodatz, Weizmann and Pascal images, are compared. In addition, we also compare the Gabor-based segmentors, which perform best, with other four widely-used segmentation techniques.

Conclusions

This paper developed a supervised figure-ground segmentation system, which utilizes GP to evolve segmentors automatically. The contributions of this work are shown as follows.

Firstly, this paper investigated seven types of image features, covering all three general feature categories (brightness, texture and shape), as the system inputs (terminal sets) to evolve segmentors. The effectiveness of different image features on the performance of evolved segmentors was studied. Results show that

References (41)

  • J. Ashburner et al.

    Unified segmentation

    Neuroimage

    (2005)
  • W. Zou et al.

    Online global transfer for automatic figure-ground segmentation

    IEEE Trans. Image Process.

    (2014)
  • E. Borenstein et al.

    Learning to segment

  • E. Sharon et al.

    Segmentation and boundary detection using multiscale intensity measurements

  • J. Liu et al.

    Application of snake model in medical image segmentation

    J. Converg. Inf. Technol.

    (2014)
  • C.-Y. Liu et al.

    Deformable templates guided discriminative models for robust 3D brain MRI segmentation

    Neuroinformatics

    (2013)
  • G. Gill et al.

    Robust initialization of active shape models for lung segmentation in CT scans: a feature-based atlas approach

    J. Biomed. Imaging

    (2014)
  • E. Borenstein et al.

    Class-specific, top-down segmentation

  • E. Borenstein et al.

    Combined top-down/bottom-up segmentation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2008)
  • M. Kass et al.

    Snakes: active contour models

    Int. J. Comput. Vis.

    (1988)
  • R. Poli et al.

    A Field Guide to Genetic Programming

    (2008)
  • R. Poli

    Genetic programming for image analysis

  • A. Song et al.

    Fast texture segmentation using genetic programming

  • A. Song et al.

    Texture segmentation by genetic programming

    Evol. Comput.

    (2008)
  • T. Singh et al.

    Genetic programming based image segmentation with applications to biomedical object detection

  • M.E. Roberts

    The effectiveness of cost based subtree caching mechanisms in typed genetic programming for image segmentation

    Applications of Evolutionary Computing

    (2003)
  • M. Dong et al.

    Automatic detection and segmentation of bovine corpora lutea in ultrasonographic ovarian images using genetic programming and rotation invariant local binary patterns

    Med. Biol. Eng. Comput.

    (2013)
  • J. Geng et al.

    Image texture classification using a multiagent genetic clustering algorithm

  • J.R. Koza
    (1992)
  • Y. Liang et al.

    A supervised figure-ground segmentation method using genetic programming

    Applications of Evolutionary Computation

    (2015)
  • Cited by (0)

    View full text