Elsevier

Neurocomputing

Volume 145, 5 December 2014, Pages 512-529
Neurocomputing

Multi-contour registration based on feature points correspondence and two-stage gene expression programming

https://doi.org/10.1016/j.neucom.2014.05.002Get rights and content

Abstract

Image registration is a fundamental task in 3D reconstruction from an image sequence. Although this topic has been studied for decades, a general, robust, and automatic image registration method is rare, and most existing image registration methods are designed for a particular application. In this paper, image registration is treated as a formula discovery problem. A novel contour registration pipeline was proposed based on a foot-point-based feature point correspondence algorithm and a two-stage evolutionary algorithm. Our proposal has three objectives. First, we introduce a novel feature point extraction method that uses estimation of the curvature and the support region for every contour in the floating image. Second, we approximate the reference contour using a Gaussian mixture model (GMM) continuous optimization algorithm followed by an order-preserved foot-point detection method used to extract the feature points that correspond to the feature points of the floating contours. Third, we propose a hybrid evolutionary algorithm used to identify the registration formula for the reference image and the floating image. The hybrid evolutionary algorithm is a two-stage algorithm based on gene expression programming (GEP) and the improved cooperative particle swarm optimizer (CPSO). The optimal or near-optimal structure is accomplished using the GEP algorithm, and the parameters embedded in the structure are optimized by an opposition based learning (OBL)-based cooperative particle swarm optimizer (CPSO). Compared with other non-rigid registration methods, the developed registration pipeline produces competitive results with high accuracy.

Introduction

Image registration is a fundamental task in 3D reconstruction that addresses the geometric alignment of a set of images. The set may consist of two or more digital images taken of a single scene at different times, from different sensors, or from different viewpoints [1]. Many methods have been proposed to address this problem. A popular approach involves treating the salient features of the image as invariant to find the geometric transformation [2]. Using a feature-based method, a number of relevant image features are first extracted from the two images, the correspondences between the feature points are subsequently identified, and a geometric matching transformation is used to provide the best match for the two sets of features.

The most common methods of feature extraction are the contour-based hierarchical method [3] and the scale invariant feature transform [4]. The general feature correspondence algorithms include the assignment algorithm [5], the graph-matching method [6], the speeded-up robust features approach [7], and the expectation conditional maximization algorithms [8]. The transformation parameters estimation method is carried out based on the detection of features that have undergone feature correspondence. In most existing feature-based techniques, the feature correspondence is still the most challenging problem.

The feature-points-based contour matching problem can be carried out as follows [9]: first, we extract a set of feature points from each object, e.g., by running an edge detector over each image and sampling from the edges. Second, we determine the pairs of corresponding features in the two feature sets. Third, we use the correspondence information to find an aligning transformation, such as the least squares transformation from a certain class.

An interesting subject in contour matching is a contour simplification that preserves the original characteristics of shape features. The feature points are those points that exhibit extreme values on the curve and that can suitably describe the curve for visual perception and recognition. The most obvious advantage of using feature points to represent the contours is the large effect on data reduction and its immediate impact on the efficiency of the subsequent contour-matching algorithms. As described in references [10], [11], there are three major categories of methods used to detect the feature points:

  • (1)

    Methods that search for feature points from the original contour scale or from a multi-scale contour representation using a significant measure other than curvature.

  • (2)

    Methods that evaluate the curvature by transforming the contour to the Gaussian scale space.

  • (3)

    Methods that search for feature points by directly estimating the curvature in the original picture space.

In this paper, an improved contour simplification method that uses a polygonal approximation is proposed. The new strategy is based on the method proposed by Wu [12], [13], which determines the region of support in finding the feature points.

The second stage of determining the point correspondences has been the subject of much research. Intuitively, one would expect an exact one-to-one correspondence between the reference contour and the floating contour. An increasingly popular approach involves building a cost matrix that records the dissimilarity between all possible pairs of points on the two shapes. The unconstrained optimal assignment problem is essentially determined by the cost matrix. If we can guarantee that all entries in the cost matrix are integral or rational numbers, the Hungarian algorithm is a good choice for arriving at one optimal solution with a finite number of iterations. Another category of solutions to the correspondence problem alternates the estimations of correspondence and transformation. The iterative closest point (ICP) algorithm [14] is the best known and most widely used among these methods, it uses the nearest-neighbor relationship to assign a binary correspondence at each step. Reference [15] enhanced this algorithm with two significant improvements: the soft-assign idea and the Robust Point Matching-Thin Plate Spline (RPMTPS) algorithm. Li et al. [16] presented an automatic approach based on multidimensional scaling to match the correspondences on 3-D human bodies in various postures, but their aim was to extract the feature points automatically. In reference [17], the problem of automatic determination of the point correspondence between two images was formulated as a multimodal function optimization, and genetic algorithms (GAs) were used to solve the optimization problem. Reference [18] proposed a method for non-rigid point-matching based on a shape context descriptor. The shape context describes the coarse distribution of the remainder of the shape with respect to a given point on the shape, parameterized by distance and angular extent with respect to the point described. The solution that minimizes the overall shape context distances becomes the optimal match between the two point sets. Although this approach has produced encouraging results in various application fields [19], [20], the neighboring points in one shape may be matched to two points that are far apart in the other shape because the method is defective in its spatial ordering constraints.

The third stage for finding aligning transformations can be treated as a problem of formula discovery [38]. Formula discovery aims to identify a formula that describes the relationship between the independent variable and dependent variable using a large number of test data. The common mathematical methods for resolving formula discovery are the curve-fitting method, the regression analysis method, approximation theory and genetic programming algorithms, among others. Genetic programming (GP) algorithms have certain advantages compared with other methods. In addition to the obvious benefit of automation, the GP provides power and flexibility that potentially allow for formula evolution.

Gene expression programming (GEP), as an extension of GP, is an evolutionary algorithm that automatically creates complex tree structures that learn and adapt by changing their sizes, shapes, and composition [21]. As the natural development of GA and GP, GEP combines the genotype of GA and the phenotype of GP. The genome of GEP consists of a linear, symbolic string or chromosome of fixed length composed of one or more genes of equal size. The chromosome can be evolved by mutation, recombination, transposition and so on [25]. All the genes of gene expression programming have the same size. However, these fixed length strings code for expression trees of different sizes. This means that the size of the coding regions varies from gene to gene, allowing for adaptation and evolution to occur smoothly. GEP overcomes many limitations of GA and GP, so it has been widely used in problem solving such as regression, classification, cluster etc.

In this study, we introduce a new model to address the formula discovery problem. Different from [24], the proposed model is a two-stage evolutionary algorithm based on GEP and an improved cooperative particle swarm optimizer. The optimal or near-optimal structure is found using the GEP algorithm, and the parameters embedded in the structure are optimized by an opposition-based-learning (OBL)-based cooperative particle swarm optimizer (CPSO). We refer to the two-stage evolutionary algorithm as CGEPSOBL. The two-stage evolutionary algorithm contains certain similarities to the flexible neural tree (FNT) algorithm, but the most distinctive feature of our algorithm is that CGEPSOBL can generate multi-expressions rather than a single expression.

This paper proposes a novel contour registration pipeline based on a foot-point-based feature point correspondence algorithm and a new evolutionary algorithm known as CGEPSOBL. At the feature point detection stage, we propose a novel feature point extraction method based on the signed discrete curvature calculation and the adaptive bending value estimation. At the stage of determining the pairs of feature points, we propose an integrated method that combines the feature point detection and pairs of features point determination. At the stage of finding the alignment transformation, the new model known as CGEPSOBL is used to identify the registration formula of the reference image and the floating image.

This paper contains two major contributions. First, we introduce an integrated method that detects the feature points of the reference image and matches the point sets simultaneously. Our approach is based on three-order B-spline approximation and foot-point detection. The classical methods commonly extract the feature points first and find the match afterwards. The drawback of the classical method is that the feature point detection and the matching are separated. An advantage of our method is that the feature point detection and the matching are integrated, and the method is sufficiently able to mine the information of the reference contours described by the B-spline curves and find the most appropriate feature points on the reference contours. Second, we propose a two-stage hybrid evolutionary algorithm, GEP is used to construct the basic structure because it can automatically establish a relational model between the given input variables and output variables. The parameters embedded in the basic structure are optimized by the OBL-based CPSO (CPSOBL) algorithm. Compared with the standard PSO, the CPSOBL algorithm contains higher diversity. The higher diversity ensures that the search space is searched more thoroughly and that the algorithm has a greater chance of reaching the global solution.

The remainder of the paper is organized as follows. The feature point detection and correspondence methods are described in Section 2. Section 3 presents the details of the proposed two-stage hybrid evolutionary algorithm. In Section 4, the results of experiments are discussed and compared with those of other methods. Section 5 concludes the paper.

Section snippets

Feature points detection and correspondence

The correspondence problem is usually based on two sets of feature points. The correspondence result of this type of method depends heavily on the inherit similarity of the two point sets. In this paper, a new algorithm is proposed to address the correspondence problem. The algorithm includes three steps. First, for every contour in the floating image, calculate its similarity to every contour in the reference image. Based on the cost matrix constituted by the similarities, we can get the one

Formula discovery based on a two-stage hybrid evolutionary algorithm

In this study, we propose a new evolutionary model to address the formula discovery problem. The evolutionary model first evolves an optimal or near-optimal basic expression using the GEP algorithm and subsequently finds the optimal parameter set of the basic structure using the OBL-based CPSO.

The two stages of the model have an inherited relationship, and a balance exists between the basic structure optimization and the parameter learning [26], [27]. The basic structure of the evolved model is

Experiments

In this section, a pair of hand contours, a pair of camel contours and two pairs of successive composite micrographs were studied. We provide the experimental results using the proposed algorithms for automated image registration. In the experiments, the floating images are shown in Fig. 8a, Fig. 9, Fig. 19 and Fig. 40a, and the reference images are shown in Fig. 8b, Fig. 10, Fig. 20 and Fig. 40b. In this paper, we use the overlap rate as the error measure. The overlap rate between two polygons

Conclusions

This paper proposes a novel image registration pipeline based on a foot-point detection algorithm and a two-stage evolutionary algorithm. At the feature point detection stage, we propose a novel feature points extraction method based on the estimated curvature and the neighboring rule. The experimental results show that the method is able to obtain a representative and relatively even set of feature points to describe the entire contour. At the stage of determining pairs of feature points, we

Acknowledgments

This research was supported by National Natural Science Foundation of China under contract (No. 61173078), Natural Science Foundation of Shandong Province under contract (Nos. ZR2010FM047 and ZR2011FL016).

Zhao Xiuyang is an associate professor of the Provincial Key Laboratory for Network-based Intelligent Computing. He received his B.Sc. degree in material science and engineering from the Shandong University of Technology of China in 1998, and Master and Ph.D. degrees in computational material science from the Shandong University of China in 2001 and 2006. His main research interests include computer graphic, machine learning, and skeletal tissue engineering.

References (43)

  • W.S. Li et al.

    Adaptive knot placement in B-spline curve approximation

    Comput.-Aided Des.

    (2005)
  • X.Y. Zhao et al.

    A hybrid approach based on MEP and CSP for contour registration

    Appl. Soft Comput.

    (2011)
  • Lizhi Peng et al.

    Parallel evolving algorithm for flexible neural tree

    Parallel Comput.

    (2011)
  • H.F. Wang et al.

    Correspondence matching using kernel principal components analysis and label consistency constraints

    Pattern Recognit.

    (2006)
  • V.E. Markaki et al.

    Application of Kohonen network for automatic point correspondence in 2D medical images

    Comput. Biol. Med.

    (2009)
  • H. Jia et al.

    Iterative multi-atlas-based multi-image segmentation with tree-based registration

    NeuroImage

    (2012)
  • X.Y. Zhao et al.

    Adaptive knot placement using a GMM based continuous optimization algorithm in B-spline curve approximation

    Comput.-Aided Des.

    (2011)
  • S. Zokai et al.

    Image registration using log-polar mappings for recovery of large-scale similarity and projective transformations

    IEEE Trans. Image Process.

    (2005)
  • D.G. Lowe

    Distinctive image features from scale-invariant key points

    Int. J. Comput. Vis.

    (2004)
  • H.W. Kuhn

    The Hungarian method for the assignment problem

    Nav. Res. Logist.

    (1955)
  • G. Sanrom et al.

    A new graph matching method for point-set correspondence using the EM algorithm and Softassign

    Comput. Vis. Image Underst.

    (2012)
  • Cited by (0)

    Zhao Xiuyang is an associate professor of the Provincial Key Laboratory for Network-based Intelligent Computing. He received his B.Sc. degree in material science and engineering from the Shandong University of Technology of China in 1998, and Master and Ph.D. degrees in computational material science from the Shandong University of China in 2001 and 2006. His main research interests include computer graphic, machine learning, and skeletal tissue engineering.

    Yang Bo is a professor and vice-president of University of Jinan, Jinan, China. He is the Director of the Provincial Key Laboratory for Network-based Intelligent Computing and also acts as the Associate Director of Shandong Computer Federation, and Member of the Technical Committee of Intelligent Control of Chinese Association of Automation. His main research interests include computer networks, artificial intelligence, machine learning, knowledge discovery, and data mining.

    Shuming Gao is a professor of the State Key Laboratory of CAD&CG and the School of Computer Science and Engineering, Zhejiang University which is located in Hangzhou, China. He received his Ph.D. degree from the Applied Mathematics Department of Zhejiang University in 1990, and was a visiting professor in the Design Automation Lab of Arizona State University and IPK, Germany respectively in 2001 and 2006. Currently he is serving as an associate editor of ASME Transaction of JCISE. He is also the committee member of a number of international conferences including ACM SPM, IEEE SMI, PLM, CSCWD, CAD/Graphics, etc.

    Yuehui Chen received his B.Sc. degree from the Department of mathematics, Shandong University in 1985, and Master and Ph.D. degrees from the School of Electrical Engineering and Computer Science, Kumamoto University of Japan in 1999 and 2001. During 2001–2003, he had worked as the Senior Researcher at the Memory-Tech Corporation, Tokyo. Since 2003 he has been a member at the Faculty of School of Information Science and Engineering, Jinan University, where he currently heads the Computational Intelligence Laboratory. His research interests include Evolutionary Computation, Neural Networks, Fuzzy Logic Systems, Hybrid Computational Intelligence.

    View full text