Elsevier

Applied Soft Computing

Volume 37, December 2015, Pages 561-571
Applied Soft Computing

GPFIS-CLASS: A Genetic Fuzzy System based on Genetic Programming for classification problems

https://doi.org/10.1016/j.asoc.2015.08.055Get rights and content

Highlights

  • Novel Genetic Fuzzy System based on Multi-Gene Genetic Programming called GPFIS-CLASS.

  • GPFIS-CLASS builds fuzzy rule premises by using Multi-Gene Genetic Programming.

  • In 22 datasets GPFIS-CLASS compares favorably to other GP-based Genetic Fuzzy Systems.

  • In other 23 datasets GPFIS-CLASS outperforms other state-of-the-art EFSs.

  • In general, GPFIS-CLASS provides more compact rule bases.

Abstract

Genetic Fuzzy Systems (GFSs) are models capable of integrating accuracy and high comprehensibility in their results. In the case of GFSs for classification, more emphasis has been given to improving the “Genetic” component instead of its “Fuzzy” counterpart. This paper focus on the Fuzzy Inference component to obtain a more accurate and interpretable system, presenting the so-called Genetic Programming Fuzzy Inference System for Classification (GPFIS-CLASS). This model is based on Multi-Gene Genetic Programming and aims to explore the elements of a Fuzzy Inference System. GPFIS-CLASS has the following features: (i) it builds fuzzy rules premises employing t-norm, t-conorm, negation and linguistic hedge operators; (ii) it associates to each rule premise a suitable consequent term; and (iii) it improves the aggregation process by using a weighted mean computed by restricted least squares. It has been evaluated in two sets of benchmarks, comprising a total of 45 datasets, and has been compared with eight different classifiers, six of them based on GFSs. The results obtained in both sets demonstrate that GPFIS-CLASS provides better results for most benchmark datasets.

Introduction

Genetic Fuzzy Systems (GFSs) [16], [17], [23], [29] have been widely employed to solve classification [4], [11], [49], regression [2], [5] and control [15], [54] problems. The main feature that highlights GFSs in respect to other mathematical, statistical and artificial intelligence models is its capability of extracting knowledge from datasets or industrial plants and state it in linguistic terms with reasonable accuracy. This is provided by the bond between a Fuzzy Inference System (FIS) and a Genetic Based Meta-Heuristic (GBMH), which is based on Darwinian concepts of natural selection and genetic recombination. Therefore, a GFS provides fair accuracy and linguistic interpretation (FIS component) through the automatic learning of its parameters/rules (GBMH component), using information extracted from a dataset or a plant.

In GFSs literature, most works focus on developing or modifying methods in the GBMH component, such as:

  • Modification of the Genetic Fuzzy Rule-Based Systems in the GBMH structure (codification, selection and evaluation) to generate fuzzy rule bases different from the standard Pittsburgh-style [29], [36], [44], such as Michigan [15], [43], Genetic Cooperative-Competitive Learning (GCCL) [11], [32], [40], [45] and Iterative Rule Learning (IRL) [18], [26], [28] approaches;

  • Application of Multi-Objective Evolutionary Algorithms (MOEAs) to search for solutions that satisfy both accuracy and interpretability criteria in GFSs development [3], [6], [21], [22], [33];

  • Employment of a GBMH to fine tune membership functions in the post-processing stage [49], [50]; and

  • Use of other Evolutionary Algorithms outside the GBMH scope (Particle Swarm and other Bio-inspired algorithms), in order to attain better results [13], [25], [37], [42].

However, few researchers have focused on developments of the Fuzzy Inference component (e.g., operators such as negation, linguistic hedges, aggregation, etc.) to improve the performance obtained by the GFS. Additionally, there is a lack of works using Genetic Programming (GP) [39], [47] as a GBMH for a GFS [16], despite its adequacy to problems that demand a non-fixed size codification – such as a fuzzy rule-based system. Therefore, to satisfy both conditions – FIS structure developments and application of Genetic Programming –, this work proposes a new GFS model: the Genetic Programming Fuzzy Inference System for Classification problems (GPFIS-CLASS). GPFIS-CLASS is based on Multi-Gene Genetic Programming (sometimes called Multi-Tree Genetic Programming) [24], [52], a generalization of Koza-style GP [39], [47] with three main features: (i) it builds fuzzy rule premises by employing t-norm, t-conorm, negation and linguistic hedge operators; (ii)it associates a given premise to a suitable consequent term; and (iii) it improves the aggregation procedure by using operators other than the maximum t-conorm. GPFIS-CLASS should be seen as an improvement to the GPF-CLASS model [38]. This does not present reasonable results due to the lack of a suitable procedure for consequent term definition and membership degrees aggregation.

In order to assess its capability, GPFIS-CLASS is evaluated in two sets of experiments, extracted from Berlanga et al. [11] and Antonelli et al. [6] respectively. Briefly, the former [11] proposes a new GFS based on GP, in which each individual is encoded in a GCCL scheme for fuzzy rule base learning. This model was applied to 24 datasets and its results have been compared to those of four other GFSs, two of them based on GP. Antonelli et al. [6], on the other hand, presents a novel approach for learning concurrently the rule and data bases of fuzzy rule-based classifiers based on a multi-objective evolutionary approach. This system is evaluated in 24 datasets, mainly by comparing its results with those from two Evolutionary Fuzzy Systems (EFSs) and two other state-of-the-art classifiers. GPFIS-CLASS performance was then evaluated in most datasets; results have been compared to those provided by other GP-based and state-of-the-art EFSs models.

This paper has four additional sections. The next section describes the main concepts of Multi-Gene Genetic Programming. Section 2 presents the proposed GPFIS-CLASS model in five steps: fuzzification, fuzzy inference, decision, evaluation and selection & recombination. Section 4 deals with the model evaluation for different benchmark classification problems, and Section 5 concludes the work.

Section snippets

Multi-Gene Genetic Programming

Genetic Programming (GP) [39], [47] belongs to the Evolutionary Computation field. Typically, it employs a population of individuals (or solutions), each of them denoted by a tree structure that codifies a mathematical equation, which describes the relationship between a set of input features Xj (j = 1, . . . , J) and the output Y. Based on these ideas, Multi-Gene Genetic Programming (MGGP) [20], [24], [31], [52] generalizes GP, as it denotes an individual as a set of tree structures, commonly

GPFIS-CLASS model

GPFIS-CLASS is a typical Pittsburgh-type GFS [29], in which each individual represents a fuzzy rule base. Fig. 3 exhibits the main modules of the GPFIS-CLASS model.

Modeling begins by mapping crisp values into membership degrees of fuzzy sets (Fuzzification). Then, a fuzzy inference procedure is performed in three substeps: (i) ggeneration of fuzzy rule premises (Formulation); (ii) assignment of the best suited consequent term for each premise (Association) and (iii) aggregation of activated

Case studies

The proposed GPFIS-CLASS has been compared with two sets of models: GP-based models, by using datasets presented in Berlanga et al. [11]; and state-of-the-art GFS models in general, based on the experiments presented in Antonelli et al. [6]. There are other works on GFSs applied to classification problems (e.g., see [4], [7], [21], [44], [49]), but [11] and [6] are the most recent contributions in GP-based GFSs and in Multi-Objective Evolutionary Fuzzy Systems (MOEFS), respectively, with a

Conclusions

This work has proposed a new Genetic Fuzzy System based on Genetic Programming, called Genetic Programming Fuzzy Inference System for Classification problems (GPFIS-CLASS). Its building blocks have been described, mainly the Fuzzy Inference process (Formulation-Association-Aggregation), which is what distinguishes it more from other Evolutionary Fuzzy Systems in the area.

In order to assess the performance of GPFIS-CLASS, several experiments and comparisons with other GFSs haven been carried out

Acknowledgments

This research was supported by Pontifical Catholic University of Rio de Janeiro, CNPq and CAPES.

References (54)

  • V. López et al.

    A hierarchical genetic fuzzy system based on genetic programming for addressing classification with highly imbalanced and borderline data-sets

    Knowl. Based Syst.

    (2013)
  • A. Tsakonas

    A comparison of classification accuracy of four genetic programming-evolved intelligent structures?

    Inf. Sci.

    (2006)
  • A. Tsakonas

    Local and global optimization for Takagi-Sugeno fuzzy system by memetic genetic programming?

    Expert Syst. Appl.

    (2013)
  • J. Alcalá-Fdez et al.

    Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework

    J. Mult. Valued Logic Soft Comput.

    (2011)
  • R. Alcalá et al.

    A fast and scalable multiobjective genetic fuzzy system for linguistic fuzzy modeling in high-dimensional regression problems

    IEEE Trans. Fuzzy Syst.

    (2011)
  • R. Alcalá et al.

    Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions?

    Soft Comput.

    (2011)
  • J. Alcalá-Fdez et al.

    A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning

    IEEE Trans. Fuzzy Syst.

    (2011)
  • K. Bache et al.

    UCI machine learning repository

    (2013)
  • G. Beliakov et al.

    Aggregation Functions: A Guide for Practitioners

    (2008)
  • G. Beliakov et al.

    Appropriate choice of aggregation operators in fuzzy decision support systems?

    IEEE Trans. Fuzzy Syst.

    (2001)
  • F.J. Berlanga et al.

    A genetic-programming-based approach for the learning of compact fuzzy rule-based classification systems

  • D. Boughaci et al.

    Fuzzy particle swarm optimization for intrusion detection

  • T. Calvo et al.

    Aggregation operators: properties, classes and construction methods

  • J. Casillas et al.

    Fuzzy-XCS: a Michigan genetic fuzzy system

    IEEE Trans. Fuzzy Syst.

    (2007)
  • O. Cordón et al.

    Ten years of genetic fuzzy systems: current framework and new trends

    Fuzzy Sets Syst.

    (2004)
  • O. Cordón et al.

    Mogul: a methodology to obtain genetic fuzzy rule-based systems under the iterative rule learning approach?

    Int. J. Intell. Syst.

    (1999)
  • K.A. Fattah

    A new approach calculate oil–gas ratio for gas condensate and volatile oil reservoirs using genetic programming

    Oil Gas Bus.

    (2012)
  • Cited by (19)

    • Knowledge transfer for labeling unknown fuzzy sets using Grammar-Guided Genetic Algorithms

      2022, Applied Soft Computing
      Citation Excerpt :

      There is particular interest in the interpretability of neuro-fuzzy systems through the use of linguistic hedges using genetic systems [18,19]. In [20] a Genetic Fuzzy System classifier called GPFIS-CLASS, that use Multi-Gene Genetic Programming to build rule premises using t-norm, t-conorm, negation, and hedge operations is proposed; their objective function consists in the measured classification error and the complexity of the generated rules. The second objective in this system, the fewer value gets, the better it is for the system.

    • Quantitative-integration-based TSK fuzzy classification through improving the consistency of multi-hierarchical structure

      2021, Applied Soft Computing
      Citation Excerpt :

      In addition, the average training accuracies of GFS-GP-C on the four Groups are higher than those of rest algorithms, as shown in Fig. 2. The study speculates that this advantage is due to fact that the features of GFS-GP-C are trained by the fuzzy classifier itself [33]. Furthermore, the fuzzy rules of GFS-GP-C are often not interpretable [31–33].

    • Diverse partner selection with brood recombination in genetic programming

      2018, Applied Soft Computing Journal
      Citation Excerpt :

      Elola et al. [10] exploited the predictive importance of intermediate solutions during a GP evolution and used it successfully for improving convergence rate. Genetic Programming in combination with fuzzy inference system was used for improving classification abilities of GP [11]. Ojha, Abraham and Snasel used diversity index measure for maintaining diversity in a multi objective GP and showed that it improved the efficiency of the proposed algorithm [13].

    • RST-BatMiner: A fuzzy rule miner integrating rough set feature selection and Bat optimization for detection of diabetes disease

      2018, Applied Soft Computing Journal
      Citation Excerpt :

      In addition to increasing interpretability of fuzzy rules, it also deals the continuous attributes very efficiently [6]. Although many Fuzzy Rule Based Classification Systems (FRBCS) are used in disease diagnosis, all of these systems fail to produce accurate and comprehensive fuzzy rules [9–12]. Thus fuzzy classification rule mining is emerging as one of the major problem in the data mining field, which aims to find a small set of comprehensive rules based on a training data set.

    View all citing articles on Scopus
    View full text