GPFIS-CLASS: A Genetic Fuzzy System based on Genetic Programming for classification problems
Graphical abstract
Introduction
Genetic Fuzzy Systems (GFSs) [16], [17], [23], [29] have been widely employed to solve classification [4], [11], [49], regression [2], [5] and control [15], [54] problems. The main feature that highlights GFSs in respect to other mathematical, statistical and artificial intelligence models is its capability of extracting knowledge from datasets or industrial plants and state it in linguistic terms with reasonable accuracy. This is provided by the bond between a Fuzzy Inference System (FIS) and a Genetic Based Meta-Heuristic (GBMH), which is based on Darwinian concepts of natural selection and genetic recombination. Therefore, a GFS provides fair accuracy and linguistic interpretation (FIS component) through the automatic learning of its parameters/rules (GBMH component), using information extracted from a dataset or a plant.
In GFSs literature, most works focus on developing or modifying methods in the GBMH component, such as:
- •
Modification of the Genetic Fuzzy Rule-Based Systems in the GBMH structure (codification, selection and evaluation) to generate fuzzy rule bases different from the standard Pittsburgh-style [29], [36], [44], such as Michigan [15], [43], Genetic Cooperative-Competitive Learning (GCCL) [11], [32], [40], [45] and Iterative Rule Learning (IRL) [18], [26], [28] approaches;
- •
Application of Multi-Objective Evolutionary Algorithms (MOEAs) to search for solutions that satisfy both accuracy and interpretability criteria in GFSs development [3], [6], [21], [22], [33];
- •
Employment of a GBMH to fine tune membership functions in the post-processing stage [49], [50]; and
- •
Use of other Evolutionary Algorithms outside the GBMH scope (Particle Swarm and other Bio-inspired algorithms), in order to attain better results [13], [25], [37], [42].
However, few researchers have focused on developments of the Fuzzy Inference component (e.g., operators such as negation, linguistic hedges, aggregation, etc.) to improve the performance obtained by the GFS. Additionally, there is a lack of works using Genetic Programming (GP) [39], [47] as a GBMH for a GFS [16], despite its adequacy to problems that demand a non-fixed size codification – such as a fuzzy rule-based system. Therefore, to satisfy both conditions – FIS structure developments and application of Genetic Programming –, this work proposes a new GFS model: the Genetic Programming Fuzzy Inference System for Classification problems (GPFIS-CLASS). GPFIS-CLASS is based on Multi-Gene Genetic Programming (sometimes called Multi-Tree Genetic Programming) [24], [52], a generalization of Koza-style GP [39], [47] with three main features: (i) it builds fuzzy rule premises by employing t-norm, t-conorm, negation and linguistic hedge operators; (ii)it associates a given premise to a suitable consequent term; and (iii) it improves the aggregation procedure by using operators other than the maximum t-conorm. GPFIS-CLASS should be seen as an improvement to the GPF-CLASS model [38]. This does not present reasonable results due to the lack of a suitable procedure for consequent term definition and membership degrees aggregation.
In order to assess its capability, GPFIS-CLASS is evaluated in two sets of experiments, extracted from Berlanga et al. [11] and Antonelli et al. [6] respectively. Briefly, the former [11] proposes a new GFS based on GP, in which each individual is encoded in a GCCL scheme for fuzzy rule base learning. This model was applied to 24 datasets and its results have been compared to those of four other GFSs, two of them based on GP. Antonelli et al. [6], on the other hand, presents a novel approach for learning concurrently the rule and data bases of fuzzy rule-based classifiers based on a multi-objective evolutionary approach. This system is evaluated in 24 datasets, mainly by comparing its results with those from two Evolutionary Fuzzy Systems (EFSs) and two other state-of-the-art classifiers. GPFIS-CLASS performance was then evaluated in most datasets; results have been compared to those provided by other GP-based and state-of-the-art EFSs models.
This paper has four additional sections. The next section describes the main concepts of Multi-Gene Genetic Programming. Section 2 presents the proposed GPFIS-CLASS model in five steps: fuzzification, fuzzy inference, decision, evaluation and selection & recombination. Section 4 deals with the model evaluation for different benchmark classification problems, and Section 5 concludes the work.
Section snippets
Multi-Gene Genetic Programming
Genetic Programming (GP) [39], [47] belongs to the Evolutionary Computation field. Typically, it employs a population of individuals (or solutions), each of them denoted by a tree structure that codifies a mathematical equation, which describes the relationship between a set of input features Xj (j = 1, . . . , J) and the output Y. Based on these ideas, Multi-Gene Genetic Programming (MGGP) [20], [24], [31], [52] generalizes GP, as it denotes an individual as a set of tree structures, commonly
GPFIS-CLASS model
GPFIS-CLASS is a typical Pittsburgh-type GFS [29], in which each individual represents a fuzzy rule base. Fig. 3 exhibits the main modules of the GPFIS-CLASS model.
Modeling begins by mapping crisp values into membership degrees of fuzzy sets (Fuzzification). Then, a fuzzy inference procedure is performed in three substeps: (i) ggeneration of fuzzy rule premises (Formulation); (ii) assignment of the best suited consequent term for each premise (Association) and (iii) aggregation of activated
Case studies
The proposed GPFIS-CLASS has been compared with two sets of models: GP-based models, by using datasets presented in Berlanga et al. [11]; and state-of-the-art GFS models in general, based on the experiments presented in Antonelli et al. [6]. There are other works on GFSs applied to classification problems (e.g., see [4], [7], [21], [44], [49]), but [11] and [6] are the most recent contributions in GP-based GFSs and in Multi-Objective Evolutionary Fuzzy Systems (MOEFS), respectively, with a
Conclusions
This work has proposed a new Genetic Fuzzy System based on Genetic Programming, called Genetic Programming Fuzzy Inference System for Classification problems (GPFIS-CLASS). Its building blocks have been described, mainly the Fuzzy Inference process (Formulation-Association-Aggregation), which is what distinguishes it more from other Evolutionary Fuzzy Systems in the area.
In order to assess the performance of GPFIS-CLASS, several experiments and comparisons with other GFSs haven been carried out
Acknowledgments
This research was supported by Pontifical Catholic University of Rio de Janeiro, CNPq and CAPES.
References (54)
- et al.
An efficient multi-objective evolutionary fuzzy system for regression problems?
Int. J. Approx. Reason.
(2013) - et al.
A fast and efficient multi-objective evolutionary learning scheme for fuzzy rule-based classifiers
Inf. Sci.
(2014) - et al.
HGA: hybrid genetic algorithm in fuzzy rule-based classification systems for high-dimensional problems?
Appl. Soft Comput.
(2012) - et al.
GP-coach: genetic programming-based learning of compact and accurate fuzzy rule-based classification systems for high-dimensional problems?
Inf. Sci.
(2010) A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems?
Int. J. Approx. Reason.
(2011)- et al.
A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms?
Swarm Evol. Comput.
(2011) - et al.
Revisiting evolutionary fuzzy systems: taxonomy, applications, new trends and challenges
Knowl. Based Syst.
(2015) - et al.
A fuzzy classification system based on ant colony optimization for diabetes disease diagnosis?
Expert Syst. Appl.
(2011) - et al.
A feature construction approach for genetic iterative rule learning algorithm?
J. Comput. Syst. Sci.
(2014) - et al.
Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning?
Int. J. Approx. Reason.
(2007)
A hierarchical genetic fuzzy system based on genetic programming for addressing classification with highly imbalanced and borderline data-sets
Knowl. Based Syst.
A comparison of classification accuracy of four genetic programming-evolved intelligent structures?
Inf. Sci.
Local and global optimization for Takagi-Sugeno fuzzy system by memetic genetic programming?
Expert Syst. Appl.
Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework
J. Mult. Valued Logic Soft Comput.
A fast and scalable multiobjective genetic fuzzy system for linguistic fuzzy modeling in high-dimensional regression problems
IEEE Trans. Fuzzy Syst.
Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions?
Soft Comput.
A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning
IEEE Trans. Fuzzy Syst.
UCI machine learning repository
Aggregation Functions: A Guide for Practitioners
Appropriate choice of aggregation operators in fuzzy decision support systems?
IEEE Trans. Fuzzy Syst.
A genetic-programming-based approach for the learning of compact fuzzy rule-based classification systems
Fuzzy particle swarm optimization for intrusion detection
Aggregation operators: properties, classes and construction methods
Fuzzy-XCS: a Michigan genetic fuzzy system
IEEE Trans. Fuzzy Syst.
Ten years of genetic fuzzy systems: current framework and new trends
Fuzzy Sets Syst.
Mogul: a methodology to obtain genetic fuzzy rule-based systems under the iterative rule learning approach?
Int. J. Intell. Syst.
A new approach calculate oil–gas ratio for gas condensate and volatile oil reservoirs using genetic programming
Oil Gas Bus.
Cited by (19)
Automatic generation of fuzzy inference systems for multivariate time series forecasting
2023, Fuzzy Sets and SystemsKnowledge transfer for labeling unknown fuzzy sets using Grammar-Guided Genetic Algorithms
2022, Applied Soft ComputingCitation Excerpt :There is particular interest in the interpretability of neuro-fuzzy systems through the use of linguistic hedges using genetic systems [18,19]. In [20] a Genetic Fuzzy System classifier called GPFIS-CLASS, that use Multi-Gene Genetic Programming to build rule premises using t-norm, t-conorm, negation, and hedge operations is proposed; their objective function consists in the measured classification error and the complexity of the generated rules. The second objective in this system, the fewer value gets, the better it is for the system.
Quantitative-integration-based TSK fuzzy classification through improving the consistency of multi-hierarchical structure
2021, Applied Soft ComputingCitation Excerpt :In addition, the average training accuracies of GFS-GP-C on the four Groups are higher than those of rest algorithms, as shown in Fig. 2. The study speculates that this advantage is due to fact that the features of GFS-GP-C are trained by the fuzzy classifier itself [33]. Furthermore, the fuzzy rules of GFS-GP-C are often not interpretable [31–33].
Diverse partner selection with brood recombination in genetic programming
2018, Applied Soft Computing JournalCitation Excerpt :Elola et al. [10] exploited the predictive importance of intermediate solutions during a GP evolution and used it successfully for improving convergence rate. Genetic Programming in combination with fuzzy inference system was used for improving classification abilities of GP [11]. Ojha, Abraham and Snasel used diversity index measure for maintaining diversity in a multi objective GP and showed that it improved the efficiency of the proposed algorithm [13].
RST-BatMiner: A fuzzy rule miner integrating rough set feature selection and Bat optimization for detection of diabetes disease
2018, Applied Soft Computing JournalCitation Excerpt :In addition to increasing interpretability of fuzzy rules, it also deals the continuous attributes very efficiently [6]. Although many Fuzzy Rule Based Classification Systems (FRBCS) are used in disease diagnosis, all of these systems fail to produce accurate and comprehensive fuzzy rules [9–12]. Thus fuzzy classification rule mining is emerging as one of the major problem in the data mining field, which aims to find a small set of comprehensive rules based on a training data set.