An evolutionary framework for machine learning applied to medical data

Abstract

Supervised learning problems can be faced by using a wide variety of approaches supported in machine learning. In recent years there has been an increasing interest in using the evolutionary computation paradigm as a search method for classifiers, helping the applied machine learning technique. In this context, the knowledge representation in the form of logical rules has been one of the most accepted machine learning approaches, because of its level of expressiveness. This paper proposes an evolutionary framework for rule-based classifier induction. Our proposal introduces genetic programming to build a search method for classification-rules (IF/THEN). From this approach, we deal with problems such as, maximum rule length and rule intersection. The experiments have been carried out on our domain of interest, medical data. The achieved results define a methodology to follow in the learning method evaluation for knowledge discovery from medical data. Moreover, the results compared to other methods have shown that our proposal can be very useful in data analysis and classification coming from the medical domain.

Introduction

Machine learning can be approached as the systematic study of algorithms and systems, which improve their knowledge or performance based on experience [1], [2], [3], [4], [5], [6]. The building of machines able to learn from experience has been for a long time a matter of debate since such machines have proven to have a meaningful level of learning ability. Thus, the introduction of machine learning techniques in computer science troubleshooting is of vital importance since there exist problems that cannot be solved through common programming techniques [7], [8], [9], [10]. For such problems there is not an available consistent mathematical model able to guide the programmer. But solving those problems has the potential of reforming aspects of our life and the used machine learning methods may provide the key to their solutions [11].

According to [12], the tasks involving machine learning can be classified into three main categories as follows: supervised learning, which builds a model from a set of inputs and a corresponding set of outputs. The goal is to find a mapping relating inputs with outputs. In contrast, unsupervised learning is not based on experience as the case of supervised learning and there are no labels on the data. The goal of this approach is to capture the true structure of the data to disclose knowledge. In reinforcement learning, intelligent processes (which can be called agents) interact with each other in a dynamic environment to reach their targets. In this approach, agents learn from a series of reinforcements, rewards or punishments, which makes the difference with supervised learning. Since such agents are endowed of a reinforcement process, they can evolve by learning from their environment [13].

The approach using examples (also called instances or patterns) to create programs is known as learning methodology and the set of examples can be referred to as the training data (or training set). The estimate of the target function, which is learned from the learning algorithm and maps inputs with outputs is known as the solution of the learning problem (or decision function). Usually, a set of candidate functions known as hypotheses is selected before to start learning the correct function. Therefore, the set of hypotheses can be seen as the key of the learning strategy. On the other hand, the method taking the training data as input and choosing the hypothesis from the space is the second key of the learning strategy, which is known as the learning method. A learning problem with binary outputs is called binary classification, one with a finite number of outputs is known as multi-class classification and one with real value outputs (continuous values) is known as regression.

The exponential growth of the amount of available medical data raises the problems of efficient storage and management of information as well as disclosing useful information from the data. The problem above is a challenge in computational medicine, claiming the development of methods and tools able to transform data into medical knowledge on the underlying mechanism. Those tools (methods) allow us to go beyond a simple description of the data and provide knowledge in form of models. Through this data abstraction involving a model, we will be able to obtain predictions of systems [8], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25].

There are several medical domains where machine learning techniques have been applied to discover knowledge, such as: diagnosis and prognosis, medical imaging and signal processing, planning and scheduling. Diagnosis and prognosis are the most common within this domain. Diagnosis is the process of selectively collecting information concerning a patient for its subsequent interpretation according to previous knowledge, as evidence for or against the presence or absence of disorders [26]. In a prognostic process, information is also collected and interpreted through the patient. But in this case, the goal is to predict the future behavior of the patient’s condition. For its predictive nature, prognosis systems are often used as tools to state medical treatments [27]. The goal of machine learning in the context of diagnosis and prognosis is knowledge discovery needed to interpret the gathered information. In some cases, this knowledge has been expressed as probabilistic relationships between clinical features and the proposed diagnosis (or prognosis). In other cases, a rule-based representation has been selected, so as to provide the expert with an explanation of the decision. Moreover, there are other cases where the system is designed as a black box decision maker, which is totally indifferent to the interpretation of its decisions. In summary, machine learning techniques are well suited to solve these kinds of problems due to their ability to carry out searches in extremely complex spaces.

We are today racking up a huge amount of data involving medical domains and we are also interested in transforming it into knowledge. Hence, the latter can be used as an auxiliary assistant in the decision-making process performed in the diagnosis of diseases [17], [20], [25]. Disclosing knowledge from data is one of the tasks supported by learning algorithms. In the case of the medical data domain, we are interested about inducing classifiers from a set of labeled patterns as a kind of supervised learning. We aim at developing classifiers in the form of a set of rules. To achieve this, we will devise an evolutionary algorithm, i.e., an algorithm inspired by the principles of evolution by natural selection, and the principles of genetics, tuned for this kind of problem.

The task of particular interest that we are going to implement aims at the development of interpretable solutions, enabling us to transform clinical data into medical knowledge and to incorporate existing clinical evidence. The concrete goal of this task is threefold: (1) to develop a novel algorithm for the building of a classifier from labeled examples and (2–3) to test and validate the algorithm with medical data sets.

As a general approach and in relation to evolutionary computation, we have developed an evolutionary framework based on genetic programming to induce rules of form $IF〈conditions〉\text{ THEN }〈class〉$, which will be able to classify patterns coming from the search space. Therefore, the implementation of the framework has been intending to achieve rule-based classifiers by basing on the idea of sequential covering (or separate and conquer) to render each rule [28]. In such a case, we assume individuals evolving from our learning algorithm as single classification rules (Michigan-style) [29]. The biggest challenge faced by the proposed framework was to solve the intersection problem of rules, i.e., patterns classified by rules in different classes. To solve such a problem, we have proposed an ensemble method to classify patterns at the intersection.

To achieve the aims above, the remainder of this paper has been outlined into the following sections: Section 2 deals with the existing background about our proposal. Section 3 presents our proposal, an evolutionary framework to induce rule-based classifiers. In this context, details such as, encoding and search strategy, rule generalization, fitness functions, genetic operators, classification model, rule intersection, the evolutionary algorithm and classifier improvement have been included in this section. Section 4 outlines the main features of the four clinical data sets involved in the experiments as well as the results reached by this proposal compared with other machine learning methods. Section 5 explains the conclusions of this work. Appendix A Example of a rule-based classifier for dataset#1, Appendix B Proof of theorems present an example of a rule-based classifier and the theoretical results respectively. Additionally, this paper provides a supplemental material in the Supplemental Material Document (SMD), which supports and complements the contributions reached in this research.