Knowledge discovery in multiobjective optimization problems in engineering via Genetic Programming
Introduction
Optimization involves the study and the use of methods to determine the parameters that lead to the best solutions according to the objectives of interest. A problem is classified as multiobjective when it presents multiple and conflicting objectives which must be simultaneously optimized. According to Deb and Srinivasan (2006), in engineering one can expect some similarities among solutions of an optimization problem, related to their optimum conditions. Nowadays, the interest of the researchers has grown in post-optimality studies for the search of intrinsic properties of the optimal solutions of a given optimization problem.
The term Innovization (innovation through optimization) was proposed in Deb and Srinivasan (2006) to refer to a process of knowledge discovery from the output of an optimization problem, in the form of mathematical relations between variables, objectives, constraints, and parameters, that could be thought of as rules of thumb for creating optimal designs. Such relations, or design principles, provide (i) the discovery of promising regions of the search space, (ii) the creation of new solutions without running a new optimization process, and (iii) a deeper understanding of the problem.
The idea of finding knowledge about the solutions can be applied to single objective problems. However, the plurality of solutions usually obtained when solving multiobjective problems provide more data to search for properties that may reveal novel characteristics about the problem (Deb & Srinivasan, 2006).
The Innovization process was extended in Bandaru and Deb (2010), Bandaru, Aslam, Ng, and Deb (2015); Bandaru and Deb (2013). In the automated Innovation (Bandaru & Deb, 2010) a Genetic Algorithm (GA) is used to search for design principles that may reveal characteristics of the solutions present along the Pareto Front. The search space is formed by the decision variables, objectives, constraints, and additional functions that can be indicated by the user.
Several other studies, based on machine learning methods, visualization techniques, or even analytical methods, were carried out to reveal information in multiobjective problems.
Kohonen self organizing maps (SOMs) (Kohonen, 1990) were used by Chiba, Imamura, Amemiya, Jeong, and Yamamoto (2006); Doncieux and Hamdaoui (2011); Obayashi and Sasaki (2003) to extract information from Pareto-Optimal Solutions. SOMs are a type of unsupervised recurrent neural network capable of spatially separating multidimensional data in groups with similar characteristics, keeping the most related groups close to each other. Obayashi and Sasaki (2003) used SOMs to search for patterns in supersonic aircraft fusion designs. SOMs were also used by Doncieux and Hamdaoui (2011) to identify patterns that affect the velocity in the design of an ornithopter’s wing.
A visualization based technique was proposed by Pryke, Mostaghim, and Nazemi (2007), where heatmaps were used to visualize the decision and objective spaces simultaneously, making it possible to identify correlations between them. An advantage of this method is that it can be applied to problems with more than three objectives, a limitation usually observed in visualization techniques.
Ulrich, Brockhoff, and Zitzler (2008) proposed the use of dendrograms to cluster non-dominated solutions for the discovery of design principles. The technique was successfully applied to the knapsack problem and also to the design of an embedded processor.
In order to extract knowledge from non-dominated solutions, (Kudo & Yoshikawa, 2012) proposed the use of the Isomap visualization method, a nonlinear dimensionality reduction technique based on the geodetic distance. The proposed method calculates the geodetic distance among objectives and decision variables.
Ulrich (2013) proposed a bi-objective formulation to the problem of finding correlations between decision variables and the objective space, along with an algorithm capable of solving it, called Pareto-front Analyzer (PAN).
A search method called Multiobjective Robust Design Exploration was proposed by Sugimura, Jeong, Obayashi, and Kimura (2009), which consists of a multiobjective optimization followed by the analysis of the solutions using association rules, searching for correlations between decision variables and objective values. Association rules are expressions of the form “if-then” which can be used to infer causality.
An extensive review of techniques and applications regarding data mining in multiobjective problems can be found in Bandaru, Ng, and Deb (2017a); 2017b).
One step towards discovering more general design principles was the adoption of Genetic Programming (GP) (Bandaru & Deb, 2013), a technique widely used for the evolution of programs, such as symbolic expressions and classifier models.
A modification was necessary to use GP as the search algorithm: besides finding principles that reflect the similarities present in the Pareto Front, the models found must correctly relate the decision variables with respect to the basic units involved. Thus, (Bandaru & Deb, 2013) proposed the use of a GP in which the consistency of the units is verified during the search process, in order to encourage the generation of valid candidate solutions. The approach chosen by the authors was penalization, which assigns an arbitrarily large penalty value to a candidate solution that presents inconsistent physical units.
Here, an alternative solution for enforcing the consistency of units is proposed, which involves performing protected operations that ignore the invalid terms of the expressions. The use of protected operations is commonly adopted in GP to handle the execution of arithmetic operations involving invalid values. For example, it is common to use a protected division operation, that returns a pre-defined value when the denominator is equal to zero.
In order to obtain more diverse solutions, the introduction of an external archive is proposed here to maintain all potential design principles of interest, which could be otherwise lost throughout the search process.
It has also been observed that some design principles can be generated which, given their simplicity, do not add knowledge about the problem, being therefore irrelevant. Such solutions are called here trivial solutions. A procedure is proposed to avoid the presence of this type of principle in the population.
The techniques proposed here are applied to four case studies in engineering that have already been studied in the Innovization literature. The problems involve the designs of: a two-member-truss, a welded beam, the cutting of a metal bar, and a composite gear.
Section snippets
Innovization
An automated Innovization process was proposed in Bandaru and Deb (2010) that consists of using a Genetic Algorithm (GA) in the search for invariant properties, that is, design principles that may reveal characteristics present in the Pareto-optimal solutions. Those principles can be expressed as symbolic expressions containing decision variables, objectives and constraints. Thus, it is desired to find functions of the form where f(x) are the objective functions, g(x) are
Proposed methods
Here an alternative solution is proposed to handle the consistency among units, in which protected operations are created to ignore the invalid terms of the models. Thereby, invalid models are not eliminated via penalty, but repaired. It is also proposed a strategy to avoid obtaining trivial solutions when a GP technique is used. Finally, it is proposed to use an external archive to maintain the solutions of interest found during the search process.
Computational experiments
Computational experiments were carried out to evaluate if the methods proposed here contribute to finding more diverse design principles. For each multiobjective problem, 30 independent runs were performed using 1 the NSGA-II algorithm (Deb, Pratap, Agarwal, & Meyarivan, 2002). The Pareto Fronts obtained in each run of a given problem are joined and a final Pareto Front is then generated containing all
Concluding remarks
In this work three modifications were proposed for the Innovization process based on Genetic Programming: (i) an alternative way to promote consistency in the use of units, (ii) the use of an external archive to maintain the promising solutions, and (iii) a procedure to avoid obtaining trivial, and therefore irrelevant, solutions.
From the results obtained in four case studies in engineering it can be concluded that (i) and (iii) contribute to obtaining a larger number of good solutions and
Acknowledgements
The authors thank the support of CAPES, CNPq (grant 310778/2013-1), FAPEMIG (grant APQ-03414-15), and PPGMC/UFJF.
References (21)
- et al.
Generalized higher-level automated innovization with application to inventory management
European Journal of Operational Research
(2015) - et al.
Data mining methods for knowledge discovery in multi-objective optimization: Part a - survey
Expert Systems with Applications
(2017) - et al.
Design exploration of shielding effect for aircraft engine noise
Eccomas cfd 2006: Proceedings of the european conference on computational fluid dynamics, egmond aan zee, the netherlands, september 5–8, 2006
(2006) - et al.
Automated discovery of vital knowledge from Pareto-optimal solutions: First results from engineering design
Ieee congress on evolutionary computation
(2010) - et al.
Towards automating the discovery of certain innovative design principles through a clustering-based optimization technique
Engineering Optimization
(2011) - et al.
A dimensionally-aware genetic programming architecture for automated innovization
International conference on evolutionary multi-criterion optimization
(2013) - et al.
Data mining methods for knowledge discovery in multi-objective optimization: Part b - new developments and applications
Expert Systems with Applications
(2017) Optimal design of a welded beam via genetic algorithms
AIAA Journal
(1991)- et al.
A fast and elitist multiobjective genetic algorithm: nsga-ii
IEEE Transactions on Evolutionary Computation
(2002) - et al.
Innovization: innovating design principles through optimization
Cited by (9)
Discovering generalized design knowledge using a multi-objective evolutionary algorithm with generalization operators
2020, Expert Systems with ApplicationsCitation Excerpt :Logical rules are easy to understand, as they can easily be used for building mental models of the knowledge they convey. Popular data mining methods to extract design rules include decision trees (Graening et al., 2008; Jahr, Calborean, Vintan, & Ungerer, 2012; Yan, Qiao, Simpson, Li, & Zhang, 2012), association rule mining (ARM) (Bandaru, Ng, & Deb, 2017; Ng, Bandaru, & Frantzén, 2016; Watanabe, Chiba, & Kanazaki, 2014), and evolutionary algorithms (EA) (Bandaru & Deb, 2010; Russo, Bernardino, & Barbosa, 2018). These methods can be used to identify design features that are common among good (e.g., non-dominated) designs.
Self-adaptive MRPBIL-DE for 6D robot multiobjective trajectory planning
2019, Expert Systems with ApplicationsCitation Excerpt :From literature, it was found that most of research work are focusing on single objective optimisation while studies on multiobjective optimisation for such a robot design problem have been rarely presented. Over the last decade, numerous MOMHs have been proposed (Gao, Zhou, Li, Pan, & Yi, 2015; Hidalgo-Paniagua, Vega-Rodríguez, & Ferruz, 2016; Liu, Zhang, He, & Jiang, 2018; Nuaekaew, Artrit, Pholdee, & Bureerat, 2017; Onan, Korukoğlu, & Bulut, 2016; Pholdee & Bureerat, 2013a,b; Qingfu & Hui, 2007; Robič & Filipič, 2005; Russo, Bernardino, & Barbosa, 2018; Wang, Jiao, & Yao, 2015; Wang, Purshouse, & Fleming, 2013; Zareizadeh, Helfroush, Rahideh, & Kazemi, 2018; Zhang, Tian, & Jin, 2015). Some of effective and efficient MOMHs are an improved two-archive algorithm (Two_Arch2) (Wang et al., 2015), a preference-inspired co-evolutionary algorithm using goal vectors (PICEA-g), a knee point driven evolutionary algorithm for many-objective optimization (KnEA) (Zhang et al., 2015), an unrestricted population size evolutionary multiobjective optimisation algorithm (UPS-EMOA) (Aittokoski & Miettinen, 2010), hybridisation of real-code population-based incremental learning and differential evolution (MRPBIL-DE) (Pholdee & Bureerat, 2013b).
A novel Error-Correcting Output Codes algorithm based on genetic programming
2019, Swarm and Evolutionary ComputationCitation Excerpt :GP is a widely deployed evolutionary algorithm, proposed by Koza [43]. Up to now, it has been successfully applied to different optimization problems in diverse fields, such as interpreting reinforcement learning policies [44] and different types of knowledge discovery [45,46] by treating individuals as symbolic expressions. And some GP based learning algorithms treated individuals as learners, so as to applied to rainfall prediction [47], building ensemble learning systems [48] and outlier elimination [49].
Heat production optimization using bio-inspired algorithms
2018, Engineering Applications of Artificial IntelligenceCitation Excerpt :While Dubey et al. (2018) presented a comparative overview of recent advances in using bio-inspired methods for managing wind power dispatch. Evolutionary computing is also reported to be efficient in computer aided detection (Morra et al., 2018) and multi-objective problems of engineering systems (Russo et al., 2018). Meanwhile in Liao et al. (2018) bio-inspired methods were used to minimize the power consumption for an application of luminance control.
Black Hole Mechanics Optimization: a novel meta-heuristic algorithm
2020, Asian Journal of Civil Engineering