- author = "Lino Alberto Rodriguez Coayahuitl",
- title = "Deep Representation Learning with Genetic Programming",
- title_es = "Aprendizaje Profundo de Representaciones mediante Programacion Genetica",
- school = "Instituto Nacional de Astrofisica, Optica y Electronica",
- year = "2020",
- address = "Sta. Ma. Tonantzintla, Puebla, Mexico",
- keywords = "genetic algorithms, genetic programming, Evolutionary algorithms, machine learning, deep learning, representation learning, image processing",
-
URL = "
https://inaoe.repositorioinstitucional.mx/jspui/bitstream/1009/2161/1/tesiscompleta_RodriguezCoayahuitl_doctorado_CCC.pdf",
- size = "141 pages",
-
abstract = "In this thesis, we propose the development of a new
Deep Learning (DL) model based on the Genetic
Programming (GP) framework. Although DL is typically
considered a machine learning (ML) field solely
concerned with certain class of artificial neural
networks (ANNs), in this thesis we approach DL as a
paradigm shift, where the two classical stages of a ML
workflow, i.e. feature extraction and prediction, are
fusioned in a single ML algorithm, represented by an
unified pipeline composed of non-linear
transformations. Our literature review, that revolves
around recent efforts from the research community at
developing new DL architectures that depart from the
classical ANN-based models, supports our point of
view.
As result from our research, we propose two GP-based DL models, one for unsupervised learning and another for supervised learning. The unsupervised learning model consists in a GP framework aimed at evolving autoencoder algorithms. To the best of the author knowledge, this is the first time autoencoder algorithms are generated though a ML model other than ANNs. The performance obtained by the evolved autoencoders is comparable to the performance of deep networks proposed ten years ago; nevertheless, we consider the GP framework more general than that of ANNs, due to the fact that GP searches for the entire solution structure, whereas in the ANNs framework only the weights are optimized, hence the relevance of this result.
The proposed model for supervised learning consists in a modification of GP individual representation. This new framework, we call Fractal GP, allows to efficiently evolve / learn deeper GP structures than a vanilla GP. Empirical assessment of the proposed model confirms its superior performance over the canonical version of GP. A further extension to the Fractal GP model inspired by Convolutional Neural Networks is proposed, that allows to evolve complete DL convolutional architectures, where all the artificial neurons are replaced by GP abstract syntax trees.
However, what we consider the most important result obtained from our research, concerns with a phenomenon we observed after a closer inspection of the individuals generated by the FractalGP. Results obtained showed that in the proposed model, where artificial neurons are replaced with syntax trees, the GP makes the syntax trees evolve towards structures that resemble artificial neurons, i.e. the artificial evolution heuristic finds the missing elements of the origin heuristic, thus losing the capacity of finding novel solutions to the problem tackled. The conclusion we arrive from this result, is that attempting to import certain heuristics aspects from classical DL into GP, might be a flawed approach. This opens new lines of research for serious consideration",
-
notes = "In english
INAOE
Supervisor: Hugo Jair Escalante Balderas",
