``Found in Translation'': An Evolutionary Framework for Auditory-Visual Relationships
Created by W.Langdon from
gp-bibliography.bib Revision:1.8081
- @Article{rodrigues:2022:Entropy,
-
author = "Ana Rodrigues and Bruna Sousa and Amilcar Cardoso and
Penousal Machado",
-
title = "``Found in Translation'': An Evolutionary Framework
for Auditory-Visual Relationships",
-
journal = "Entropy",
-
year = "2022",
-
volume = "24",
-
number = "12",
-
pages = "Article No. 1706",
-
keywords = "genetic algorithms, genetic programming",
-
ISSN = "1099-4300",
-
URL = "https://www.mdpi.com/1099-4300/24/12/1706",
-
DOI = "doi:10.3390/e24121706",
-
abstract = "The development of computational artifacts to study
cross-modal associations has been a growing research
topic, as they allow new degrees of abstraction. In
this context, we propose a novel approach to the
computational exploration of relationships between
music and abstract images, grounded by findings from
cognitive sciences (emotion and perception). Due to the
problem’s high-level nature, we rely on
evolutionary programming techniques to evolve this
audio–visual dialogue. To articulate the
complexity of the problem, we develop a framework with
four modules: (i) vocabulary set, (ii) music generator,
(iii) image generator, and (iv) evolutionary engine. We
test our approach by evolving a given music set to a
corresponding set of images, steered by the expression
of four emotions (angry, calm, happy, sad). Then, we
perform preliminary user tests to evaluate if the
user’s perception is consistent with the
system’s expression. Results suggest an agreement
between the user’s emotional perception of the
music–image pairs and the system outcomes,
favoring the integration of cognitive science
knowledge. We also discuss the benefit of employing
evolutionary strategies, such as genetic programming on
multi-modal problems of a creative nature. Overall,
this research contributes to a better understanding of
the foundations of auditory–visual associations
mediated by emotions and perception.",
-
notes = "also known as \cite{e24121706}",
- }
Genetic Programming entries for
Ana Rodrigues
Bruna Sousa
F Amilcar Cardoso
Penousal Machado
Citations