Skip to main content
SpringerLink
Log in

Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Evolutionary optimization aims to tune the hyper-parameters during learning in a computationally fast manner. For optimization of multi-task problems, evolution is done by creating a unified search space with a dimensionality that can include all the tasks. Multi-task evolution is achieved via selective imitation where two individuals with the same type of skill are encouraged to crossover. Due to the relatedness of the tasks, the resulting offspring may have a skill for a different task. In this way, we can simultaneously evolve a population where different individuals excel in different tasks. In this paper, we consider a type of evolution called Genetic Programming (GP) where the population of genes have a tree-like structure and can be of different lengths and hence can naturally represent multiple tasks. We apply the model to multi-task neuroevolution that aims to determine the optimal hyper-parameters of a neural network such as number of nodes, learning rate, and number of training epochs using evolution. Here each gene is encoded with the hyper parameters for a single neural network. Previously, optimization was done by enabling or disabling individual connections between neurons during evolution. This method is extremely slow and does not generalize well to new neural architectures such as Seq2Seq. To overcome this limitation, we follow a modular approach where each sub-tree in a GP can be a sub-neural architecture that is preserved during crossover across multiple tasks. Lastly, in order to leverage on the inter-task covariance for faster evolutionary search, we project the features from both tasks to common space using fuzzy membership functions. The proposed model is used to determine the optimal topology of a feed-forward neural network for classification of emotions in physiological heart signals and also a Seq2seq chatbot that can converse with kindergarten children. We can outperform baselines by over 10% in accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://github.com/ichaturvedi/fuzzy-aggregated-topology-evolution

References

  1. Cambria E, Hussain A. Sentic album: content-, concept-, and context-based online personal photo management system. Cogn Comput 2012;4:477–496.

    Article  Google Scholar 

  2. Cambria E, Mazzocco T, Hussain A, Eckl C. Sentic medoids: organizing affective common sense knowledge in a multi-dimensional vector space. ISNN; 2011. p. 601–610.

  3. Cambria E, Hussain A, Havasi C, Eckl C. Sentic computing: exploitation of common sense for the development of emotion-sensitive systems. Development of multimodal interfaces: active listening and synchrony; 2010. p. 148–156.

  4. Chaturvedi I, Ong YS, Arumugam RV. Deep transfer learning for classification of time-delayed gaussian networks. Signal Process 2015;110:250–262.

    Article  Google Scholar 

  5. Da B, Gupta A, Ong YS. Curbing negative influences online for seamless transfer evolutionary optimization. IEEE Trans Cybern 2019;49(12):4365–4378.

    Article  Google Scholar 

  6. He T, Liu Y, Ko TH, Chan KCC, Ong YS. Contextual correlation preserving multiview featured graph clustering. IEEE Trans Cybern 2019;50(10):4318–4331.

    Article  Google Scholar 

  7. Zhong J, Feng L, Cai W, Ong YS. Multifactorial genetic programming for symbolic regression problems. IEEE Trans Sys Man Cybern Sys 2020;50(11):4492–4505.

    Article  Google Scholar 

  8. Li H, Ong YS, Gong M, Wang Z. Evolutionary multitasking sparse reconstruction: framework and case study. IEEE Trans Evol Comput 2019;23(5):733–747.

    Article  Google Scholar 

  9. Oneto L, Bisio F, Cambria E, Anguita D. Semi-supervised learning for affective common-sense reasoning. Cogn Comput 2016;9:18–42.

    Article  Google Scholar 

  10. Majumder N, Poria S, Peng H, Chhaya N, Cambria E, Gelbukh A. Sentiment and sarcasm classification with multitask learning. IEEE Intell Syst 2019;34:38–43.

    Article  Google Scholar 

  11. Miikkulainen R. Neuroevolution. New York: Springer; 2010.

    Google Scholar 

  12. Stanley KO, Bryant BD, Miikkulainen R. Real-time neuroevolution in the nero video game. IEEE Trans Evol Comput 2005;9(6):653–668.

    Article  Google Scholar 

  13. Li Y, Pan Q, Yang T, Wang S, Tang J, Cambria E. Learning word representations for sentiment analysis. Cogn Comput 2017;9:843–851.

    Article  Google Scholar 

  14. Chaturvedi I, Cambria E, Cavallari S, Welsch RE. 2020. Genetic Programming for Domain Adaptation in Product Reviews, CEC. art. no. 9185713.

  15. Bonilla EV, Chai KM, Williams C. Multi-task gaussian process prediction. NIPS. In: Platt JC, Koller D, Singer Y, and Roweis ST, editors; 2008. p. 153–160.

  16. Ong YS, Gupta A. Evolutionary multitasking: a computer science view of cognitive multitasking. Cogn Comput 2016;8(2):125–142.

    Article  Google Scholar 

  17. Feng L, Zhou L, Zhong J, Gupta A, Ong YS, Tan K, Qin AK. Evolutionary multitasking via explicit autoencoding. IEEE Trans Cybern 2019;49(9):3457–3470.

    Article  Google Scholar 

  18. Snoek J, Larochelle H, Adams RP. Practical Bayesian optimiza tion of machine learning algorithms. NIPS; 2012. p. 2960–2968.

  19. Richards N, Moriarty DE, Miikkulainen R. Evolving neural networks to play go. Appl Intel 1998;8(1):85–96.

    Article  Google Scholar 

  20. Suganuma M, Shirakawa S, Nagao T. A genetic programming approach to designing convolutional neural network architectures. IJCAI; 2018. p. 5369–5373.

  21. Yao X, Islam MM. Evolving artificial neural network ensembles. IEEE Comput Intel Mag 2008; 3(1):31–42.

    Article  Google Scholar 

  22. Fernandes E, De Carvalho AC, Yao X. Ensemble of classifiers based on multiobjective genetic sampling for imbalanced data. IEEE Trans Knowl Data Eng 2020;32(6):1104–1115.

    Article  Google Scholar 

  23. Susanto Y, Livingstone AG, Ng BC, Cambria E. The hourglass model revisited. IEEE Intelligent Systems 2020;35(5):96–102.

    Article  Google Scholar 

  24. Chaturvedi I, Satapathy R, Cavallari S, Cambria E. Fuzzy commonsense reasoning for multimodal sentiment analysis. Pattern Recogn Lett 2019;125:264–270.

    Article  Google Scholar 

  25. Chung H, Iorga M, Voas J, Lee S. Alexa, can i trust you? Computer 2017;50(9):100–104.

    Article  Google Scholar 

  26. Ghazvininejad M, Brockett C, Chang M-W, Dolan B, Gao J, tau Yih W, Galley M. A knowledge-grounded neural conversation model. AAAI; 2018. p. 5110–5117.

  27. Cambria E, Li Y, Xing FZ, Poria S, Kwok K. 2020. SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis. CIKM, pp. 105–114.

  28. Chen C-Y, Yu D, Wen W, Yang YM, Zhang J, Zhou M, Jesse K, Chau A, Bhowmick A, Iyer S, Sreenivasulu G, Cheng R, Bhandare A, Yu Z. 2018. Gunrock: building a human-like social bot by leveraging large scale real user data.

  29. Young T, Cambria E, Chaturvedi I, Huang M, Zhou H, Biswas S. Augmenting end-to-end dialog systems with commonsense knowledge. AAAI; 2018. p. 4970–4977.

  30. Chaturvedi I, Ragusa E, Gastaldo P, Cambria E. 2020. COAL: Convolutional Online Adaptation Learning for Opinion Mining, ICDM Workshops.

  31. Katsigiannis S, Ramzan N. Dreamer: a database for emotion recognition through eeg and ecg signals from wireless low-cost off-the-shelf devices. IEEE J Biomed Health Info 2018;22(1):98–107.

    Article  Google Scholar 

  32. Miranda Correa JA, Abadi MK, Sebe N, Patras I. 2018. Amigos: a dataset for affect, personality and mood research on individuals and groups. IEEE Trans Affect Comput: 1–1.

  33. Subramanian R, Wache J, Abadi MK, Vieriu RL, Winkler S, Sebe N. Ascertain: emotion and personality recognition using commercial sensors. IEEE Trans Affect Comput 2018;9(2):147–160.

    Article  Google Scholar 

  34. Golberg H, Paradis J, Crago M. Lexical acquisition over time in minority first language children learning english as a second language. Applied Psycholinguistics 2008;29(1):41–65.

    Article  Google Scholar 

Download references

Funding

This work is internally funded by the College of Science and Engineering at Jame Cook University. This work is also partially supported by the Data Science and Artificial Intelligence Center (DSAIR) at the Nanyang Technological University.

Author information

Authors and Affiliations

  1. College of Science and Engineering, James Cook University, Townsville, Australia

    Iti Chaturvedi

  2. SCSE, Nanyang Technological University, Singapore, Singapore

    Chit Lin Su

  3. Sloan School of Management, MIT, Boston, USA

    Roy E. Welsch

Authors
  1. Iti Chaturvedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chit Lin Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roy E. Welsch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Iti Chaturvedi.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chaturvedi, I., Su, C.L. & Welsch, R.E. Fuzzy Aggregated Topology Evolution for Cognitive Multi-tasks. Cogn Comput 13, 96–107 (2021). https://doi.org/10.1007/s12559-020-09807-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-020-09807-4

Keywords

Search

Navigation