Skip to main content

Research Repository

Advanced Search

Sample and time efficient policy learning with CMA-ES and Bayesian Optimisation

Le Goff, Léni K.; Buchanan, Edgar; Hart, Emma; Eiben, Agoston E.; Li, Wei; de Carlo, Matteo; Hale, Matthew F.; Angus, Mike; Woolley, Robert; Timmis, Jon; Winfield, Alan; Tyrrell, Andrew M.

Sample and time efficient policy learning with CMA-ES and Bayesian Optimisation Thumbnail


Authors

Léni K. Le Goff

Edgar Buchanan

Emma Hart

Agoston E. Eiben

Wei Li

Matteo de Carlo

Matt Hale Matt.Hale@uwe.ac.uk
Research Associate - Autonomous Robot Evolution

Mike Angus

Robert Woolley

Jon Timmis

Andrew M. Tyrrell



Contributors

Josh Bongard
Editor

Juniper Lovato
Editor

Laurent Hebert-Dufrésne
Editor

Radhakrishna Dasari
Editor

Lisa Soros
Editor

Abstract

In evolutionary robot systems where morphologies and controllers of real robots are simultaneously evolved, it is clear that there is likely to be requirements to refine the inherited controller of a ‘newborn’ robot in order to better align it to its newly generated morphology. This can be accomplished via a learning mechanism applied to each individual robot: for practical reasons, such a mechanism should be both sample and time-efficient. In this paper, We investigate two ways to improve the sample and time efficiency of the well-known learner CMA-ES on navigation tasks. The first approach combines CMA-ES with Novelty Search, and includes an adaptive restart mechanism with increasing population size. The second bootstraps CMA-ES using Bayesian Optimisation, known for its sample efficiency. Results using two robots built with the ARE project's modules and four environments show that novelty reduces the number of samples needed to converge, as does the custom restart mechanism; the latter also has better sample and time efficiency than the hybridised Bayesian/Evolutionary method.

Citation

Le Goff, L. K., Buchanan, E., Hart, E., Eiben, A. E., Li, W., de Carlo, M., …Tyrrell, A. M. (2020). Sample and time efficient policy learning with CMA-ES and Bayesian Optimisation. In J. Bongard, J. Lovato, L. Hebert-Dufrésne, R. Dasari, & L. Soros (Eds.), Artificial Life Conference Proceedings. , (432-440). https://doi.org/10.1162/isal_a_00299

Conference Name The 2020 Conference on Artificial Life
Start Date Jul 13, 2020
End Date Jul 18, 2020
Acceptance Date Jun 1, 2020
Online Publication Date Jul 31, 2020
Publication Date Jul 31, 2020
Deposit Date Jul 22, 2020
Publicly Available Date Jul 24, 2020
Pages 432-440
Series Title Artificial Life Conference Proceedings
Series Number 32
Book Title Artificial Life Conference Proceedings
DOI https://doi.org/10.1162/isal_a_00299
Public URL https://uwe-repository.worktribe.com/output/6284398
Publisher URL https://www.mitpressjournals.org/doi/abs/10.1162/isal_a_00299

Files







You might also like



Downloadable Citations