Using the XCS classifier system for multi-objective reinforcement learning problems

Studley, Matthew; Bull, Larry

Abstract

We investigate the performance of a learning classifier system in some simple multi-objective, multi-step maze problems, using both random and biased action-selection policies for exploration. Results show that the choice of action-selection policy can significantly affect the performance of the system in such environments. Further, this effect is directly related to population size, and we relate this finding to recent theoretical studies of learning classifier systems in single-step problems. © 2006 Massachusetts Institute of Technology.

Citation

Studley, M., & Bull, L. (2007). Using the XCS classifier system for multi-objective reinforcement learning problems. Artificial Life, 13(1), 69-86. https://doi.org/10.1162/artl.2007.13.1.69

• Journal Article Type: Article
• Publication Date: Dec 1, 2007
• Deposit Date: Jan 22, 2010
• Publicly Available Date: Apr 12, 2016
• Journal: Artificial Life
• Print ISSN: 1064-5462
• Electronic ISSN: 1530-9185
• Publisher: Massachusetts Institute of Technology Press (MIT Press)
• Peer Reviewed: Peer Reviewed
• Volume: 13
• Issue: 1
• Pages: 69-86
• DOI: https://doi.org/10.1162/artl.2007.13.1.69
• Keywords: action selection, autonomous agents, genetic algorithm, population size
• Public URL: https://uwe-repository.worktribe.com/output/1030465
• Publisher URL: http://dx.doi.org/10.1162/artl.2007.13.1.69
• Additional Information: Additional Information : XCS is a Learning Classifier System (LCS), in which evolutionary search techniques and reinforcement learning are used to discover general rule-sets which optimally solve a given problem. This paper presents the first published work applying a LCS to problems with more than one goal that must be solved optimally. Examples might be robots which must learn to recharge their batteries, learn to perform useful work, and learn the optimal balance between the two. It is shown that XCS is capable of solving such problems but that its ability to do so is related to its action selection policy. This extends previously published XCS theory. Conclusions from experimental data suggest some design considerations for XCS use on physical robots where resources may be limited. The authors built upon this work to demonstrate the first implementation of a multiple-goal XCS-based system on a real robot.