Learning a robot controller using an adaptive hierarchical fuzzy rule-based system

Waldock, Antony; Carse, Brian

Abstract

© 2015, Springer-Verlag Berlin Heidelberg. The majority of machine learning techniques applied to learning a robot controller generalise over either a uniform or pre-defined representation that is selected by a human designer. The approach taken in this paper is to reduce the reliance on the human designer by adapting the representation to improve the generalisation during the learning process. An extension of a Hierarchical Fuzzy Rule-Based System (HFRBS) is proposed that identifies and refines inaccurate regions of a fuzzy controller, while interacting with the environment, for both supervised and reinforcement learning problems. The paper shows that a controller using an adaptive HFRBS can learn a suitable control policy using a fewer number of fuzzy rules for both a supervised and reinforcement learning problem and is not sensitive to the layout as with a uniform representation. In supervised learning problems, a small number of extra trials are required to find an effective representation but for reinforcement learning problems, the process of adapting the representation is shown to significantly reduce the time taken to learn a suitable control policy and hence open the door to high-dimensional problems.

• Journal Article Type: Article
• Acceptance Date: Apr 1, 2015
• Online Publication Date: Apr 22, 2015
• Publication Date: Jul 1, 2016
• Journal: Soft Computing
• Print ISSN: 1432-7643
• Electronic ISSN: 1433-7479
• Publisher: Springer (part of Springer Nature)
• Peer Reviewed: Peer Reviewed
• Volume: 20
• Issue: 7
• Pages: 2855-2881
• DOI: https://doi.org/10.1007/s00500-015-1688-3
• Keywords: fuzzy systems, reinforcement learning, robotics
• Public URL: https://uwe-repository.worktribe.com/output/924181
• Publisher URL: http://dx.doi.org/10.1007/s00500-015-1688-3