Reinforcement learning for a perched landing in the presence of wind

Fletcher, Liam J.; Clarke, Robert J.; Richardson, Thomas S.; Hansen, Mark

Abstract

Previous research by the University of Bristol's Flight Lab demonstrated the feasibility of using reinforcement learning to generate a controller to perform an agile perched landing flight manoeuvre. However, flight testing demonstrated the limits of the agent's ability to generalise to real-world flight conditions, in particular when encountering wind. This work builds on the previous project, adding simulated steady-state wind and turbulence during training. Improvements were made to the reinforcement learning process, such as the use of the more modern Proximal Policy Optimisation (PPO) algorithm, and refinement of the reward function. Using domain randomisation techniques, a series of models were trained in three simulated environments. The performance of each model was assessed in simulation by obtaining the mean reward across a range of conditions. The best performing models from each test case were deployed on the sweep-wing flight test vehicle. An improved flight testing system was developed to allow for a more repeatable testing process, with less variance in manoeuvre start conditions. Flight testing demonstrated that models trained with atmospheric disturbances perform better in the real world, achieving higher mean rewards than the baseline models that were trained without simulated wind. The testing also demonstrated areas of improvement to overcome performance discrepancies between simulation and reality, and improve real-world performance.

Citation

Fletcher, L. J., Clarke, R. J., Richardson, T. S., & Hansen, M. (2021). Reinforcement learning for a perched landing in the presence of wind. . https://doi.org/10.2514/6.2021-1282

• Conference Name: AIAA Scitech 2021 Forum
• Conference Location: Online
• Start Date: Jan 11, 2021
• End Date: Jan 21, 2022
• Online Publication Date: Jan 4, 2021
• Publication Date: Jan 4, 2021
• Deposit Date: Feb 2, 2022
• Publisher: American Institute of Aeronautics and Astronautics
• DOI: https://doi.org/10.2514/6.2021-1282
• Public URL: https://uwe-repository.worktribe.com/output/8809102