Hear here: Sonification as a design strategy for robot teleoperation using virtual reality

Simmons, J; Bown, A; Bremner, P; McIntosh, V; Mitchell, T J

Abstract

This paper introduces a novel methodology for the sonification of data, and shares the results of a usability study, putting the method- ology into practice within an industrial use case. Working with partners at Sellafield nuclear facility, we explore the effectiveness of the sonification strategy in a scenario where operators in a vir- tual reality simulation manage a team of semi-autonomous robots. The robots use onboard sensors to map an unexplored space within a simulated nuclear facility, and the sensor data is sonified, using principles of cognitively aligned metaphor, legibility and user com- fort. Operators use the auditory information in the VR scene to identify key hazards: radiation, temperature and flammable gas. They are tasked with tagging contaminated areas and protecting the robots from harmful exposure to hazards. We offer insight into features that impact user experience and system usability within our scenario. In particular we examine the importance of aligning AI robot behaviours with user expectations, and the implications of alarm fatigue when high priority sounds are repeatedly triggered. We also discuss the potential limitations of the use case, including the suitability of virtual reality environments of this nature for regular, durational use in the workplace. Overall, results of the study suggest that our distinctive approach to sonification as a design strategy for the teleoperation of robots using virtual reality is effective and easily adopted.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: VAM-HRI Virtual, Augmented, and Mixed-Reality for Human-Robot Interactions at HRI
• Start Date: Mar 13, 2023
• End Date: Mar 13, 2023
• Acceptance Date: Feb 26, 2023
• Online Publication Date: Mar 31, 2023
• Publication Date: Mar 31, 2023
• Deposit Date: Apr 20, 2023
• Publicly Available Date: Apr 27, 2023
• Publisher: Association for Computing Machinery (ACM)
• Pages: 1-9
• Keywords: sonification; sound design; virtual reality; robotics; teleoperation; nuclear decommissioning
• Public URL: https://uwe-repository.worktribe.com/output/10636580
• Publisher URL: https://openreview.net/forum?id=1ZPQtSzYAP
• Related Public URLs: https://vam-hri.github.io/