Skip to main content

Research Repository

Advanced Search

The impact of data sonification in virtual reality robot teleoperation

Bremner, Paul; Mitchell, Thomas J; McIntosh, Verity


Paul Bremner
Associate Professor in Human Robotics Interactions

Tom Mitchell
Associate Professor in Creative Technologies

Profile Image

Verity McIntosh
Senior Lecturer - MA Virtual & Extended Realities


Virtual Reality (VR) is being increasingly used to provide a more intuitive and embodied approach to robotic teleoperation, giving operators a sense of presence in the remote environment. Prior research has shown that presence can be enhanced when additional sensory cues such as sound are introduced. Data sonification is the use of non-speech audio to convey information and, in the context of VR robot teleoperation, it has the potential to 1) improve task performance by enhancing an operator’s sense of presence and 2) reduce task load by spreading data between sensory modalities. Here we present a novel study methodology to investigate how the design of data sonification impacts on these important metrics and other key measures of user experience, such as stress. We examine a nuclear decommissioning application of robotic teleoperation where the benefits of VR in terms of spatial reasoning and task performance are desirable. However, as the operational environment is hazardous, a sense of presence may not be desirable as it can lead to heightened operator stress. We conduct a study in which we compare the effects of diegetic sounds (literal and established sonifications) with abstract sounds (non-established sonifications). Our findings show that the diegetic sounds decrease workload, whereas abstract sounds increase workload, and are more stressful. Additionally, and contrary to expectations, sonification does not impact presence. These findings have implications for the design of sonic environments in virtual reality.


Bremner, P., Mitchell, T. J., & McIntosh, V. (2022). The impact of data sonification in virtual reality robot teleoperation. Frontiers in Virtual Reality, 3, 904720.

Journal Article Type Article
Acceptance Date Jul 7, 2022
Online Publication Date Aug 23, 2022
Publication Date Aug 23, 2022
Deposit Date Aug 24, 2022
Publicly Available Date Aug 24, 2022
Journal Frontiers in Virtual Reality
Print ISSN 2673-4192
Electronic ISSN 2673-4192
Publisher Frontiers Media
Peer Reviewed Peer Reviewed
Volume 3
Pages 904720
Keywords Virtual Reality, virtual reality, teleoperation, sonification, presence, robotics, auditory display, digital twin, human robot interaction
Public URL
Publisher URL
Related Public URLs Supplementary material

The original contributions presented in the study are included in the article/Supplementary Materials, further inquiries can be directed to the corresponding author.

The Supplementary Material for this article can be found online at:

Supplementary Video S1 | Diegetic sonification sound set.

Supplementary Video S2 | Abstract sonification sound set.


You might also like

Downloadable Citations