Skip to main content

Research Repository

Advanced Search

The impact of data sonification in virtual reality robot teleoperation

Bremner, Paul; Mitchell, Thomas J; McIntosh, Verity

The impact of data sonification in virtual reality robot teleoperation Thumbnail


Paul Bremner
Associate Professor in Human Robotics Interactions

Tom Mitchell
Professor of Audio and Music Interaction

Profile Image

Verity McIntosh
Associate Professor of Virtual and Extended Realities


Virtual Reality (VR) is being increasingly used to provide a more intuitive and embodied approach to robotic teleoperation, giving operators a sense of presence in the remote environment. Prior research has shown that presence can be enhanced when additional sensory cues such as sound are introduced. Data sonification is the use of non-speech audio to convey information and, in the context of VR robot teleoperation, it has the potential to 1) improve task performance by enhancing an operator’s sense of presence and 2) reduce task load by spreading data between sensory modalities. Here we present a novel study methodology to investigate how the design of data sonification impacts on these important metrics and other key measures of user experience, such as stress. We examine a nuclear decommissioning application of robotic teleoperation where the benefits of VR in terms of spatial reasoning and task performance are desirable. However, as the operational environment is hazardous, a sense of presence may not be desirable as it can lead to heightened operator stress. We conduct a study in which we compare the effects of diegetic sounds (literal and established sonifications) with abstract sounds (non-established sonifications). Our findings show that the diegetic sounds decrease workload, whereas abstract sounds increase workload, and are more stressful. Additionally, and contrary to expectations, sonification does not impact presence. These findings have implications for the design of sonic environments in virtual reality.

Journal Article Type Article
Acceptance Date Jul 7, 2022
Online Publication Date Aug 23, 2022
Publication Date Aug 23, 2022
Deposit Date Aug 24, 2022
Publicly Available Date Aug 24, 2022
Journal Frontiers in Virtual Reality
Print ISSN 2673-4192
Electronic ISSN 2673-4192
Publisher Frontiers Media
Peer Reviewed Peer Reviewed
Volume 3
Pages 904720
Keywords Virtual Reality, virtual reality, teleoperation, sonification, presence, robotics, auditory display, digital twin, human robot interaction
Public URL
Publisher URL
Related Public URLs Supplementary material

The original contributions presented in the study are included in the article/Supplementary Materials, further inquiries can be directed to the corresponding author.

The Supplementary Material for this article can be found online at:

Supplementary Video S1 | Diegetic sonification sound set.

Supplementary Video S2 | Abstract sonification sound set.


The impact of data sonification in virtual reality robot teleoperation (1.7 Mb)


Publisher Licence URL

Copyright Statement
© 2022 Bremner, Mitchell and McIntosh. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

You might also like

Downloadable Citations