Skip to main content

Research Repository

Advanced Search

Assessing the Role of Gaze Tracking in Optimizing Humans-In-The-Loop Telerobotic Operation Using Multimodal Feedback

Bolarinwa, Joseph; Eimontaite, Iveta; Mitchell, Tom; Dogramadzi, Sanja; Caleb-Solly, Praminda

Assessing the Role of Gaze Tracking in Optimizing Humans-In-The-Loop Telerobotic Operation Using Multimodal Feedback Thumbnail


Authors

Joseph Bolarinwa

Tom Mitchell Tom.Mitchell@uwe.ac.uk
Professor of Audio and Music Interaction

Praminda Caleb-Solly



Abstract

A key challenge in achieving effective robot teleoperation is minimizing teleoperators’ cognitive workload and fatigue. We set out to investigate the extent to which gaze tracking data can reveal how teleoperators interact with a system. In this study, we present an analysis of gaze tracking, captured as participants completed a multi-stage task: grasping and emptying the contents of a jar into a container. The task was repeated with different combinations of visual, haptic, and verbal feedback. Our aim was to determine if teleoperation workload can be inferred by combining the gaze duration, fixation count, task completion time, and complexity of robot motion (measured as the sum of robot joint steps) at different stages of the task. Visual information of the robot workspace was captured using four cameras, positioned to capture the robot workspace from different angles. These camera views (aerial, right, eye-level, and left) were displayed through four quadrants (top-left, top-right, bottom-left, and bottom-right quadrants) of participants’ video feedback computer screen, respectively. We found that the gaze duration and the fixation count were highly dependent on the stage of the task and the feedback scenario utilized. The results revealed that combining feedback modalities reduced the cognitive workload (inferred by investigating the correlation between gaze duration, fixation count, task completion time, success or failure of task completion, and robot gripper trajectories), particularly in the task stages that require more precision. There was a significant positive correlation between gaze duration and complexity of robot joint movements. Participants’ gaze outside the areas of interest (distractions) was not influenced by feedback scenarios. A learning effect was observed in the use of the controller for all participants as they repeated the task with different feedback combination scenarios. To design a system for teleoperation, applicable in healthcare, we found that the analysis of teleoperators’ gaze can help understand how teleoperators interact with the system, hence making it possible to develop the system from the teleoperators’ stand point.

Citation

Bolarinwa, J., Eimontaite, I., Mitchell, T., Dogramadzi, S., & Caleb-Solly, P. (2021). Assessing the Role of Gaze Tracking in Optimizing Humans-In-The-Loop Telerobotic Operation Using Multimodal Feedback. Frontiers in Robotics and AI, 8, Article 578596. https://doi.org/10.3389/frobt.2021.578596

Journal Article Type Article
Acceptance Date Aug 5, 2021
Online Publication Date Oct 4, 2021
Publication Date Oct 4, 2021
Deposit Date Oct 6, 2021
Publicly Available Date Oct 6, 2021
Journal Frontiers in Robotics and AI
Electronic ISSN 2296-9144
Publisher Frontiers Media
Peer Reviewed Peer Reviewed
Volume 8
Article Number 578596
DOI https://doi.org/10.3389/frobt.2021.578596
Keywords Artificial Intelligence; Computer Science Applications
Public URL https://uwe-repository.worktribe.com/output/7910369

Files





You might also like



Downloadable Citations