Towards multimodal interactions: Robot jogging in mixed reality

Sita, Enrico; Studley, Matthew; Dailami, Farid; Pipe, Anthony G; Thomessen, Trygve

Abstract

© 2017 Copyright held by the owner/author(s). The recent progress made in the field of Augmented Reality/Mixed Reality (AR/MR) has opened new possibilities and approaches to research areas that can benefit from 3D visualization of digital content in the real world.In fact, human-robot interaction design and the design of user interfaces have very much to gain from MR technologies. Nonetheless, designing the user-robot interaction and processing multimodal feedbacks are very challenging tasks. In this paper we focus in particular on interactions in mixed reality. The main contribution of this paper is the implementation of a control system for an industrial manipulator through the user's interactions with MR content displayed with the Microsoft HoloLens. The system is based on the communication between Unity3D (used to design the user experience) and ROS, therefore extendible to any ROS-compatible robotic hardware.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST
• Start Date: Nov 8, 2017
• End Date: Nov 10, 2017
• Acceptance Date: Sep 14, 2017
• Publication Date: Nov 8, 2017
• Deposit Date: Nov 13, 2017
• Publicly Available Date: Nov 13, 2017
• Peer Reviewed: Not Peer Reviewed
• Volume: Part F131944
• Pages: 1-2
• Book Title: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology - VRST '17
• ISBN: 9781450355483
• DOI: https://doi.org/10.1145/3139131.3141200
• Keywords: mixed reality, unity, robotics, industrial robot
• Public URL: https://uwe-repository.worktribe.com/output/881134
• Publisher URL: http://dx.doi.org/10.1145/3139131.3141200
• Additional Information: Title of Conference or Conference Proceedings : 23rd ACM Symposium on Virtual Reality Software and Technology
• Contract Date: Nov 13, 2017