Static and temporal differences in social signals between error-free and erroneous situations in human-robot collaboration

Cahya, Dito Eka; Ramakrishnan, Rahul; Giuliani, Manuel

Abstract

The capability of differentiating error situations from error-free situations in human-robot collaboration is a mandatory skill for collaborative robots. One of the variables that robots can analyse to differentiate both situations are the social signals from the human interaction partner. We performed an extensive human-robot collaboration user study involving 50 participants in which the robot purposefully executed erroneous behaviours. We annotated the occurrences and the duration of multimodal social signals from the participants during both error-free situations and error situations using an automatic video annotation method based on OpenFace. An analysis of the annotation shows that the participants express more facial expressions, head gestures, and gaze shifts during erroneous situations than in error-free situations. The duration of the facial expressions and gaze shifts is also longer during error situations. Our results additionally show that people look at the robot and the table with a longer duration and look at the objects with a shorter duration in error situations compared to error-free situations. The results of this research are essential for the development of automatic error recognition and error handling in human-robot collaboration.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: International Conference on Social Robotics
• Start Date: Nov 26, 2019
• End Date: Nov 29, 2019
• Acceptance Date: Nov 17, 2019
• Online Publication Date: Nov 17, 2019
• Publication Date: Nov 17, 2019
• Deposit Date: Apr 8, 2020
• Publisher: Springer Verlag
• Pages: 189-199
• Series Title: Lecture Notes in Computer Science
• Series Number: 11876
• Series ISSN: 0302-9743
• Book Title: Social Robotics
• ISBN: 9783030358877
• DOI: https://doi.org/10.1007/978-3-030-35888-4_18
• Public URL: https://uwe-repository.worktribe.com/output/5456488