Audio-driven robot upper-body motion synthesis

Ondras, Jan; Celiktutan, Oya; Bremner, Paul; Gunes, Hatice

Abstract

Body language is an important aspect of human communication, which an effective human-robot interaction interface should mimic well. Human beings exchange information and convey their thoughts and feelings through gaze, facial expressions, body language, and tone of voice along with spoken words, and infer 65% of the meaning of the communicated messages from these nonverbal cues. Modern robotic platforms are, however, limited in their ability to automatically generate behaviors that align with their speech. In this article, we develop a neural-network-based system that takes audio from a user as an input and generates upper-body gestures, including head, hand, and torso movements of the user on a humanoid robot, namely, Softbank Robotics' Pepper. Our system was evaluated quantitatively as well as qualitatively using Web surveys when driven by natural speech and synthetic speech. We compare the impact of generic and person-specific neural-network models on the quality of synthesized movements. We further investigate the relationships between quantitative and qualitative evaluations and examine how the speaker's personality traits affect the synthesized movements.

• Journal Article Type: Article
• Acceptance Date: Jan 12, 2020
• Online Publication Date: Feb 10, 2020
• Publication Date: Nov 1, 2021
• Deposit Date: Mar 30, 2020
• Journal: IEEE Transactions on Cybernetics
• Print ISSN: 2168-2267
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 51
• Issue: 11
• Pages: 5445-5454
• DOI: https://doi.org/10.1109/tcyb.2020.2966730
• Keywords: Control and Systems Engineering; Human-Computer Interaction; Electrical and Electronic Engineering; Software; Information Systems; Computer Science Applications
• Public URL: https://uwe-repository.worktribe.com/output/5596496