Mary Ellen Foster
Task-based evaluation of context-sensitive referring expressions in human–robot dialogue
Foster, Mary Ellen; Giuliani, Manuel; Isard, Amy
Authors
Abstract
© 2013 Taylor & Francis. The standard referring-expression generation task involves creating stand-alone descriptions intended solely to distinguish a target object from its context. However, when an artificial system refers to objects in the course of interactive, embodied dialogue with a human partner, this is a very different setting; the references found in situated dialogue are able to take into account the aspects of the physical, interactive and task-level context, and are therefore unlike those found in corpora of standalone references. Also, the dominant method of evaluating generated references involves measuring corpus similarity. In an interactive context, though, other extrinsic measures such as task success and user preference are more relevant –and numerous studies have repeatedly found little or no correlation between such extrinsic metrics and the predictions of commonly used corpus-similarity metrics. To explore these issues, we introduce a humanoid robot designed to cooperate with a human partner on a joint construction task. We then describe the context-sensitive reference-generation algorithm that was implemented for use on this robot, which was inspired by the referring phenomena found in the Joint Construction Task corpus of human–human joint construction dialogues. The context-sensitive algorithm was evaluated through two user studies comparing it to a baseline algorithm, using a combination of objective performance measures and subjective user satisfaction scores. In both studies, the objective task performance and dialogue quality were found to be the same for both versions of the system; however, in both cases, the context-sensitive system scored more highly on subjective measures of interaction quality.
Citation
Foster, M. E., Giuliani, M., & Isard, A. (2014). Task-based evaluation of context-sensitive referring expressions in human–robot dialogue. Language Cognition and Neuroscience, 29(8), 1018-1034. https://doi.org/10.1080/01690965.2013.855802
Journal Article Type | Article |
---|---|
Acceptance Date | Oct 2, 2013 |
Publication Date | Jan 1, 2014 |
Journal | Language, Cognition and Neuroscience |
Print ISSN | 2327-3798 |
Electronic ISSN | 2327-3801 |
Publisher | Taylor and Francis |
Peer Reviewed | Peer Reviewed |
Volume | 29 |
Issue | 8 |
Pages | 1018-1034 |
DOI | https://doi.org/10.1080/01690965.2013.855802 |
Keywords | referring expressions in interactive settings, task-based evaluation, human–robot dialogue |
Public URL | https://uwe-repository.worktribe.com/output/926581 |
Publisher URL | http://dx.doi.org/10.1080/01690965.2013.855802 |
Related Public URLs | http://www.tandfonline.com/doi/abs/10.1080/01690965.2013.855802 |
You might also like
Usability study of a novel triple-arm mixed-reality robot teleoperation system
(2023)
Conference Proceeding
Measuring visual social engagement from proxemics and gaze
(2022)
Conference Proceeding
User requirements for a robot teleoperation system for general medical examination
(2022)
Conference Proceeding
Demo: Untethered haptic teleoperation for nuclear decommissioning using a low-power wireless control technology
(2022)
Conference Proceeding
Downloadable Citations
About UWE Bristol Research Repository
Administrator e-mail: repository@uwe.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search