A corroborative approach to verification and validation of human–robot teams

Webster, Matt; Western, David; Araiza-Illan, Dejanira; Dixon, Clare; Eder, Kerstin; Fisher, Michael; Pipe, Anthony G

Abstract

© The Author(s) 2019. We present an approach for the verification and validation (V&V) of robot assistants in the context of human–robot interactions, to demonstrate their trustworthiness through corroborative evidence of their safety and functional correctness. Key challenges include the complex and unpredictable nature of the real world in which assistant and service robots operate, the limitations on available V&V techniques when used individually, and the consequent lack of confidence in the V&V results. Our approach, called corroborative V&V, addresses these challenges by combining several different V&V techniques; in this paper we use formal verification (model checking), simulation-based testing, and user validation in experiments with a real robot. This combination of approaches allows V&V of the human–robot interaction task at different levels of modeling detail and thoroughness of exploration, thus overcoming the individual limitations of each technique. We demonstrate our approach through a handover task, the most critical part of a complex cooperative manufacturing scenario, for which we propose safety and liveness requirements to verify and validate. Should the resulting V&V evidence present discrepancies, an iterative process between the different V&V techniques takes place until corroboration between the V&V techniques is gained from refining and improving the assets (i.e., system and requirement models) to represent the human–robot interaction task in a more truthful manner. Therefore, corroborative V&V affords a systematic approach to “meta-V&V,” in which different V&V techniques can be used to corroborate and check one another, increasing the level of certainty in the results of V&V.

Citation

Webster, M., Western, D., Araiza-Illan, D., Dixon, C., Eder, K., Fisher, M., & Pipe, A. G. (2020). A corroborative approach to verification and validation of human–robot teams. International Journal of Robotics Research, 39(1), 73-99. https://doi.org/10.1177/0278364919883338

• Journal Article Type: Article
• Acceptance Date: Jul 23, 2019
• Online Publication Date: Nov 25, 2019
• Publication Date: Jan 1, 2020
• Deposit Date: Aug 28, 2019
• Publicly Available Date: Nov 26, 2019
• Journal: International Journal of Robotics Research
• Print ISSN: 0278-3649
• Electronic ISSN: 1741-3176
• Publisher: SAGE Publications
• Peer Reviewed: Peer Reviewed
• Volume: 39
• Issue: 1
• Pages: 73-99
• DOI: https://doi.org/10.1177/0278364919883338
• Keywords: human–robot interaction, verification, validation, model checking, simulation, testing
• Public URL: https://uwe-repository.worktribe.com/output/2525693
• Publisher URL: https://doi.org/10.1177/0278364919883338