GestureChords: Transparency in gesturally controlled digital musical instruments through iconicity and conceptual metaphor

Brown, Dom; Nash, Chris; Mitchell, Tom

Abstract

This paper presents GestureChords, a mapping strategy for chord selection in freehand gestural instruments. The strategy maps chord variations to a series of hand postures using the concepts of iconicity and conceptual metaphor, influenced by their use in American Sign Language (ASL), to encode meaning in gestural signs. The mapping uses the conceptual metaphors MUSICAL NOTES ARE POINTS IN SPACE and INTERVALS BETWEEN NOTES ARE SPACES BETWEEN POINTS, which are mapped respectively to the number of extended fingers in a performer’s hand and the abduction or adduction between them. The strategy is incorporated into a digital musical instrument and tested in a preliminary study for transparency by both performers and spectators, which gave promising results for the technique.

• Journal Article Type: Article
• Conference Name: Sound and Music Computing
• Start Date: Aug 30, 2016
• End Date: Sep 3, 2016
• Acceptance Date: Jun 24, 2016
• Publication Date: Jan 1, 2016
• Deposit Date: Aug 3, 2016
• Publicly Available Date: Sep 8, 2016
• Journal: Proceedings SMC 2016
• Print ISSN: 2518-3672
• Peer Reviewed: Peer Reviewed
• Pages: 85-92
• Keywords: gestural interaction, digital musical instrument, American Sign Language, iconicity, conceptual metaphor
• Public URL: https://uwe-repository.worktribe.com/output/917262
• Publisher URL: http://www.georghajdu.de/computermusicnotation/wopre/wp-content/uploads/smc/SMC2016_proceedings_final-2.pdf
• Additional Information: Title of Conference or Conference Proceedings : The Proceedings of the 13th Sound and Music Computing Conference
• Contract Date: Aug 3, 2016