Multi-modality gesture detection and recognition with un-supervision, randomization and discrimination

Wu, Di; Chen, Guang; Clarke, Daniel; Weikersdorfer, David; Giuliani, Manuel; Gaschler, Andre; Knoll, Alois

Abstract

© Springer International Publishing Switzerland 2015. We describe in this paper our gesture detection and recognition system for the 2014 ChaLearn Looking at People (Track 3: Gesture Recognition) organized by ChaLearn in conjunction with the ECCV 2014 conference. The competition’s task was to learn a vacabulary of 20 types of Italian gestures and detect them in sequences. Our system adopts a multi-modality approach for detecting as well as recognizing the gestures. The goal of our approach is to identify semantically meaningful contents from dense sampling spatio-temporal feature space for gesture recognition. To achieve this, we develop three concepts under the random forest framework: un-supervision; discrimination; and randomization. Un-supervision learns spatio-temporal features from two channels (grayscale and depth) of RGB-D video in an unsupervised way. Discrimination extracts the information in dense sampling spatio-temporal space effectively. Randomization explores the dense sampling spatio-temporal feature space efficiently. An evaluation of our approach shows that we achieve a mean Jaccard Index of 0.6489, and a mean average accuracy of 90.3% over the test dataset.

• Presentation Conference Type: Conference Paper (published)
• Conference Name: ChaLearn Looking at People Workshop, European Conference on Computer Vision (ECCV2014)
• Acceptance Date: Sep 6, 2014
• Publication Date: Jan 1, 2015
• Journal: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
• Print ISSN: 0302-9743
• Publisher: Springer Verlag
• Peer Reviewed: Peer Reviewed
• Volume: 8925
• Pages: 608-622
• DOI: https://doi.org/10.1007/978-3-319-16178-5_43
• Keywords: multi-modality gesture, unsupervised learning, random forest, discriminative training
• Public URL: https://uwe-repository.worktribe.com/output/812125
• Publisher URL: http://dx.doi.org/10.1007/978-3-319-16178-5_43
• Additional Information: Title of Conference or Conference Proceedings : ChaLearn Looking at People Workshop, European Conference on Computer Vision (ECCV2014)