Gender and gaze gesture recognition for human-computer interaction

Zhang, Wenhao; Smith, Melvyn L.; Smith, Lyndon N.; Farooq, Abdul

Abstract

© 2016 Elsevier Inc. The identification of visual cues in facial images has been widely explored in the broad area of computer vision. However theoretical analyses are often not transformed into widespread assistive Human-Computer Interaction (HCI) systems, due to factors such as inconsistent robustness, low efficiency, large computational expense or strong dependence on complex hardware. We present a novel gender recognition algorithm, a modular eye centre localisation approach and a gaze gesture recognition method, aiming to escalate the intelligence, adaptability and interactivity of HCI systems by combining demographic data (gender) and behavioural data (gaze) to enable development of a range of real-world assistive-technology applications. The gender recognition algorithm utilises Fisher Vectors as facial features which are encoded from low-level local features in facial images. We experimented with four types of low-level features: greyscale values, Local Binary Patterns (LBP), LBP histograms and Scale Invariant Feature Transform (SIFT). The corresponding Fisher Vectors were classified using a linear Support Vector Machine. The algorithm has been tested on the FERET database, the LFW database and the FRGCv2 database, yielding 97.7%, 92.5% and 96.7% accuracy respectively. The eye centre localisation algorithm has a modular approach, following a coarse-to-fine, global-to-regional scheme and utilising isophote and gradient features. A Selective Oriented Gradient filter has been specifically designed to detect and remove strong gradients from eyebrows, eye corners and self-shadows (which sabotage most eye centre localisation methods). The trajectories of the eye centres are then defined as gaze gestures for active HCI. The eye centre localisation algorithm has been compared with 10 other state-of-the-art algorithms with similar functionality and has outperformed them in terms of accuracy while maintaining excellent real-time performance. The above methods have been employed for development of a data recovery system that can be employed for implementation of advanced assistive technology tools. The high accuracy, reliability and real-time performance achieved for attention monitoring, gaze gesture control and recovery of demographic data, can enable the advanced human-robot interaction that is needed for developing systems that can provide assistance with everyday actions, thereby improving the quality of life for the elderly and/or disabled.

• Journal Article Type: Article
• Acceptance Date: Mar 18, 2016
• Online Publication Date: Mar 29, 2016
• Publication Date: Aug 1, 2016
• Deposit Date: Mar 21, 2016
• Publicly Available Date: Mar 29, 2017
• Journal: Computer Vision and Image Understanding
• Print ISSN: 1077-3142
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 149
• Pages: 32-50
• DOI: https://doi.org/10.1016/j.cviu.2016.03.014
• Keywords: Gender; Human computer interaction; Assistive HCI; Gender recognition; Eye centre localisation; Gaze analysis; Directed advertising
• Public URL: https://uwe-repository.worktribe.com/output/924791
• Publisher URL: http://dx.doi.org/10.1016/j.cviu.2016.03.014
• Contract Date: Mar 21, 2016