Skip to main content

Research Repository

Advanced Search

Gender and gaze gesture recognition for human-computer interaction

Zhang, Wenhao; Smith, Melvyn L.; Smith, Lyndon N.; Farooq, Abdul

Gender and gaze gesture recognition for human-computer interaction Thumbnail


Wenhao Zhang
Senior Lecturer in Machine Vision

Profile Image

Melvyn Smith
Research Centre Director Vision Lab/Prof

Lyndon Smith
Professor in Computer Simulation and Machine

Abdul Farooq
Associate Head of Departmemt Business Engagement and Partnerships


© 2016 Elsevier Inc. The identification of visual cues in facial images has been widely explored in the broad area of computer vision. However theoretical analyses are often not transformed into widespread assistive Human-Computer Interaction (HCI) systems, due to factors such as inconsistent robustness, low efficiency, large computational expense or strong dependence on complex hardware. We present a novel gender recognition algorithm, a modular eye centre localisation approach and a gaze gesture recognition method, aiming to escalate the intelligence, adaptability and interactivity of HCI systems by combining demographic data (gender) and behavioural data (gaze) to enable development of a range of real-world assistive-technology applications. The gender recognition algorithm utilises Fisher Vectors as facial features which are encoded from low-level local features in facial images. We experimented with four types of low-level features: greyscale values, Local Binary Patterns (LBP), LBP histograms and Scale Invariant Feature Transform (SIFT). The corresponding Fisher Vectors were classified using a linear Support Vector Machine. The algorithm has been tested on the FERET database, the LFW database and the FRGCv2 database, yielding 97.7%, 92.5% and 96.7% accuracy respectively. The eye centre localisation algorithm has a modular approach, following a coarse-to-fine, global-to-regional scheme and utilising isophote and gradient features. A Selective Oriented Gradient filter has been specifically designed to detect and remove strong gradients from eyebrows, eye corners and self-shadows (which sabotage most eye centre localisation methods). The trajectories of the eye centres are then defined as gaze gestures for active HCI. The eye centre localisation algorithm has been compared with 10 other state-of-the-art algorithms with similar functionality and has outperformed them in terms of accuracy while maintaining excellent real-time performance. The above methods have been employed for development of a data recovery system that can be employed for implementation of advanced assistive technology tools. The high accuracy, reliability and real-time performance achieved for attention monitoring, gaze gesture control and recovery of demographic data, can enable the advanced human-robot interaction that is needed for developing systems that can provide assistance with everyday actions, thereby improving the quality of life for the elderly and/or disabled.


Zhang, W., Smith, M. L., Smith, L. N., & Farooq, A. (2016). Gender and gaze gesture recognition for human-computer interaction. Computer Vision and Image Understanding, 149, 32-50.

Journal Article Type Article
Acceptance Date Mar 18, 2016
Online Publication Date Mar 29, 2016
Publication Date Aug 1, 2016
Deposit Date Mar 21, 2016
Publicly Available Date Mar 30, 2017
Journal Computer Vision and Image Understanding
Print ISSN 1077-3142
Electronic ISSN 1090-235X
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 149
Pages 32-50
Keywords Gender; Human computer interaction; Assistive HCI; Gender recognition; Eye centre localisation; Gaze analysis; Directed advertising
Public URL
Publisher URL


You might also like

Downloadable Citations