Skip to main content

Research Repository

Advanced Search

Test-retest reliability for common tasks in vision science

Clark, Kait; Birch-Hurst, Kayley; Pennington, Charlotte R.; Petrie, Austin C. P.; Lee, Joshua T.; Hedge, Craig

Test-retest reliability for common tasks in vision science Thumbnail


Profile Image

Dr Kait Clark
Senior Lecturer in Psychology (Cognitive and Neuro)

Kayley Birch-Hurst

Charlotte R. Pennington

Austin C. P. Petrie

Joshua T. Lee

Craig Hedge


Research in perception and attention has typically sought to evaluate cognitive mechanisms according to the average response to a manipulation. Recently, there has been a shift toward appreciating the value of individual differences and the insight gained by exploring the impacts of between-participant variation on human cognition. However, a recent study suggests that many robust, well-established cognitive control tasks suffer from surprisingly low levels of test-retest reliability (Hedge, Powell, & Sumner, 2018b). We tested a large sample of undergraduate students (n = 160) in two sessions (separated by 1-3 weeks) on four commonly used tasks in vision science. We implemented measures that spanned a range of perceptual and attentional processes, including motion coherence (MoCo), useful field of view (UFOV), multiple-object tracking (MOT), and visual working memory (VWM). Intraclass correlations ranged from good to poor, suggesting that some task measures are more suitable for assessing individual differences than others. VWM capacity (intraclass correlation coefficient [ICC] = 0.77), MoCo threshold (ICC = 0.60), UFOV middle accuracy (ICC = 0.60), and UFOV outer accuracy (ICC = 0.74) showed good-to-excellent reliability. Other measures, namely the maximum number of items tracked in MOT (ICC = 0.41) and UFOV number accuracy (ICC = 0.48), showed moderate reliability; the MOT threshold (ICC = 0.36) and UFOV inner accuracy (ICC = 0.30) showed poor reliability. In this paper, we present these results alongside a summary of reliabilities estimated previously for other vision science tasks. We then offer useful recommendations for evaluating test-retest reliability when considering a task for use in evaluating individual differences.

Journal Article Type Article
Acceptance Date May 24, 2022
Online Publication Date Jul 29, 2022
Publication Date Jul 29, 2022
Deposit Date Aug 1, 2022
Publicly Available Date Aug 1, 2022
Journal Journal of vision
Electronic ISSN 1534-7362
Publisher Association for Research in Vision and Ophthalmology
Peer Reviewed Peer Reviewed
Volume 22
Issue 8
Pages 1-18
Keywords individual differences; perception; attention; visual cognition
Public URL
Publisher URL


You might also like

Downloadable Citations