Skip to main content

Research Repository

Advanced Search

A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth

Bernotas, Gytis; Scorza, Livia C.T.; Hansen, Mark F.; Hales, Ian J.; Halliday, Karen J.; Smith, Lyndon N.; Smith, Melvyn L.; McCormick, Alistair J.

A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth Thumbnail


Gytis Bernotas
Research Fellow in Computer Vision and Machine Learning

Livia C.T. Scorza

Mark Hansen
Professor of Machine Vision and Machine Learning

Ian J. Hales

Karen J. Halliday

Lyndon Smith
Professor in Computer Simulation and Machine

Profile Image

Melvyn Smith
Research Centre Director Vision Lab/Prof

Alistair J. McCormick


© The Author(s) 2019. Published by Oxford University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. Background: Tracking and predicting the growth performance of plants in different environments is critical for predicting the impact of global climate change. Automated approaches for image capture and analysis have allowed for substantial increases in the throughput of quantitative growth trait measurements compared with manual assessments. Recent work has focused on adopting computer vision and machine learning approaches to improve the accuracy of automated plant phenotyping. Here we present PS-Plant, a low-cost and portable 3D plant phenotyping platform based on an imaging technique novel to plant phenotyping called photometric stereo (PS). Results: We calibrated PS-Plant to track the model plant Arabidopsis thaliana throughout the day-night (diel) cycle and investigated growth architecture under a variety of conditions to illustrate the dramatic effect of the environment on plant phenotype. We developed bespoke computer vision algorithms and assessed available deep neural network architectures to automate the segmentation of rosettes and individual leaves, and extract basic and more advanced traits from PS-derived data, including the tracking of 3D plant growth and diel leaf hyponastic movement. Furthermore, we have produced the first PS training data set, which includes 221 manually annotated Arabidopsis rosettes that were used for training and data analysis (1,768 images in total). A full protocol is provided, including all software components and an additional test data set. Conclusions: PS-Plant is a powerful new phenotyping tool for plant research that provides robust data at high temporal and spatial resolutions. The system is well-suited for small- and large-scale research and will help to accelerate bridging of the phenotype-to-genotype gap.


Bernotas, G., Scorza, L. C., Hansen, M. F., Hales, I. J., Halliday, K. J., Smith, L. N., …McCormick, A. J. (2019). A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth. GigaScience, 8(5), Article giz056.

Journal Article Type Article
Acceptance Date Apr 22, 2019
Online Publication Date May 25, 2019
Publication Date Jan 1, 2019
Deposit Date May 3, 2019
Publicly Available Date May 7, 2019
Journal GigaScience
Electronic ISSN 2047-217X
Publisher Oxford University Press (OUP)
Peer Reviewed Peer Reviewed
Volume 8
Issue 5
Article Number giz056
Keywords arabidopsis thaliana, leaf angle, segmentation, machine learning, near-infrared (NIR) LEDs, photomorphogenesis, thermomorphogenesis
Public URL
Publisher URL


You might also like

Downloadable Citations