Research Repository

See what's under the surface

A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth

Bernotas, Gytis; Scorza, Livia; Hansen, Mark; Hales, Ian; Halliday, Karen; Smith, Lyndon; Smith, Melvyn; McCormick:, Alistair


Gytis Bernotas
Agri-Tech Research Scientist - KTP Associate

Livia Scorza

Mark Hansen
Senior Research Fellow - Centre for Machine Vision

Ian Hales

Karen Halliday

Lyndon Smith
Professor in Computer Simulation and Machine

Melvyn Smith
Research Centre Director Vision Lab/Prof

Alistair McCormick:


Background: Tracking and predicting the growth performance of plants in different environments is critical for predicting the impact of global climate change. Automated approaches for image capture and analysis have allowed for substantial increases in the throughput of quantitative growth trait measurements compared to manual assessments. Recent work has focused on adopting computer vision and machine learning approaches to improve the accuracy of automated plant phenotyping. Here we present PS-Plant, a low-cost and portable 3D plant phenotyping platform based on an imaging technique novel to plant phenotyping called photometric stereo (PS).
Results: We calibrated PS-Plant to track the model plant Arabidopsis thaliana throughout the day-night (diel) cycle and investigated growth architecture under a variety of conditions to illustrate the dramatic effect of the environment on plant phenotype. We developed bespoke computer vision algorithms and assessed available deep neural network architectures to automate the segmentation of rosettes and individual leaves, and extract basic and more advanced traits from PS-derived data, including the tracking of 3D plant growth and diel leaf hyponastic movement. Furthermore, we have produced the first PS training data set, which includes 221 manually annotated Arabidopsis rosettes that were used for training and data analysis (1768 images in total). A full protocol is provided, including all software components and an additional test data set.
Conclusions: PS-Plant is a powerful new phenotyping tool for plant research that provides robust data at high temporal and spatial resolutions. The system is well-suited for small and large-scale research and will help to accelerate bridging of the phenotype-to-genotype gap.

Journal Article Type Article
Journal GigaScience
Publisher Oxford University Press (OUP)
Peer Reviewed Peer Reviewed
Institution Citation Bernotas, G., Scorza, L., Hansen, M., Hales, I., Halliday, K., Smith, L., …McCormick:, A. (in press). A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth. GigaScience,
Keywords arabidopsis thaliana, leaf angle, segmentation, machine learning, near-infrared (NIR) LEDs, photomorphogenesis, thermomorphogenesis
Publisher URL


You might also like

Downloadable Citations