Convolutional neural network ensemble learning for hyperspectral imaging-based blackberry fruit ripeness detection in uncontrolled farm environment

Olisah, Chollette C; Trewhella, Ben; Li, Bo; Smith, Melvyn L; Winstone, Benjamin; Charles Whitfield, E; Fernández, Felicidad Fernández; Duncalfe, Harriet

Abstract

Fruit ripeness estimation models have for decades depended on spectral index features or colour-based features, such as mean, standard deviation, skewness, colour moments, and/or histograms for learning traits of fruit ripeness. Recently, few studies have explored the use of deep learning techniques to extract features from images of fruits with visible ripeness cues. However, the blackberry (Rubus fruticosus) fruit does not show obvious and reliable visible traits of ripeness when mature and therefore poses great difficulty to fruit pickers. The mature blackberry, to the human eye, is black before, during, and post-ripening. To address this problem, this paper proposes a novel multi-input convolutional neural network (CNN) ensemble classifier (MCE) for detecting subtle traits of ripeness in blackberry fruits. The multi-input CNN was created from a pre-trained visual geometry group 16-layer deep convolutional network (VGG16) model trained on the ImageNet dataset. The fully connected layers were optimized for learning traits of ripeness of mature blackberry fruits. The resulting model of about 600 K trainable parameters served as the base for building homogeneous ensemble learners t ensembled using the stack generalization ensemble (SGE) framework. The input to the network are images acquired with a stereo sensor using visible and near-infrared (Vis-NIR) spectral filters at wavelengths of 700 nm and 770 nm. Through experiments, the proposed model achieved 95.1 % accuracy on unseen sets and 90.2 % accuracy with in-field conditions. Further experiments reveal that machine sensory is highly and positively correlated to human sensory over blackberry fruit skin texture

• Journal Article Type: Article
• Acceptance Date: Jan 18, 2024
• Online Publication Date: Feb 6, 2024
• Publication Date: Jun 30, 2024
• Deposit Date: Feb 6, 2024
• Publicly Available Date: Feb 6, 2024
• Journal: Engineering Applications of Artificial Intelligence
• Print ISSN: 0952-1976
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 132
• Article Number: 107945
• DOI: https://doi.org/10.1016/j.engappai.2024.107945
• Keywords: blackberry ripeness; convolutional neural network; hyperspectral imaging; homogeneous ensemble learning; stack generalization ensemble; uncontrolled farm environment 2
• Public URL: https://uwe-repository.worktribe.com/output/11669512