Machine vision and deep learning for robotic harvesting of shiitake mushrooms

Rowland, Thomas; Hansen, Mark; Smith, Melvyn; Smith, Lyndon

Abstract

There is a substantial requirement for the modernisation of our agricultural practices. Current methods can be susceptible to labour shortages, damaging to the environment, and inefficient; and these factors can lead to food wastage so threatening food security. This paper aims to address this issue by employing 2D/3D computer vision and machine learning to assist with robotic harvesting of shiitake mushrooms. Shiitake mushrooms are one of the most valuable gourmet mush-rooms, but also the most labour intensive to grow. The work described demonstrates that state-of-the-art machine vision and deep learning techniques are useful for this application; and are effective at multiple stages throughout the harvest of shiitake mushrooms. Following the creation of the first publicly available segmented shiitake mushroom dataset, YOLOv8-seg and Detectron2 Mask R-CNN models were trained to an Average Precision of 94.9% and 77.7% respectively, segmenting up to 85 shiitake mushrooms in a single image. Additional exploratory work with a smaller keypoint dataset was conducted to determine the suitability of both of these architectures to plot the cut points for the harvest operation, with results proving the feasibility for the task

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 19th International Symposium on Visual Computing
• Start Date: Oct 21, 2024
• End Date: Oct 23, 2024
• Acceptance Date: Aug 26, 2024
• Online Publication Date: Jan 22, 2025
• Publication Date: Jan 22, 2025
• Deposit Date: Oct 28, 2024
• Publicly Available Date: Jan 23, 2026
• Journal: Lecture Notes in Computer Science
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 15047
• Pages: 148–158
• Book Title: Advances in Visual Computing
• ISBN: 9783031773884
• DOI: https://doi.org/10.1007/978-3-031-77389-1_12
• Public URL: https://uwe-repository.worktribe.com/output/13321473
• Additional Information: The paper was accepted for oral presentation and delivered on 23/10/24. The papers are being collated for publication (by the end of November I believe). I'm not sure whether it will be open access or not.