Home

AOC FRUIT DETECTOR

The FruitDetector module is a part of the larger Agri-OpenCore (AOC) framework and provides tools for training, prediction and visualisation of detecting fruit for harvesting applications.

An overview of components of Fruit Detector module, its input and output.

A. Trained Models

Trained Mask-RCNN models for instance segmentation of Strawberries and Tomatoes. Available pre-trained models are

• Fruit Detector model for Strawberry
  • Models/aoc_strawberry_class_fruit.pth
• Fruit Detector and Ripeness Classifier model for Strawberry
  • Models/aoc_strawberry_class_ripeness.pth
• Fruit Detector model for Tomato
  • Models/aoc_tomato_class_fruit.pth
• Fruit Detector and Ripeness Classifier model for Tomato
  • Models/aoc_tomato_class_ripeness.pth

The datasets employed for training and test are also shared for evaluation purposes.

B. ROS2 Framework Integration

ROS2 integrated framework for fruit detection, meaning

• Images, including colour and depth information, can be subscribed through a topic published by an RGBD camera node.
• Prediction results that contain fruit info, especially on detected fruit locations in terms of bounding boxes, segmentation masks, and 3D pose (position is available, but orientation is in progress), ripeness and size of detected fruits, and some other unique IDs of the detected fruits can be published on ROS2 network.

A ROS2 bag file is also provided to test the integration of the module into ROS2.

C. Training Procedure

An easy-to-use training module to

• train better models for strawberry and tomato crops with larger datasets or
• generate new models for different crop types, e.g. red pepper, asparagus, and apple.

Data preview: Video showing detection of fruits and classification based on ripeness for strawberry use case.