New cookbook: Small Object Detection with SAHI

[ediardo]

Description

Small Object Detection with SAHI.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

Jupyter Notebook, Google Colab

Any specific deployment considerations

For example, documentation changes, usability, usage/costs, secrets, etc.

Docs

• Docs updated? What were the changes:

[github-actions]

Preview

Preview and run these notebook edits with Google Colab:

• docs/notebooks/small-object-detection-with-sahi.ipynb

Rendered notebook diffs available on ReviewNB.com. If commits are added to the pull request, synchronize your local branch: git pull origin docs/cookbook-sahi

[ediardo]

Hey everyone, I recently solved a small object detection problem and wanted to share my experience using supervision.

It’s great to see the fixes from @SkalskiP (#1434) that addressed the issue of calculating overlaps correctly, now with overlap_wh. Additionally, I understand there’s a WIP to simplify the calculation of tiles (#1415), which I approached in the following way:

def calculate_tile_size(image_shape: tuple[int, int], tiles: tuple[int, int], overlap_ratio_wh: tuple[float, float] = (0.0, 0.0)):
    """
    Calculate the size of the tiles based on the image shape, the number of tiles, and the overlap ratio.

    Parameters:
    ----------
    image_shape : tuple[int, int]
        The dimensions of the image as (width, height).
    
    tiles : tuple[int, int]
        The tiling strategy defined as (rows, columns), specifying the number of tiles along the height and width of the image.
    
    overlap_ratio_wh : tuple[float, float], optional
        The overlap ratio for width and height as (overlap_ratio_w, overlap_ratio_h). This defines the fraction of overlap between adjacent tiles. Default is (0.0, 0.0), meaning no overlap.

    Returns:
    -------
    tuple[tuple[int, int], tuple[int, int]]
        A tuple containing:
        - The size of each tile as (tile_width, tile_height), accounting for overlap.
        - The overlap dimensions as (overlap_width, overlap_height).

    Example:
    -------
    >>> image_shape = (1024, 768)
    >>> tiles = (4, 4)
    >>> overlap_ratio_wh = (0.15, 0.15)
    >>> calculate_tile_size(image_shape, tiles, overlap_ratio_wh)
    ((281, 230), (39, 29))

    Notes:
    -----
    - The function calculates the width and height of each tile based on the number of rows and columns specified.
    - It then applies the overlap ratio to adjust the tile size, ensuring that tiles overlap by a specified fraction.
    - The overlap dimensions are rounded up to the nearest integer to ensure complete coverage.
    """

    w, h = image_shape
    rows, columns = tiles
    
    tile_width = (w / columns)
    tile_height = (h / rows)
    
    overlap_ratio_w, overlap_ratio_h = overlap_ratio_wh
    overlap_wh = (math.ceil(tile_width * overlap_ratio_w), math.ceil(tile_height * overlap_ratio_h))
    
    tile_width = math.ceil(tile_width + overlap_wh[0])
    tile_height = math.ceil(tile_height + overlap_wh[1])
    tile_size = (tile_width, tile_height)
       
    return tile_size, overlap_wh

In this cookbook, I’m installing supervision from a specific commit on its GitHub repository using the command %pip install git+https://github.com/roboflow/supervision.git@89000c1c33673588e0f13df9624019f60437e924, as it depends on fix #1434 , which has yet to be released in the next version.

[onuralpszr]

Small friendly tip using

%pip install git+https://github.com/roboflow/supervision.git@develop is okay because #1434 already merged.

[onuralpszr]

Maybe adding this blog to top with badge (https://blog.roboflow.com/detect-small-objects/) (since cookbook about InferenceSlicer) instead of keeping down there.

[ediardo]

https://blog.roboflow.com/detect-small-objects/

Great idea and piece of content. Added!

[ediardo]

Small friendly tip using

%pip install git+https://github.com/roboflow/supervision.git@develop is okay because #1434 already merged.

Done .

[onuralpszr]

Hello @ediardo 👋

Here is my initial technical review

• Consolidation of Imports: It would enhance readability and maintainability to move all import statements to the top of the notebook.

• Redundant Installation of Packages: The notebook installs opencv-python and numpy, which are already included in the supervision library. These installations can be omitted to avoid redundancy.

• File Download via wget: Instead of using a custom Python function to download files, it might be simpler and more efficient to use wget. While the Pythonic approach is clear, wget is a straightforward alternative for this task.

• Displaying Step-by-Step Image Processing: The notebook lacks visibility of the image processing steps. Instead of using comments like imwrite, consider using sv.plot_image to visually display each stage of image manipulation.

• Documentation Links: Adding links to the documentation for the key functions and features used in the notebook would be beneficial. Linking to docs would assist users in understanding the purpose and functionality of the code.

• Long Functions: Some functions in the notebook are unnecessarily long. For example, instead of writing a custom function for tiling images, you could utilize sv.plot_images_grid from the supervision library, which is more concise and effective.

• Geospatial Library for Image Comparison: While using leafmap for image comparison (e.g., before/after) is innovative, a lighter alternative might be more suitable. There are Jupyter notebook solutions that support offline mode and offer interactive sliders to show image changes, providing a more engaging user experience.

• Notebook Instability in Google Colab: The notebook has caused crashes during test runs, likely due to the use of leafmap. It would be worthwhile to investigate the root cause and explore lighter alternatives to prevent crashes and improve performance.

I am probably missing points but I hope it helps

[ediardo]

Thank you so much for taking the time to review my pull request so thoroughly. I really appreciate the detailed feedback.

• Consolidation of Imports: It would enhance readability and maintainability to move all import statements to the top of the notebook.

On it.

• Redundant Installation of Packages: The notebook installs opencv-python and numpy, which are already included in the supervision library. These installations can be omitted to avoid redundancy.

TIL.

• File Download via wget: Instead of using a custom Python function to download files, it might be simpler and more efficient to use wget. While the Pythonic approach is clear, wget is a straightforward alternative for this task.

I had a version of download wget. I'll bring it back.

• Displaying Step-by-Step Image Processing: The notebook lacks visibility of the image processing steps. Instead of using comments like imwrite, consider using sv.plot_image to visually display each stage of image manipulation.

Those cv2.imwrite shouldn't really be there.

• Documentation Links: Adding links to the documentation for the key functions and features used in the notebook would be beneficial. Linking to docs would assist users in understanding the purpose and functionality of the code.

Great idea! I’ll add links to the documentation for the key functions and features to help users better understand the code.

• Long Functions: Some functions in the notebook are unnecessarily long. For example, instead of writing a custom function for tiling images, you could utilize sv.plot_images_grid from the supervision library, which is more concise and effective.

Thanks for the suggestion! I’ll look into using sv.plot_images_grid to simplify the image tiling and make the code more concise.

• Geospatial Library for Image Comparison: While using leafmap for image comparison (e.g., before/after) is innovative, a lighter alternative might be more suitable. There are Jupyter notebook solutions that support offline mode and offer interactive sliders to show image changes, providing a more engaging user experience.

Could you please provide some examples? I’m looking forward to learning more about how I could use them.

• Notebook Instability in Google Colab: The notebook has caused crashes during test runs, likely due to the use of leafmap. It would be worthwhile to investigate the root cause and explore lighter alternatives to prevent crashes and improve performance.

As much as I liked the image comparison experience, I think it’s time to look for an alternative way to compare the grids, as I also started experiencing undesired behavior towards the end of writing this cookbook

[ediardo]

hi @onuralpszr ,

I made it leaner:

• Dropped leafmap
• Removed unnecesarry pkgs
• Consolidated imports
• Use sv.plot_image
• Download with wget
• Refactor code

[onuralpszr]

hi @onuralpszr ,

I made it leaner:

• Dropped leafmap
• Removed unnecesarry pkgs
• Consolidated imports
• Use sv.plot_image
• Download with wget
• Refactor code

Thank you very much I will do new review again as well.

[ediardo]

Hello @onuralpszr, I hope you're doing well!

Is there anything else I can do or address to help move the cookbook PR towards merging?

[LinasKo]

Hi @ediardo 👋

This is the nicest cookbook I've ever seen.

Let's address a few things and merge it in:

I'd remove the mention of [assets]. It's great to let people know it exists, but my experience is that most often developers fail to install if not given an exact command.

With a standard release it is pip install "supervision[assets]" (quotes are important), and from a branch it should be

pip install "assets @ git+https://github.com/roboflow/supervision.git@develop" (failing at the moment).

Line #14. results = client.infer(image, model_id=MODEL_ID)

Within InferenceSlicer later we're using get_model - could we use that here too, instead of the HTTP client? This way, there's less cognitive load for the users.

Line #20. COLOR_BBOX_PEOPLE=sv.ColorPalette.DEFAULT.colors[6]
Could we instead use sv.Color.GREEN?

In most of our examples, we use colors directly and don't make a COLOR_BBOX_PEOPLE variable. Up to you if you want to keep it.

4. Let's link to NMM and NMS blog posts an extra time - in the Key Features section.

https://blog.roboflow.com/how-to-code-non-maximum-suppression-nms-in-plain-numpy
https://blog.roboflow.com/non-max-merging

[ediardo]

@onuralpszr, thank you for your contributions :).

@LinasKo: I addressed your comments. Thanks!

[onuralpszr]

@onuralpszr, thank you for your contributions :).

@LinasKo: I addressed your comments. Thanks!

Awesome !! I am checking changes if all good let us merge in as well.

[onuralpszr]

@ediardo 👋 I talked with @LinasKo previously and based on our convo and I did my last changes and merging in. 🚀