Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: faster cropping; parallel imread/crop; mt_loop and mp_loop helper functions #43

Merged
merged 3 commits into from
Mar 14, 2024

Conversation

yxlao
Copy link
Owner

@yxlao yxlao commented Mar 14, 2024

Summary

This PR focuses on optimizing performance for cropping and reading operations by leveraging parallel processing. The addition of mt_loop and mp_loop functions introduces flexible utilities for applying any operation in parallel, offering significant speed improvements for processing large sets of images or data.

Changes

  1. Performance Enhancements for Image Cropping:
    Improved the image cropping performance by introducing a new version of the ct.image.compute_cropping function for enhanced efficiency.

    import camtools as ct
    image = ct.io.imread("example.png")
    cropping = ct.image.compute_cropping(image)
  2. Parallel Image Reading and Cropping:
    Integrated parallel processing for reading and cropping images, utilizing the newly added mt_loop and mp_loop functions in camtools/utility.py. These functions enable multi-threaded and multi-process execution, respectively, significantly speeding up the operations when working with multiple images.

    # This will be automatically parallelized
    ct crop-boarders --same_crop *.png
  3. Utility Functions for Parallel Loops:
    Added mt_loop for multi-threading and mp_loop for multi-processing in camtools/utility.py, facilitating parallel execution of functions over iterables. These utility functions aim to generalize parallel execution patterns, making it easy to apply any function in a multi-threaded or multi-processed manner across a list of inputs.

    • Multi-threading Example:

      import camtools as ct
      inputs = range(10)
      results = ct.utility.mt_loop(my_function, inputs)
    • Multi-processing Example:

      import camtools as ct
      inputs = range(10)
      results = ct.utility.mp_loop(my_function, inputs)

@yxlao yxlao changed the title perf: parallel imread; mt_loop and mp_loop helper functions perf: faster cropping; parallel imread/crop; mt_loop and mp_loop helper functions Mar 14, 2024
@yxlao yxlao merged commit 7506d36 into master Mar 14, 2024
3 checks passed
@yxlao yxlao deleted the parallel-crop branch March 14, 2024 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant