perf: faster cropping; parallel imread/crop; mt_loop and mp_loop helper functions #43
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR focuses on optimizing performance for cropping and reading operations by leveraging parallel processing. The addition of
mt_loop
andmp_loop
functions introduces flexible utilities for applying any operation in parallel, offering significant speed improvements for processing large sets of images or data.Changes
Performance Enhancements for Image Cropping:
Improved the image cropping performance by introducing a new version of the
ct.image.compute_cropping
function for enhanced efficiency.Parallel Image Reading and Cropping:
Integrated parallel processing for reading and cropping images, utilizing the newly added
mt_loop
andmp_loop
functions incamtools/utility.py
. These functions enable multi-threaded and multi-process execution, respectively, significantly speeding up the operations when working with multiple images.Utility Functions for Parallel Loops:
Added
mt_loop
for multi-threading andmp_loop
for multi-processing incamtools/utility.py
, facilitating parallel execution of functions over iterables. These utility functions aim to generalize parallel execution patterns, making it easy to apply any function in a multi-threaded or multi-processed manner across a list of inputs.Multi-threading Example:
Multi-processing Example: