-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Utils Methods and TIE Speedups #250
Conversation
…lid numerical data
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @RHammond2, thanks for making these improvements. I ran the example 03 notebook and the speedup from the new filter implementation is huge! My only request is to run the last few cells of the example 03 notebook so the outputs are already printed when viewing the notebook.
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## develop_v3 #250 +/- ##
==============================================
+ Coverage 63.30% 63.32% +0.01%
==============================================
Files 28 28
Lines 3911 3910 -1
==============================================
Hits 2476 2476
+ Misses 1435 1434 -1
☔ View full report in Codecov by Sentry. |
This PR address the remaining, easy-to-address slowness that bogs down the TIE analysis method with Example 3 run time decreasing to ~2.5-3.5 minutes from the previous ~16 minute runtime. The speedups come from:
Series
methods to the underlying NumPy methods and converting the data to NumPy arraysopenoa.utils.filters.bin_filter
to array math from nestedfor
loops on top of the improvements from 1One of the tests was changed because the input data was using mixed data types, but all data being passed through the the various
utils
methods will be converted to their correct data types, or should by the time they're used in an analysis. In either case mixing string and float data made little sense to test the robustness of a method operating on floats.