Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utils Methods and TIE Speedups #250

Merged
merged 17 commits into from
Aug 4, 2023

Conversation

RHammond2
Copy link
Collaborator

This PR address the remaining, easy-to-address slowness that bogs down the TIE analysis method with Example 3 run time decreasing to ~2.5-3.5 minutes from the previous ~16 minute runtime. The speedups come from:

  1. converting operations relying Pandas Series methods to the underlying NumPy methods and converting the data to NumPy arrays
  2. Converting openoa.utils.filters.bin_filter to array math from nested for loops on top of the improvements from 1

One of the tests was changed because the input data was using mixed data types, but all data being passed through the the various utils methods will be converted to their correct data types, or should by the time they're used in an analysis. In either case mixing string and float data made little sense to test the robustness of a method operating on floats.

@RHammond2 RHammond2 added enhancement v3 All updates and changes for OpenOA v3 labels Jul 18, 2023
@RHammond2 RHammond2 added this to the V3.0.rc3 milestone Jul 18, 2023
@RHammond2 RHammond2 requested a review from ejsimley July 18, 2023 22:42
Copy link
Collaborator

@ejsimley ejsimley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @RHammond2, thanks for making these improvements. I ran the example 03 notebook and the speedup from the new filter implementation is huge! My only request is to run the last few cells of the example 03 notebook so the outputs are already printed when viewing the notebook.

@codecov-commenter
Copy link

codecov-commenter commented Aug 3, 2023

Codecov Report

Patch coverage: 96.66% and project coverage change: +0.01% 🎉

Comparison is base (aad3ede) 63.30% compared to head (6f08565) 63.32%.

Additional details and impacted files
@@              Coverage Diff               @@
##           develop_v3     #250      +/-   ##
==============================================
+ Coverage       63.30%   63.32%   +0.01%     
==============================================
  Files              28       28              
  Lines            3911     3910       -1     
==============================================
  Hits             2476     2476              
+ Misses           1435     1434       -1     
Files Changed Coverage Δ
openoa/utils/filters.py 94.84% <95.83%> (+1.29%) ⬆️
openoa/analysis/turbine_long_term_gross_energy.py 77.23% <100.00%> (-0.50%) ⬇️
openoa/utils/timeseries.py 86.95% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@RHammond2 RHammond2 merged commit c7f3414 into NREL:develop_v3 Aug 4, 2023
2 checks passed
@RHammond2 RHammond2 deleted the enhancement/tie_speedup branch August 4, 2023 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement v3 All updates and changes for OpenOA v3
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants