N-dimensional interpolation

[kylecarow]

Issue

https://github.nrel.gov/MBAP/fastsim/issues/328

Completed

• investigate dimensionality reduction
  • removing dimensionality reduction for hardcoded stuff sped it up, but multilinear still benefits from it, probably due to the 'views' that ndarray employs for this
• investigate .windows() vs. binary search for finding indices
  • on a benchmark of 100x100x100 3d data (using InterpND) the binary search from RouteE and the .windows search perform almost exactly the same, so I'll keep it as is. This is easy to test by placing the index finding code into a function and swapping out the function body

Fairly Urgent

• get rid of standalone interp1d everywhere and replace with Interpolator

Not Urgent but Important

• put PhantomData in pretty much every named field struct and maybe also the tuple structs to prevent init bypassing
• manually generate jsonized getters and setters for Interpolator and maybe other enums
• automatically generate jsonized getters and setters for Interpolator and maybe other enums, making use of FieldOptions

[kylecarow]

@nreinicke made some good points:

• the dimensionality lowering code does cloning to create a lower dim interpolator that also owns the data
  • I think this dimensionality lowering stuff could be unnecessary, the performance when passing a grid point value is probably not that bad compared to cloning
• binary search instead of using .windows may be faster

EDIT: resolved, see top comment

[kylecarow]

I want to switch to binary search rather than .windows and move the other unwrap rewrites as I did in compass

[kylecarow]

@calbaker This is ready to review once tests pass. I'll add benchmarks for the 1 & 2 dimensional cases later on in fastsim-3 I added these in 1412dc2.

I'd appreciate your review on the structure and usage that I've designed here, along with any other thoughts you have!

[kylecarow]

@calbaker Maybe we can review this in the FASTSim development meeting today

[kylecarow]

Converted to draft so I can implement interpolation throughout FASTSim before merging

[kylecarow]

@calbaker Uh oh

https://pyo3.rs/v0.21.2/class.html?highlight=enum#complex-enums

An enum is complex if it has any non-unit (struct or tuple) variants.

Currently PyO3 supports only struct variants in a complex enum. Support for unit and tuple variants is planned.

What I have written is an enum with tuple variants that contain structs (as there is no way to impl on enum struct variants). This will take some thinking to solve.

[kylecarow]

@robinsteuteville

• We need ways to interact with the data within the Interpolator enum (e.g. x, y, ... f(x, y, ...)) from Python, like setting x-values or something from Python
• PyO3 doesn't support enums with tuple variants, so replacing component efficiencies with the new Interpolator enum will not work out of the box
• It can work by doing something like this:

  #[api(skip_get, skip_set)]
  pub eff_interp: utils::interp::Interpolator,

  But then we can't access the variable at all from Python (because we skip adding the getter and setter with the attribute), and we would not be able to access any of its methods (which is where we would have the stuff to achieve the first bullet above)
  • This is what you see when trying to get via sd.veh.fc.eff_interp: AttributeError: 'fastsim.FuelConverter' object has no attribute 'eff_interp'
• I made a 'wrapper' struct that can be seen from Python without issue, as is in the latest commit in this PR:

  pub eff_interp: utils::interp::InterpolatorWrapper,

  Getting this from Python works sd.veh.fc.eff_interp: <InterpolatorWrapper at 0x176dc26b0>
  • We can make the methods to get x-values etc. on this (probably in a #[pyo3_api(...)] block, no need to have Rust-side methods when Rust users can edit the data directly)

We also need to fix the tests that fail (expecting the old struct format)

[calbaker]

• finish F3/interp with benchmarking #146 when this is done