perf: mechanical Python-level optimizations — 34% faster simulation#221
perf: mechanical Python-level optimizations — 34% faster simulation#221rcbevans wants to merge 1 commit intoNatLabRockies:devfrom
Conversation
…ulation
Eliminate Python overhead in the hot simulation loop through mechanical,
behavior-preserving replacements. No architecture or physics changes.
Benchmark: 30-day simulation at 1-minute timesteps (43,560 steps) using
BEopt_example.xml + BEopt_example_schedule.csv + Denver TMY3 weather,
starting June 1 2018, with PV (5kW), Battery (6kWh/3kW), and EV (BEV
Level 2). Deterministic via seed=42.
Before: 37.67s → After: 24.86s (34.0% faster, 12.81s saved)
Function calls: 74.3M → 44.6M (40% reduction)
All 244 tests pass. Output identical within tolerances.
Changes by category:
Schedule/dispatch overhead:
- Cache DataFrame.empty as bool flag, recomputed in reset_time()
- Replace to_dict("records") iterator with array-backed dict construction
- Reuse schedule dict in-place via clear() instead of allocating new dict each step
- Replace isinstance() checks with bit-flag dispatch on Equipment._kind
- Skip per-sub datetime comparisons when all sub-simulators share resolution
- Convert all_schedule_inputs membership test from list to frozenset
Unit conversion and constant caching:
- Cache pint convert() results as module-level constants for natural ventilation
- Also fixes natural ventilation area unit: ft^2 → m^2 (pre-existing bug)
- Pre-compute f-string schedule keys at init instead of per-step
- Pre-format zone temperature schedule keys at Dwelling init
- Cache time_res.total_seconds() as _dt_seconds/_dt_hours/_dt_minutes at init
(eliminates repeated timedelta method calls across Simulator, Water, Humidity,
WaterHeater, Generator, and EV hot paths)
Numpy small-array allocation elimination:
- Replace np.insert() with pre-allocated buffer in Water.update_model
- Replace np.array() + np.dot() biquadratic with inline scalar polynomial
- Pre-allocate np.zeros buffers in Envelope.update_inputs/update_model
- Pre-allocate surface temperature buffers for interior radiation
- Pre-allocate np.zeros/np.concatenate buffers in Water model
- Skip inputs_init.copy() in SSM when full control_signal provided
Scalar math dispatch:
- Python sum()/any() on numpy arrays → ndarray .sum()/.any()
- np.exp/np.sqrt on scalars → math.exp/math.sqrt
- Python min()/max() on numpy arrays → .min()/.max() (including Water.update_results)
- Precompute vol_fractions cumulative sums in Water model
- Replace list-wrapped any([...]) and sum([...]) with generator expressions
across Dwelling, HVAC, WaterHeater, Envelope, and Simulator
Lookup optimization:
- Dict index maps for StateSpaceModel input/output/state name lookups
- Scalar fast path for RCModel.solve_for_input (common infiltration case)
- Cache np.nonzero(~disable_speeds) in HVAC
- Pre-compute Generator._ramp_minutes at init
Bug fix:
- Fix Battery.py typo: "Zone Temperture (C)" → "Zone Temperature (C)"
(battery thermal model was writing to wrong schedule key, so its Water
sub-model never received correct zone temperature from Dwelling)
Tests:
- Add tests for scalar solver equivalence and nat-vent unit conversion
9d15e62 to
2258677
Compare
|
This is awesome, thanks Rich! It's going to take us a little bit to look through this, make sure we're comfortable with everything (and taking on the maintenance burden of any new methods). If you find issues with the physics or other bugs please do keep calling them out, but for speed I think give us a little time to digest here before going too much farther down the optimizing path. We don't normally have such productive collaborators! |
I have a second branch which would go into this one which refactored core formulas to use numba (again, it matches a golden parque so no behavioral deviation). https://github.com/AZX-PBC-OSS/OCHRE/tree/perf/tier1-numba-optimizations This reduces the 30 day simulation further, down to ~19s from the original 37.67 in the dev branch. |
| # Add zone temperature to schedule inputs for water tank | ||
| if not self.main_simulator and self.thermal_model: | ||
| schedule_inputs["Zone Temperture (C)"] = schedule_inputs[f"{self.zone_name} Temperature (C)"] | ||
| schedule_inputs["Zone Temperature (C)"] = schedule_inputs[f"{self.zone_name} Temperature (C)"] |
| run_nat_vent = (w_amb < max_oa_hr) and (t_zone > t_ext) and (t_zone > t_base) | ||
| if run_nat_vent and self.open_window_area is not None: | ||
| area = self.open_window_area * 0.6 | ||
| nat_vent_area = convert(area, "ft^2", "cm^2") |
Summary
Mechanical, behavior-preserving Python-level optimizations that eliminate overhead in the hot simulation loop. No architecture or physics changes. All existing tests pass. Simulation output is numerically identical to baseline within defined tolerances.
Benchmark setup: 30-day simulation at 1-minute timesteps (43,560 steps) using
BEopt_example.xml+BEopt_example_schedule.csv+ Denver TMY3 weather (USA_CO_Denver.Intl.AP.725650_TMY3.epw), starting June 1 2018, with PV (5 kW), Battery (6 kWh / 3 kW, self-consumption), and EV (BEV, Level 2, 150 mi range). Deterministic viaseed=42.Results:
atol=0.01, rtol=1e-4; power:atol=0.001, rtol=1e-3)Changes
Schedule/dispatch overhead (~5s saved)
DataFrame.emptyas bool flag, recomputed inreset_time()to_dict("records")iterator with array-backed dict constructionclear()instead of allocating new dict each stepisinstance()checks with bit-flag dispatch onEquipment._kindall_schedule_inputsmembership test from list to frozensetUnit conversion and constant caching (~4.5s saved)
pint convert()results as module-level constants for natural ventilationft^2→m^2(pre-existing bug —open_window_areais computed in m² but was being converted as ft²)Dwellinginittime_res.total_seconds()as_dt_seconds/_dt_hours/_dt_minutesat init (eliminates repeatedtimedeltamethod calls acrossSimulator,Water,Humidity,WaterHeater,Generator, andEVhot paths)Numpy small-array allocation elimination (~2s saved)
np.insert()with pre-allocated buffer inWater.update_modelnp.array()+np.dot()biquadratic with inline scalar polynomial (HVAC, GasBoiler, HPWH)np.zerosbuffers inEnvelope.update_inputs/update_modelnp.zeros/np.concatenatebuffers in Water modelinputs_init.copy()inStateSpaceModelwhen full control_signal providedScalar math dispatch (~1s saved)
sum()/any()on numpy arrays →.sum()/.any()np.exp/np.sqrton scalars →math.exp/math.sqrtmin()/max()on numpy arrays →.min()/.max()(includingWater.update_results)vol_fractionscumulative sums in Water modelany([...])andsum([...])with generator expressions acrossDwelling,HVAC,WaterHeater,Envelope, andSimulatorLookup optimization (~1s saved)
StateSpaceModelinput/output/state name lookupsRCModel.solve_for_input(common single-input infiltration case)np.nonzero(~disable_speeds)in HVACGenerator._ramp_minutesat initBug fix
Battery.pytypo:"Zone Temperture (C)"→"Zone Temperature (C)"— battery thermal model was writing to wrong schedule key, so itsWatersub-model never received correct zone temperature fromDwellingCorrectness
Every change is a mechanical replacement that produces identical numerical results:
sum(arr)→arr.sum()— same valuesnp.exp(scalar)→math.exp(scalar)— same valuesnp.array([1, x, x**2]) @ coeffs→coeffs[0] + coeffs[1]*x + coeffs[2]*x**2— same valuespint.convert()— same conversion factorsisinstance(x, Class)→x._kind & FLAG— same dispatch decisionsclear()+ repopulate — same key/value pairs each stepany([genexpr])→any(genexpr)— same boolean result, avoids intermediate listTest plan
uv run python -m unittest discover test— 244 pass, 1 skipped (pre-existing)--comparemode)seed=42confirms output stability across runstest_rcmodel.py) and nat-vent unit conversion (test_envelope.py)