Skip to content

perf - coalesce_raw #11926

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 22, 2025
Merged

perf - coalesce_raw #11926

merged 3 commits into from
Jan 22, 2025

Conversation

cscheid
Copy link
Collaborator

@cscheid cscheid commented Jan 21, 2025

With some A/B testing, we find evidence that the code in the PR is faster than that of main (I can share the script later)

$ N_TRIALS=3000 QUARTO_NO_TYPECHECK=1 ./run_experiment_2.py coalesce-raw.csv main tmp-2025-01-21-coalesce-raw quarto render --quiet docs/references/formats/presentations/revealjs.qmd
┏━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━┓
┃ Variant ┃ Count ┃ Mean    ┃ Std Dev   ┃
┡━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━┩
│ A       │ 10    │ 2.51012 │ 0.0106346 │
│ B       │ 10    │ 2.49222 │ 0.0101653 │
└─────────┴───────┴─────────┴───────────┘
{'outcome': 'significant', 'winner': 'B', 'p_value': np.float64(0.0018407964064409569), 'total_samples': 20, 'alpha_spent': 0.05, 'statistics': {'counts': {'A': 10, 'B': 10}, 'means': {'A': 2.510115218162537, 'B': 2.4922211170196533}, 'variances': {'A': 0.00011309452203267512, 'B': 0.00010333396089645674}}}
Statistical Summary:
===================
Variant     Count         Mean      Std Dev
-------- -------- ------------ ------------
A              10     2.510115     0.010635
B              10     2.492221     0.010165

I used docs/references/formats/presentations/revealjs.qmd from quarto-web because I measured that file to be the one to take the longest time on coalesce_raw in a site render.

@cscheid cscheid merged commit 634916c into main Jan 22, 2025
47 checks passed
@cscheid cscheid deleted the tmp-2025-01-21-coalesce-raw branch January 22, 2025 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant