Skip to content

Conversation

timkpaine
Copy link
Member

@timkpaine timkpaine commented Jul 13, 2024

xref: #38
Following @trallard's suggestion, exploring using asv to build a benchmark suite, starting with @AdamGlustein's code in #322 (comment).

Example output, running on just my machine through a few commits:

Screenshot 2024-07-13 at 18 50 25 Screenshot 2024-07-13 at 18 50 34

@timkpaine timkpaine added tag: wip PRs that are a work in progress - converted to drafts tag: internal Issues and PRs for maintainance of the project - not interesting to external users type: feature Issues and PRs related to new features labels Jul 13, 2024
@timkpaine timkpaine force-pushed the tkp/bm branch 2 times, most recently from d1969cc to a09002a Compare July 13, 2024 21:44
@@ -104,6 +104,11 @@ csp/lib/
*.so
*.tsbuildinfo

# Benchmarks
.asv
ci/benchmarks/*
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get some standardized machines before putting in raw data

@ptomecek
Copy link
Collaborator

I had missed the original discussion at #322 , that explains where a lot of this came from. Would be good to tidy up in any case. There are a whole bunch of benchmark tests that I want to write after...

@timkpaine timkpaine force-pushed the tkp/bm branch 2 times, most recently from f5b9763 to 8790a83 Compare July 14, 2024 01:04
@timkpaine
Copy link
Member Author

timkpaine commented Jul 14, 2024

Example from other PR:
Screenshot 2024-07-14 at 00 09 53

timkpaine added a commit that referenced this pull request Jul 15, 2024
Signed-off-by: Tim Paine <[email protected]>
@AdamGlustein
Copy link
Collaborator

I had missed the original discussion at #322 , that explains where a lot of this came from. Would be good to tidy up in any case. There are a whole bunch of benchmark tests that I want to write after...

That benchmark script was just something I hacked together, I didn't intend for it to be a canonical example.
If we are making standard benchmarks then we should use a more general framework.

@timkpaine
Copy link
Member Author

If we are making standard benchmarks then we should use a more general framework.

I did a first pass approximation of a more general framework, extnded to the quantile function to demonstrate https://github.com/Point72/csp/blob/fb36b1b6f448d8c39aaab4edfb4b0ba7eed82ed7/csp/benchmarks/stats/basic.py

@timkpaine
Copy link
Member Author

@arhamchopra would be good to see if we can use our benchmark infrastructure to run csp benchmarks as well, and maybe fix some of the ugliness I introduced in this PR if we can.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tag: internal Issues and PRs for maintainance of the project - not interesting to external users tag: wip PRs that are a work in progress - converted to drafts type: feature Issues and PRs related to new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants