-
Notifications
You must be signed in to change notification settings - Fork 7
add pytest-benchmark to benchmarking section #596
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
docs/pages/benchmarking-profiling.md
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can also run comparative benchmarks (regression-tests) using pytest-benchmark. We've done this in the glass project
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although we are a little sceptical of their reliability but I think that matches what the linked blog says
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @connoraird - good to know! I guess it is still not as fully featured as asv's comparison though? i.e. it's more geared towards comparing specific runs / subsets of runs, rather than the full project history?
How about something like:
Provides a benchmark fixture with full pytest integration, and the ability to compare performance between multiple runs. A simple, light-weight alternative to asv.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because you said
It was a useful alternative to asv
... feels worth mentioning.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks both - I've updated the text now 👍
samcunliffe
commented
Jan 16, 2026
@all-contributors please add @connoraird for review.
I've put up a pull request to add @connoraird! 🎉
Adds @connoraird as a contributor for review. This was requested by samcunliffe [in this comment](#596 (comment)) --------- Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com> Co-authored-by: Sam Cunliffe <samcunliffe@users.noreply.github.com>
Adds pytest-benchmark as another benchmarking option.
@ruaridhg and I used this recently on the HEFTIE project to benchmark reading / writing ome-zarr images. It was a useful alternative to
asv, as we didn't need to track performance over time (rather just benchmark specific functions / packages).Potentially @connoraird used this on a project recently too (?) - if so, do double check that you agree with my points here. If you ended up going for another benchmarking package, maybe we can add that in too 😄