-
Notifications
You must be signed in to change notification settings - Fork 56
[CD] Set different time measure for microbench #1915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
13faaf7
to
eda75dd
Compare
3a723b6
to
618e946
Compare
Ondemand test results: https://github.com/intel/torch-xpu-ops/actions/runs/16987046730 |
After rebasing, ondemand test results: https://github.com/intel/torch-xpu-ops/actions/runs/17570650066 |
Hi @RUIJIEZHONG66166 , I have checked the on-demand test summary, there are a lot of regression/improvement cases, does it cause by test variance? Please hide improvement and mixed changes tables by default. And if we run 2 methods, let's mark those regression both on 2 metrics as regression cases |
Updated the comparison method:
For these regression/improvement cases, the root cause may be the baseline commit number has not been updated due to an issue with write permissions in the workflow. I set the baseline to the latest on-demand test results(https://github.com/intel/torch-xpu-ops/actions/runs/17570650066) and did a new round of on-demand testing(https://github.com/intel/torch-xpu-ops/actions/runs/17669054947) compared with 17570650066, which showed no significant regression or improvement. |
Enhance the op microbench test with device, test method and iter number ondemanded control.
disable_ut