Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: raise error is no partial_fit in hyperparameter search #840

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

stsievert
Copy link
Member

What does this PR implement?
It raises an error if a model is passed to an incremental hyperparameter optimization that doesn't support partial_fit. It points to https://ml.dask.org/hyper-parameter-search.html#hyperparameter-scaling.

It also tests to make sure that a passive search like RandomizedSearchCV works.

Reference issues/PRs
This resolves #839. It doesn't solve the issue, but it makes the error more clear.

dask_ml/model_selection/_incremental.py Outdated Show resolved Hide resolved
"requirement for doing incremental hyperparameter "
"optimization. For more detail, see\n\n"
" https://ml.dask.org/hyper-parameter-search.html#hyperparameter-scaling"
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hooray for informative error messages!

@stsievert
Copy link
Member Author

stsievert commented Jun 1, 2021

In ba8b4fe, I tried to make the header more informative. Now it reads

Hyperparameter optimization becomes difficult with large data and/or complicated searches, and choosing the right tool requires some consideration. All of Dask-ML's hyperparameter optimization focuses on using Scikit-Learn API-compatible models.

I think we should more clearly expose the tradeoffs of hyperparameter searches. Give me a while on that. I'm trying to resolve #838 (comment). I think it wouldn't be a bad idea to have a summary table.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

XGBRegressor does not work with HyperbandSearchCV
2 participants