How to solve out of memory error while fitting the model?

[prateekgml]

Hi ,

Can you please help me to resolve out of memory error while fitting the model-

[prateekgml]

If I take a fraction of the data for example 50% of data then I am able to proceed but getting the following error now:

So it will be helpful if anybody can tell how to fit the model using whole data and about the above error.

[sonichi]

@prateekgml I tried lighgbm.LGBMClassifier.fit without flaml, it gets OOM error too on the whole dataset. So it's not an FLAML-specific issue. Your dataset is too large to fit in RAM of this VM.
FLAML supports GPU for XGBoost. To use that, please limit the estimator_list to be ["xgboost", "xgb_limitdepth"] and specify gpu_per_trial=1 in AutoML.fit().
BTW, when you use "max_iter=1", it will only train one model. For a general guide about how to set time_budget, check https://microsoft.github.io/FLAML/docs/Use-Cases/Task-Oriented-AutoML#how-to-set-time-budget

[prateekgml]

@sonichi Thanks!

But I can see many individuals' notebooks using catboost or lightgbm or xgboost and they were able to train their models. A few examples are as below-

https://www.kaggle.com/competitions/amex-default-prediction/code

https://www.kaggle.com/code/karandora/xgboost-optuna-baseline

Meanwhile, I will continue with my experiments and let you know if I succeed with FLAML.

[sonichi]

I can't access these notebooks. Have you ever made one successful run with catboost/lightgbm/xgboost in the same environment?

[prateekgml]

@sonichi I have done my experiment using XGBoost directly to the same dataset and the entire training with 5 fold has been done in 12 minutes. You can check 15the version of my notebook that I had shared with you-
https://www.kaggle.com/code/iamprateek/amex-perquet?scriptVersionId=98308716

[sonichi]

In that case, you can use the tune user-defined function API to do hyperparameter search. As long as the training for each configuration can succeed, the tuning would not cause OOM.